FastAPI won't deploy: 502 and 'Container Died' errors

saginawjHOBBY

7 months ago

I'm having issues deploying a FastAPI app, which is strange since I've deployed several on Railway before.

No build or deploy log errors, I see the app starting in the logs, but every request results in a 502, and the logs show 'Stopping Container. Container Event Container Died.' but no other info.

App runs fine locally, I'm using what I believe should be a standard dockerfile.

Thanks!

Solved

19 Replies

7 months ago

The 502 + Container Event Container Died. means your application has crashed, I would recommend adding more verbose logging so you can hopefully get some insights into why it is crashing.


saginawjHOBBY

7 months ago

Thanks for the response. I do have logging throughout, and logging on app startup and shutdown. the 'shutdown' log never executes, and no other events are relevant (I can't execute any api calls- app dies before that happens).

I am running the app locally in Docker with the same Dockerfile, and there isn't an issue, so i assume maybe something specific with the Railway config.

My CMD is this: any reason to believe it has something to do with port not being configured correctly?

CMD uvicorn myappname.api:app --host 0.0.0.0 --port $PORT --proxy-headers 

7 months ago

> I do have logging throughout, and logging on app startup and shutdown. the 'shutdown' log never executes

Per your deploy logs -- yes those logs are being printed -

INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1]

But they lack verbosity, aka why did the application exit, what was it doing when it exited, etc etc.

In addition to that, there are no other logs printed before that to show what your application is doing (besides that its starting), there is no observability.

It is extremely challenging to debug application issues if you don't know what your application is doing.

> My CMD is this: any reason to believe it has something to do with port not being configured correctly?

Nope, that CMD line looks good to me, Railway sets a PORT for you.


saginawjHOBBY

7 months ago

Hmm I actually don't see those events in my Deploy log. I just see the below. And when I restart, I can see that the container restarts, then dies. Maybe I'm looking in the wrong place? I've looked at Deploy logs for the service, and also the Logs for the Project.

Attachments


7 months ago

I've looked at the deployment logs for the deployment you linked when creating this thread, regardless, there is a significant lack of observability, I suggest more verbose logging.


saginawjHOBBY

7 months ago

thank you for looking at the logs. I added more robust logging, including fastApi lifespan events and logging of every exception. yet I'm still just seeing the below in the logs- just a startup, and then a message indicating the container died.

Attachments


7 months ago

Are you making sure to log unbuffered to stdout / stderr?


saginawjHOBBY

7 months ago

I just added unbuffering, and still not seeing anything in the railway logs. Are you seeing anything different?


7 months ago

I am not seeing sufficient verbose logging either, please make sure to add verbose logging, every function call, every step in a function, etc etc.

The more observability, the more insight you get into your application's lifecycle!


saginawjHOBBY

7 months ago

The problem is, not a single function ever executes! I added logging after every step, every install. I just see the below, where the app starts, then the container dies.

Attachments


7 months ago

Some logs are printing, meaning some code is running, and thus, some code is crashing.


saginawjHOBBY

7 months ago

I'm running this exact docker container locally and it never crashes. I reduced the size of the container, added additional logging.

I use FastAPI's lifespan to track startup and shutdown events. you can see that on railway, the app starts, but i never get the 'Application startup complete' like I do locally. But I also don't get a lifespan event when the app crashes. I only get a 'container died' event which i assume is generated from Railway/Docker infra (certainly outside my app).



Attachments


7 months ago

Can you tell me more about what your app does during startup? specifically these inboxwatch logs.


saginawjHOBBY

7 months ago

sure! it's a FastAPI app, and this is the main.py entry point. It mounts static files (for jinja templates), initializes a few backend classes, sets up routes, sets up middleware, sets up state, sets up exception handling. all this seems to work on Railway - bc i see the API INITIALIZED. And than i instantiate the app, and the start lifespan event triggers. but then nothing else happens. I get the 'container died' error, and the Application Crashed error never triggers.

Attachments


7 months ago

Your latest deploy seems to be running?


saginawjHOBBY

7 months ago

ha! yes, it looks like it. very strange- didn't change anything really. just a little restructuring. i'll take a closer look but hopefully thing will work now. would love to know what the issue is in case it happens again.


7 months ago

Unfortunately we have no way of telling you why your app exited, risking sounding like a broken record, if you didn't get any logs before the exist, that means there is room for more logging! there is never not room for more logging though!


saginawjHOBBY

7 months ago

Advice accepted :). thanks for the support!


7 months ago

No problem!


Status changed to Solved brody 7 months ago


FastAPI won't deploy: 502 and 'Container Died' errors - Railway Help Station