a year ago
My system was running fine this morning, no code updates and then it stopped. When I try to start again, I get this...
Starting Container
[2025-09-23 15:34:05 +0000] [1] [INFO] Starting gunicorn 23.0.0
[2025-09-23 15:34:05 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2025-09-23 15:34:05 +0000] [1] [INFO] Using worker: sync
[2025-09-23 15:34:05 +0000] [4] [INFO] Booting worker with pid: 4
๐ Starting SummitFX app.py
ENV: production
SET PORT
SET ENV
SET DATABASE_URL
SET OPENAI_API_KEY
SET OPEN_EXCHANGE_RATES_APP_ID
SET PIPEDRIVE_API_KEY
SET META_PHONE_NUMBER_ID
SET META_ACCESS_TOKEN
SET VERIFY_TOKEN
Startup checks complete
INFO:app:๐ Running in PRODUCTION mode
100.64.0.2 - - [23/Sep/2025:15:34:07 +0000] "GET /healthz HTTP/1.1" 200 32 "-" "RailwayHealthCheck/1.0"
Stopping Container
[2025-09-23 15:34:09 +0000] [1] [INFO] Handling signal: term
[2025-09-23 15:34:09 +0000] [4] [INFO] Worker exiting (pid: 4)
[2025-09-23 15:34:09 +0000] [1] [INFO] Shutting down: Master
100.64.0.3 - - [23/Sep/2025:15:47:57 +0000] "GET /healthz HTTP/1.1" 200 32 "-" "curl/8.7.1"
37 Replies
a year ago
Hey there! We've found the following might help you get unblocked faster:
- ๐งต Django + Gunicorn container stops immediately after starting
- ๐งต Sigterm recieved
- ๐งต Container SIGTERM after 60-65 seconds despite healthy keepalive and health checks
- ๐งต Application Crashes on Deployment - ModuleNotFoundError for MySQLdb
If you find the answer from one of these, please let us know by solving the thread!
a year ago
Looks like your health check endpoint is causing it to crash. Is it supposed to be the endpoint /healthz? Did a "z" get added on there perhaps?
a year ago
It's currently /healthz - is that what it's supposed to be?
a year ago
I'm just wondering if the Z is an accident. The issue is your health endpoint not returning 200. Have you made any changed recently? Can you share the repository?
a year ago
No changes and I even deployed a working version from 17th September in case any changes from recent days may have done something.
From ChatGPT..
Yes, the health path is intentionally /healthz (not a typo).
It returns 200; hereโs the proof from access logs:
GET /healthz ... "RailwayHealthCheck/1.0" 200
- The earlier โservice unavailableโ was because the app failed to boot due to a duplicate
/memroute. Thatโs fixed. - The โStopping Containerโ lines right after a 200 are Railway ending its deploy healthcheck container, not your app crashing.
cornolly
No changes and I even deployed a working version from 17th September in case any changes from recent days may have done something. From ChatGPT.. **Yes, the health path is intentionally** `/healthz` (not a typo). It **returns 200**; hereโs the proof from access logs: `GET /healthz ... "RailwayHealthCheck/1.0" 200` * The earlier โservice unavailableโ was because the app **failed to boot** due to a duplicate `/mem` route. Thatโs fixed. * The โStopping Containerโ lines right after a 200 are **Railway ending its deploy healthcheck container**, not your app crashing.
a year ago
ChatGPT is useless.
Can you confirm when running locally the endpoint works? Can you share the repository?
a year ago
Can you join a Teams or Zoom call?
a year ago
Hey raj5612,
Unfortunately thats not applicable to the current situation. The user is on the pro plan which features 32g of ram.
cornolly
Can you join a Teams or Zoom call?
a year ago
No.
You're right about it returning code 200... I'm just confused as to why it crashes right after. Is there any way I can see exactly what this code is doing? Is this project open source?
a year ago
It would be curious to see though.
cornolly are you able to confirm that locally hitting the /healthz endpoint returns 200 and the application is fine?
Any info about the project would also be helpful.
a year ago
No it doesn't look to be working locally...
AttributeError: 'NoneType' object has no attribute 'encode'
[2025-09-23 17:58:11 +0100] [90916] [INFO] Worker exiting (pid: 90916)
[2025-09-23 17:58:11 +0100] [90915] [ERROR] Worker (pid:90916) exited with code 3
[2025-09-23 17:58:11 +0100] [90915] [ERROR] Shutting down: Master
[2025-09-23 17:58:11 +0100] [90915] [ERROR] Reason: Worker failed to boot.
((.venv) ) nickcornford@Nicks-MacBook-Air-8 Quote %
cornolly
No it doesn't look to be working locally... AttributeError: 'NoneType' object has no attribute 'encode' \[2025-09-23 17:58:11 +0100\] \[90916\] \[INFO\] Worker exiting (pid: 90916) \[2025-09-23 17:58:11 +0100\] \[90915\] \[ERROR\] Worker (pid:90916) exited with code 3 \[2025-09-23 17:58:11 +0100\] \[90915\] \[ERROR\] Shutting down: Master \[2025-09-23 17:58:11 +0100\] \[90915\] \[ERROR\] Reason: Worker failed to boot. ((.venv) ) nickcornford@Nicks-MacBook-Air-8 Quote %
a year ago
Ah theres your issue!
So it looks like an AttributeError on your project.
I believe the full error should tell you a bit better as to where things are failing.
After you fix that area things should be good!
a year ago
I changed to 10 but it still happened after 4 seconds
noahd
Ah theres your issue! So it looks like an AttributeError on your project. I believe the full error should tell you a bit better as to where things are failing. After you fix that area things should be good!
a year ago
What's the recommendation sorry?
cornolly
What's the recommendation sorry?
a year ago
Well the thing is your health check endpoint is failing locally.
That means it will also fail on Railway!
So my suggestion is I'd figure out where in the endpoint its having this AttributeError and solve that. Once it works locally it'll work on Railway!
noahd
Well the thing is your health check endpoint is failing locally. That means it will also fail on Railway! So my suggestion is I'd figure out where in the endpoint its having this `AttributeError` and solve that. Once it works locally it'll work on Railway!
a year ago
Yeah I don't know where to start in doing that
cornolly
Yeah I don't know where to start in doing that
a year ago
Are you able to provide the /healthz endpoint so I can take a peek?
noahd
Are you able to provide the `/healthz` endpoint so I can take a peek?
a year ago
How do I do that?
a year ago
I assume this is a python project from the logs. Are you able to copy paste the code here?
Or if its in a repo that's open sourced I'm happy to look there.
However, in terms of issue thread it seems like its been identified.
a year ago
You may have gathered I'm somewhat out of my depth here. Again a CGPT response...
This is a Python/Flask app. Health is intentionally on /healthz.
Start command:
gunicorn app:app -b 0.0.0.0:$PORT --access-logfile - --log-level info
Python: 3.12
Minimal env to boot locally:
export ENV=production
export PORT=8080
export VERIFY_TOKEN=dummy
Endpoints to test:
- Health:
http://127.0.0.1:8080/healthzโ 200 - Webhook verify:
http://127.0.0.1:8080/webhook?hub.mode=subscribe&hub.verify_token=dummy&hub.challenge=1234โ echoes1234I can share the repo (link below) or the minimal files (app.py + requirements.txt).
a year ago
Completely understand how that can feel! I'd ideally like to deal with you instead of ChatGPT but no worries at all.
Let me take a step back and explain what I'm trying to get to.
We're trying to fix the issue where your app was crashing after hitting the /healthz endpoint.
We have identified by your local testing (thanks for that!) that its broken both locally and on Railway which simply means theres some code at fault! Nothing big just a bug fix to tweak.
Thanks to your logs we can see that your main error here is: AttributeError: 'NoneType' object has no attribute 'encode'.
Feel free to ask away if you need me to explain/shed some light on anything!
Now, for next steps.
The project crashes because of the healthcheck failing. So we need to fix that!
I think what would help best is pasting the entire code from /healthz into this chat or sending me the repo would be ideal!
Would love to see that so I could possibly help with any issues.
a year ago
The issue is I don't know how to do that securely. The code is about 3500 lines - you want it all pasted in here?
cornolly
The issue is I don't know how to do that securely. The code is about 3500 lines - you want it all pasted in here?
a year ago
Well if theres a safety issue yes please do not paste it here.
Also if its 3500 lines dont paste it either.
As I had mentioned though the error lies on the /healthz route not the entire project.
Can i get some info about that project possibly?
a year ago
This?
100.64.0.2 - - [23/Sep/2025:17:37:30 +0000] "GET /healthz HTTP/1.1" 200 32 "-" "RailwayHealthCheck/1.0"
a year ago
This is the raw data...
{
"message": "100.64.0.2 - - [23/Sep/2025:17:37:30 +0000] \"GET /healthz HTTP/1.1\" 200 32 \"-\" \"RailwayHealthCheck/1.0\"",
"attributes": {
"level": "info"
},
"tags": {
"project": "4beed22f-9680-4164-ac53-a9361127b24c",
"environment": "fc02c4e3-cb0b-4465-8b5f-db41ce5e6356",
"service": "b94417cd-6f1e-4565-aaeb-509273bc02b7",
"deployment": "df91cc0c-4e34-4099-ac6c-455592f101a1",
"replica": "29af8d4d-a2d8-4175-b6df-2b8eb36656e4"
},
"timestamp": "2025-09-23T17:37:30.489486638Z"
}
a year ago
What are we doing here?
Also, how is it currently showing as deployed but still has ...
Stopping Container
[2025-09-23 17:37:32 +0000] [1] [INFO] Handling signal: term
[2025-09-23 17:37:32 +0000] [4] [INFO] Worker exiting (pid: 4)
[2025-09-23 17:37:33 +0000] [1] [INFO] Shutting down: Master
Stopping Container
a year ago
Unfortunately this looks to be an application (your code) issue and not a platform (railway) issue.
If the nature of the code is private and secure I don't think we can really provide much help for it.
a year ago
But I've gone back to multiple previous working versions and they also don't work?
Please can you answer this as well...
Also, how is it currently showing as deployed but still has ...
Stopping Container
[2025-09-23 17:37:32 +0000] [1] [INFO] Handling signal: term
[2025-09-23 17:37:32 +0000] [4] [INFO] Worker exiting (pid: 4)
[2025-09-23 17:37:33 +0000] [1] [INFO] Shutting down: Master
Stopping Container
a year ago
And any reason I'm being killed when you're returning a healthz 200?
a year ago
Another strange thing... I have production and staging under the same project. Staging hasn't been touched in about 2 months and is half the size of production. I used that today and it worked but it also is down. Here are the most logs from staging...
Attachments
a year ago
Can anyone help/provide some ideas? This project is what I'm running my business on so it's particularly distressing it not working and having no idea of even a path to take to try and find the problem
samgordon
No. You're right about it returning code 200... I'm just confused as to why it crashes right after. Is there any way I can see exactly what this code is doing? Is this project open source?
a year ago
Is there a way I can provide you with better/more access?
cornolly
Is there a way I can provide you with better/more access?
a year ago
Sharing the repository would be great.
a year ago
Note, the user said it happens locally too; their issue is not related to the Railway platform.
And yes, please refrain from using AI for bounty answers, as it is a bannable offense (bannable from participating in bounties, not banned from the platform).
a year ago
Hey Cornolly,
Wanted to ask another thing. In your application the /healthz endpoint itself is failing as we discussed earlier today.
Can you share a bit more about all that it does? Just some info on what it checks.
a year ago
It would also be great if you could at minimum share the source code for the /healthz endpoint. That should be all we need.
Status changed to Solved cornolly โข 11 months ago