a year ago
I made a very minor text change, and now my deployment isn't working where as it was working fine yesterday. The deploy logs look normal, but my build logs show that my health check isn't working (it works locally).
0 Replies
My health check is /health
which serves:
def health_check(request):
return HttpResponse(status=200)
a year ago
what is the health check failing with?
a year ago
are you on the legacy or v2 runtime? check your service settings
a year ago
on the v2 runtime your app needs to listen on ::
a year ago
might have
I don't love that. Would've been nice to know about a breaking change like this.
a year ago
You can indeed switch to legacy.
a year ago
it has been mentioned in two places, what else would work best for you?
a year ago
We expect the legacy runtime to stay in place for as long as we get the expected behavior that our users need.
a year ago
a year ago
if the healthcheck issue is the only issue you face, I cannot recommend switching back to the legacy runtime
a year ago
fwiw the health check issue has been reported to the team
That's fair, but I'm not always looking at the changelog unless I'm interested to see what new features are available to me. I do get emails about the changelog with some basic bullet points, but it would've been nice to have in this email, or a separate email a message long the lines of:
"Starting 6/x/2024, all services will be switched from Legacy to V2, and here's what you need to do to before then:"
a year ago
its a fair assumption that the changelogs would only include new features, and they do, but that also mention migration timelines and such for new features and new features always have the possibility to cause issues
True, but imo known breaking changes should be communicated more directly. I don't always have the time to read changelogs for all of the services I use. My project is a hobby project, so no bigs, but for the enterprise customers, that could put a snag in their work. Luckily the support here is really on top of things!
a year ago
i dont think this was known tbh, but i have no way to know for sure
a year ago
waiting to hear back from char on this issue
Yeah, in that case, it's a hiccup. And good on the Railway team for testing with Hobby accounts first so they can find these issues before they reach enterprise customers.
I'm still having issues getting this to work. I've add [::1]:$PORT
to my gunicorn command. I've confirmed this working locally, but still having trouble with the health check
So it was gunicorn project.wsgi
and now it's gunicorn -b 127.0.0.1:$PORT -b [::]:$PORT project.wsgi
a year ago
it needs to be ::
not ::1
a year ago
Dumb ask and unsure if you did this in the past, switching to Legacy confirmed will fix the issue? Wanna make sure our network engineer can do a proper repro.
a year ago
yes, check <#880575219541114940>
a year ago
He is flagging me to another case 🙂
a year ago
We just wanna have more languages to test runtime with hence why I ask.
a year ago
The more cases the better.
a year ago
And sorry to use you as a test pig, I can comp you the month since you are doing QA work.
a year ago
gunicorn -b [::]:$PORT project.wsgi
a year ago
deploy logs please -
a year ago
new role added
a year ago
comped, test away, let us know when you have recovered the healthcheck
Ah, looks like that only gets the newest logs. Here's what it shows:
[2024-06-11 19:11:19 +0000] [1] [INFO] Starting gunicorn 21.2.0
[2024-06-11 19:11:19 +0000] [1] [ERROR] Connection in use: ('::', 65090)
[2024-06-11 19:11:19 +0000] [1] [ERROR] Retrying in 1 second.
[2024-06-11 19:11:20 +0000] [1] [ERROR] Connection in use: ('::', 65090)
[2024-06-11 19:11:20 +0000] [1] [ERROR] Retrying in 1 second.
[2024-06-11 19:11:21 +0000] [1] [ERROR] Connection in use: ('::', 65090)
[2024-06-11 19:11:21 +0000] [1] [ERROR] Retrying in 1 second.
[2024-06-11 19:11:22 +0000] [1] [ERROR] Connection in use: ('::', 65090)
[2024-06-11 19:11:22 +0000] [1] [ERROR] Retrying in 1 second.
[2024-06-11 19:11:23 +0000] [1] [ERROR] Connection in use: ('::', 65090)
[2024-06-11 19:11:23 +0000] [1] [ERROR] Retrying in 1 second.
[2024-06-11 19:11:24 +0000] [1] [ERROR] Can't connect to ('::', 65090)
container event container died
a year ago
ill try to reproduce
a year ago
what version of gunicorn?
I changed my gunicorn command back to what it was, and flipped the runtime to Legacy and it deployed successfully. And that's including the minor code change mentioned in the original post.
a year ago
Gotcha- that seems to be enough, going to add this case on the Runtime V2 blockers in the root thread.
a year ago
my start command is gunicorn -b [::]:$PORT main:app
on the v2 runtime with the same gunicorn version you are using, so this new error doesnt look like a v2 vs legacy issue
a year ago
does your container run gunicorn and only gunicorn?
a year ago
can you provide the full command
python [manage.py](manage.py) migrate && python [manage.py](manage.py) collectstatic --noinput && gunicorn project.wsgi
a year ago
and what was the command when you got this error?
a year ago
that would do it, gunicorn supports dual stack binding anyway so that wouldnt be needed, 127.0.0.1 would also be the incorrect address
Their documentation seems to suggest that you need to state both: https://docs.gunicorn.org/en/stable/settings.html#bind
a year ago
Yes, binding on 127.0.0.01 won't bind properly.
a year ago
But wondering why legacy did it.
I didn't have that for legacy. I was just adding it in because I assumed I needed it if I also needed to have IPv6. My bad.
Alright, well. It worked with python [manage.py](manage.py) migrate && python [manage.py](manage.py) collectstatic --noinput && gunicorn -b [::]:$PORT project.wsgi
a year ago
by default gunicorn binds to 0.0.0.0:$PORT
so that would have worked for legacy
a year ago
as i suggested 🙂
a year ago
no worries
Guess this isn't a new bug then, Angelo! I apologize. New to messing with IPv6.
a year ago
it is a new bug
a year ago
you should not need to listen on ipv6 just for the health check to work
a year ago
Yea, if any behavior is different vs. old, its a bug.
a year ago
You did us a favor.
a year ago
technically solved
10 months ago
Update, health checks can now pass if your app only listens on 0.0.0.0
but if you have already changed it to ::
there's no point in changing anything back as listening on ::
has no known drawbacks.