2 years ago
Hi there, I saw that I could set RAILWAY_DEPLOYMENT_DRAINING_SECONDS to give my change an app to shut down gracefully.
I have some logic in there that works fine on Heroku - I can see my dynos cleaning up and shutting down when I scale down my dynos. I see all my clean up logs and messages.
When I try the same on Railway, I don't see any logs. I am not sure if my app is gracefully shutting down. I am not sure how to debug this.
61 Replies
2 years ago
are you catching sigterm?
yes, should be, i use enableShutdownHooks in my nest app which by default catches…
SIGHUP = 'SIGHUP', SIGINT = 'SIGINT', SIGQUIT = 'SIGQUIT', SIGILL = 'SIGILL', SIGTRAP = 'SIGTRAP', SIGABRT = 'SIGABRT', SIGBUS = 'SIGBUS', SIGFPE = 'SIGFPE', SIGSEGV = 'SIGSEGV', SIGUSR2 = 'SIGUSR2', SIGTERM = 'SIGTERM',
2 years ago
do you see logs during regular app activity?
yeah I see my app working normally, and when I restart or redeploy my service it just 'cuts out' where it is and then the new one boots up
2 years ago
okay thanks for the info, ill try to get a proof of concept working
ok that would be fantastic, thanks. I thought maybe it is working in the background but the logs aren't being captured anymore? But i have no way of verifying it without seeing the logs
2 years ago
one idea popped into my head, what logger are you using?
2 years ago
have you disabled log buffering?
2 years ago
sounds good, i shall get back to my tests
i wonder if it is because i am using newrelic and starting my app that way, ie node -r newrelic dist/apps/ssr-game-server/main
2 years ago
that is very possible
2 years ago
i wouldn't know if newrelic forwards the signals along, are you doing the same on heroku?
2 years ago
try without it
2 years ago
it does work, i set it to 15, the default being 3
2 years ago
any luck taking newrelic out of the equation?
2 years ago
nixpacks or dockerfile?
2 years ago
iirc nixpacks wraps the start command in a shell, that also might not be passing down the signals
2 years ago
how hard would it be to write a dockerfile for your app? it would give you much better control and allow you to ensure the node process is started in exec mode
I have no experience in it, so honestly I have no idea! I am transitioning from Heroku to Railway, solo indie game dev wearing so many hats, haha. My experience is limited. Though I am not doing anything fancy. It's just a boring old nodejs/nest app
2 years ago
no problem, ill see if i can replicate this issue when starting my app wrapped in a shell like how nixpacks starts node apps
2 years ago
there is an issue on nixpacks about this
but neither me or dush can reproduce, and i've tried wrapping my app in a bash and sh shell.
i think it might still be worth a try to write a dockerfile, i can try to help you with that if you'd like?
i'd love a hand with that, sounds like a good test considering the open issue
2 years ago
for me to have a good chance at writing a working dockerfile for you, please share your most recent build log, anything you have changed nixpacks or service settings wise, and your repo
Hey thanks for offering, i did some more digging and I belive i've found the root cause
2 years ago
oh yeah? do tell!
but having a hard time verifying as there seems to be some issues with logs right now..? I have a scattering of logs missing from obervability
2 years ago
hmmm I haven't seen any other reports of that, what kind of logs are you missing?
my deploy logs for a fresh deploy look like this:
nothing
business as usual
It's missing the typical 'Nest app started' etc. Its not even visible in the Observability panel when i narrow down to the specific service …

2 years ago
incorrect log level? are we sure next doesn't buffer logs by default?
anyway, lets ignore that for now, I did observe a log that shows my shutdown signals are working
2 years ago
what was the fix?
though, i mentioned the log thing because this is just a few of my graceful shutdown logs haha (I have some logs missing within the same function that those 2 are from hah)
so for me it was a matter of changing my custom start command from
npm run start-game-server:newrelic
to
node -r newrelic dist/apps/ssr-game-server/main
2 years ago
I thought you where using node -r already?
no, i had a custom start command as i have a mono repo and had to specify which to run
2 years ago
ah I see
2 years ago
happy to help where I can
2 years ago
if you are still having issues with logs, please open another thread! 🙂

