Apps Handling Removing Deploys (or redeploys)
shreyasayyengar
HOBBYOP

10 months ago

hello! i have a java project hosted on Railway (id: d8c2d7d4-9cc2-4de3-a784-cb082a468bda), in which i have registered JVM shutdown hooks. essentially initialised but unstarted java threads that start when the virtual machine begins its shutdown sequence.

unfortunately, when i remove a deploy on Railway, or push a commit to GH, which pushes a redeploy, these shutdown hooks do not get fired. is there something i can configure to make this work?

many thanks

Solved

22 Replies

brody
EMPLOYEE

10 months ago

Hello,

When an old deploy is being shut down we send sigterm and then wait 3 seconds, are your shut down hooks listening for sigterm specifically?


shreyasayyengar
HOBBYOP

10 months ago

hi brody,

yes, from my understanding, shutdown hooks are triggered for SIGTERM, and SIGINT, but not SIGKILL, for obvious reasons.

my shutdown hooks print to the console, and this works locally, but those same print messages do not show up when i locate my removed deploy logs.


brody
EMPLOYEE

10 months ago

perhaps there is something that is not passing the signals down to your app? for example, for node, npm will not pass the signals


shreyasayyengar
HOBBYOP

10 months ago

hmm, that's a fair point, but i honestly have no idea how i would know. is there anything specific you can share about how railway stops java applications/processess?


brody
EMPLOYEE

10 months ago

we dont stop java apps in any way different from any other application, we send the sigterm, wait 3 seconds, and kill the container


shreyasayyengar
HOBBYOP

10 months ago

is there an option to wait longer before the container is killed?


shreyasayyengar
HOBBYOP

10 months ago

the last thing that's viewable in the deploy logs is the container stopping. would it log the container being killed?

wondering if my shutdownhooks perhaps are taking too long, so Railway just ends up killing the container

1356437060185231600


brody
EMPLOYEE

10 months ago

its very possible they are taking longer than 3 seconds, and that can be increased with a RAILWAY_DEPLOYMENT_DRAINING_SECONDS variable


shreyasayyengar
HOBBYOP

10 months ago

unfortunately, it does look like this is the case.

locally, if i send a SIGTERM to my java program, the small debug runs. on Railway, this does not seem to be the case.

1356443260461256700


brody
EMPLOYEE

10 months ago

I promise you we send sigterm, I can write a simple test program to prove this behaviour if you'd like? (it won't be in java though)


shreyasayyengar
HOBBYOP

10 months ago

i do have a small hacky solution that sort of works. for context, this is a discord bot that i am hosting. i have a slash-command /exit that internally makes the java program run exit with code 0. that ends up triggering my Shutdown hooks on railway, and then marks the build as COMPLETED rather than REMOVED. Downside of this is that each time i push a commit, i need to manually run /exit to run my shutdown hooks. and then let railway redeploy.

is this (this being SIGTERM not being received) something that Railway could even take a look at? im happy to spin up a very basic java program to see if it's working as intended


brody
EMPLOYEE

10 months ago

its not something we could take a look at because its an issue at the application level and not an issue at the platform level


shreyasayyengar
HOBBYOP

10 months ago

so even if i were to spin up a very elementary java project, to see if the SIGTERM is being received, if the program isn't receiving it, that's no longer Railway's jurisdiction i guess?


brody
EMPLOYEE

10 months ago

given the platform did send the sigterm, yes that would still be an application level issue


shreyasayyengar
HOBBYOP

10 months ago

hmm okay. is it okay if i leave this open? i have a feeling that maybe this is a docker related issue and that process IDs are not being referenced correctly. is that something that maybe railway would look into?

if something has to be changed with the Dockerfile, is that something i must be responsible for?


brody
EMPLOYEE

10 months ago

all we are doing is running java -jar target/commissionsmanager-1.0-SNAPSHOT.jar


shreyasayyengar
HOBBYOP

10 months ago

i see. how is the docker container stopped? does it just issue a docker stop command?

reason why i ask, is that this thread may be of some interest:

https://stackoverflow.com/questions/31836498/sigterm-not-received-by-java-process-using-docker-stop-and-the-official-java-i


brody
EMPLOYEE

10 months ago

have you tried the suggestion in the first response?


brody
EMPLOYEE

10 months ago

1356449828250452200


shreyasayyengar
HOBBYOP

10 months ago

cheers brody, i really appreciate your patience with me! was able to get things working as expected by prefixing with exec.

really loving railway the more and more i use it :)


brody
EMPLOYEE

10 months ago

awsome, glad i could help in some way haha


brody
EMPLOYEE

10 months ago

!s


Status changed to Solved brody 11 months ago


Loading...