previous deployment crash
whitetown
PROOP

a year ago

When I deploy the API (express, nodejs, socketio) everything goes well,
but right before the new deployment will replace the old one,
I see on the dashboard and I get an email notification
**Deployment crashed for [service name] in [project name]! **
why this happens? is there something I can do?
project id: b799250d-dd7b-47bb-be65-0ae878afd304

0 Replies

brody
EMPLOYEE

a year ago

did the deployment actually crash or is it a false report? I don't think you specified this information


whitetown
PROOP

a year ago

I think it's a false report. however it's slightly annoying.
The new deployment is completely fine, deployed, works, and never crashes.
But the previous one, when it's killed by railway, marked as CRASHED in the control panel and I also get an email about this.
with the next deployment it happens again.
unfortunately, there are no logs for previous deployments or explanation the source of the crash.


brody
EMPLOYEE

a year ago

is your app handing sigterm properly?


whitetown
PROOP

a year ago

I guess, it did not. maybe this is what I am looking for


brody
EMPLOYEE

a year ago

its not like you are using nodemon or pm2 right?


whitetown
PROOP

a year ago

no, I do not use nothing external. pure railway



brody
EMPLOYEE

a year ago

what do the logs say of that crashed service?


whitetown
PROOP

a year ago

nothing in logs,
yarn run v1.22.22
$ node index.js
Connection has been established successfully.
server started at 3040


brody
EMPLOYEE

a year ago

those are logs from the service that was marked as crashed?


whitetown
PROOP

a year ago

yes


whitetown
PROOP

a year ago

found something like How do I shut down my Express server gracefully…
trying this. thanks for the tip


brody
EMPLOYEE

a year ago

let me know how that goes!


brody
EMPLOYEE

a year ago

but, are you on the legacy or v2 runtime? check your service settings


whitetown
PROOP

a year ago

the service/project was created last month. I did not choose the runtime, and I hope that by default it's latest


brody
EMPLOYEE

a year ago

well could you please check


whitetown
PROOP

a year ago

I do not know where yo see it


brody
EMPLOYEE

a year ago

the service settings


whitetown
PROOP

a year ago

aha.. legacy


whitetown
PROOP

a year ago

shall I change it?


brody
EMPLOYEE

a year ago

you can try it


whitetown
PROOP

a year ago

```process.on('SIGTERM', shutDown);
process.on('SIGINT', shutDown);

function shutDown() {
console.log('Received kill signal, shutting down gracefully');
server.close(() => {
console.log('Closed out remaining connections');
process.exit(0);
});

setTimeout(() => {
    console.error('Could not close connections in time, forcefully shutting down');
    process.exit(1);
}, 10000);

}```

neither that code, nor v2 did not help.
okay, I will try to make some experiments and come back later


brody
EMPLOYEE

a year ago

do you see the Received kill signal, shutting down gracefully logs on the crashed service?


whitetown
PROOP

a year ago

no 😦


brody
EMPLOYEE

a year ago

back to the drawing board


whitetown
PROOP

a year ago

okay, locally it catches sigterm and sigint.
if I run the api locally and kill it somehow, I get in the console
yarn start yarn run v1.22.21 $ node index.js Connection has been established successfully. server started at 3040 Shutting down gracefully...

on Railway this message does not appear.

https://expressjs.com/en/advanced/healthcheck-graceful-shutdown.html


brody
EMPLOYEE

a year ago

what is your start command on railway?


whitetown
PROOP

a year ago

just yarn start


brody
EMPLOYEE

a year ago

thanks, what is your start script?


whitetown
PROOP

a year ago

{
    "$schema": "https://railway.app/railway.schema.json",
    "build": {
        "builder": "NIXPACKS",
        "watchPatterns": ["/apps/ai-api"]
    },
    "deploy": {
        "numReplicas": 1,
        "startCommand": "yarn --cwd apps/ai-api start",
        "healthcheckPath": "/",
        "sleepApplication": false,
        "restartPolicyType": "ON_FAILURE",
        "restartPolicyMaxRetries": 10
    }
}

brody
EMPLOYEE

a year ago

what is the start script for apps/ai-api?


whitetown
PROOP

a year ago

"scripts": {
"dev": "node --env-file=.env.local index.js",
"start": "node index.js"
},


brody
EMPLOYEE

a year ago

I don't think yarn is passing signals down, can you have railway start index.js directly with node?


whitetown
PROOP

a year ago

ok


whitetown
PROOP

a year ago

@Brody I owe you a beer. Resolved. Thanks a lot!


brody
EMPLOYEE

a year ago

no problem!


Loading...