Deployment crash errors when deploying new versions, but new deployment continues to deploy successfully
mbrookson
PROOP

9 months ago

I have noticed a strange issue recently where I get email notifications saying my deployment has crashed. It happens at the point where I have release a new deployment.

The strange thing is the new deployment works successfully and there are no apparent application errors that would cause a crash.

This never used to happen but wondering if it's general infrastructure related? Maybe related to Metal?

To note - I have a pre-deploy step to run Prisma migrations. It looks like this container logs the following

No pending migrations to apply.

┌─────────────────────────────────────────────────────────┐

│  Update available 6.5.0 -> 6.9.0                        │

│  Run the following to update                            │

│    npm i --save-dev prisma@latest                       │

│    npm i @prisma/client@latest                          │

└─────────────────────────────────────────────────────────┘

Stopping Container

 ELIFECYCLE  Command failed.

The "box" message warning about a Prisma upgrade is treated as "error" logs in Railway. Perhaps this is the cause of the issue? I can't see any other clues.

Any insights appreciated. Thanks!

Solved$10 Bounty

Pinned Solution

mbrookson
PROOP

7 months ago

Determined this is definitely related to the Prisma warning for available version updates being treated as error logs. Not sure why that is the case though.

To fix, I've added the PRISMA_HIDE_UPDATE_MESSAGE=true environment variable which suppresses the message.

5 Replies

chandrika
EMPLOYEE

9 months ago

Hey there, that seems unusual — it is likely not metal related or the warning you mentioned. Could you confirm the email you receive for the crash and the service checked is the same one? Think there should be a direct link to the service logs in the email (see there were some crashing deploys in your staging env


Status changed to Awaiting User Response Railway 9 months ago


chandrika

Hey there, that seems unusual — it is likely not metal related or the warning you mentioned. Could you confirm the email you receive for the crash and the service checked is the same one? Think there should be a direct link to the service logs in the email (see there were some crashing deploys in your staging env

mbrookson
PROOP

9 months ago

Thanks for the response.

Here is the email copy:

Deploy Crashed!

Uh oh. Your deployment for Website in Project crashed within the production environment.

It does clearly state that it was the production environment.

The last email I received this was June 5th at 21:45 (UK time).


Status changed to Awaiting Railway Response Railway 9 months ago


mbrookson

Thanks for the response.Here is the email copy:Deploy Crashed!Uh oh. Your deployment for Website in Project crashed within the production environment.It does clearly state that it was the production environment.The last email I received this was June 5th at 21:45 (UK time).

mbrookson
PROOP

9 months ago

I just remembered something else. Not sure if it could be related. I previously had the environment set up to be serverless. I have since disabled that feature on production.

The reason I disabled serverless was because on that same day my customer noticed the website would never spin up when they accessed it. I noticed the deployment was sleeping. I force redeployed it which brought it back, but then when it went to sleep again it never woke up. Switching away from serverless was the only choice at that point.

I can raise a separate issue if needed. Also not sure if that’s resolved now. It’s not been a problem for me before.


dikaaajs
FREE

9 months ago

Hey — I’ve run into this too recently, and yeah, it’s kind of confusing.

From the looks of it, it’s not your app crashing, but the pre-deploy container (the one running your Prisma migration) exiting with a non-zero status. Railway seems to flag that as a crash, even though the next deploy still works fine.

That Prisma update message might actually be the cause. Sometimes npm treats those update prompts or post-install logs as part of the lifecycle, and if anything in there triggers a non-zero exit, Railway logs it as a failed deployment.

You could try suppressing that message using --no-update-notifier or --silent in your command, or just wrap your migration step like this: npx prisma migrate deploy || true, so it doesn’t exit with a failure if there’s nothing to apply.

Doesn’t seem to be Metal-related — more like a false positive from how Railway interprets exit codes and logs from pre-deploy steps.


mbrookson
PROOP

7 months ago

Determined this is definitely related to the Prisma warning for available version updates being treated as error logs. Not sure why that is the case though.

To fix, I've added the PRISMA_HIDE_UPDATE_MESSAGE=true environment variable which suppresses the message.


Status changed to Solved christian 7 months ago


Loading...