You can definitely get crash notifications!

https://railway.com/changelog/2021-11-26#crash-notification-emails

Webhook ones too

https://railway.com/changelog/2021-12-03

Thanks for the reply - I have a HTTP endpoint which causes a test crash, and I'm not receiving email notifications for these, nor other crashes.

Are you sure "Crashed deploys" doesn't refer to "unsuccessful deploys" - i.e. I push an update and it crashes while deploying, and I receive an email?

As opposed to a server that has successfully deployed, has been running for a few hours, and then crashes.

It indeed refers to the latter

Ah okay, so by the latter you mean it only applies to "unsuccessful deploys"

So is there no email notification for server crashes, besides hacky approaches? For the use cases mentioned:

This is essential as if the server crashes during a certain operation (and therefore the operation didn't complete successfully), or a certain route or process caused the server to crash - you want to investigate. Such crashes may not be reported to e.g. Sentry successfully.

Yeah, there isn’t a way to get email notifications for server crashes that happen after a successful deployment. The existing notification system is primarily for deployment status changes (deploy succeeding, crashing etc)

Your use case makes sense, you could drop us a feedback / feature request or upvote an existing one to see if other users would like this but for your current use case you could set up tools like Checkly, etc to ping your endpoints

I do have uptime monitoring but even if this pings every 1-5 minutes, it's a hacky and unreliable way to check for crashes. After a crash, the server could recover within a minute or two without sending an alert.

It would also send false alarms for server restarts or redeploys, so would quickly become noise. And uptime monitors also tend to have a "grace/recovery period" in case of false positives and this means they would miss crashes.

SDK's in your server code aren't reliable because if they crash, they might not get the message out.

Maybe if you export Railway logs to an external provider you could monitor for keywords to report crashes - but that would have to be Railway logs (which 100% describe a crash) not your own server logs which wouldn't reliably report it. Not sure if that's possible with Railway.

There isn't a way to set up keyword monitoring for Railway logs and to send an alert when certain keywords (e.g. describing a crash) are detected? Or to send a webhook?

From my end this seems like a basic requirement for a host of any serious production app, I'm really surprised Railway doesn't have it. Really impressed with Railway otherwise, since we moved from Heroku a couple weeks ago.

Totally understand where you're coming from - there's a good amount of backlog in the observability land.

If that's okay with you I'd like to invite you to share this as public feedback here: https://station.railway.com/feedback

Our public feedback board often turns into features, and having it out there helps us understand which features our customers want most