@Brody could you confirm if this is a Railway issue or a Cloudflare issue?

Neither.

Have you redeployed your service?

we started having issues 2 hours ago but no luck understanding what's going on, no recent deployments. DBs are working fine

just these logs showing up

Please do then.

ok doing now

we just restarted before but it didn't work, now I'm doing a redeploy

ok it's back, could you help us understand what happened?

also the redeploy was almost instant, usually take 5mins+

AWS shut outbound off for the host you where on.

ok cool, followup Qs:

• aren't we on Railway metal? or just for building the containers?
• can't we detect and auto-redeploy once issues like this happen?

also anything we should do on our end to prevent stuff like this?

Railway metal isn't relevant anymore, services may be put on GCP or AWS depending on availability at any given time, same performance and same cost, so no worries there.

This is something we are already talking to our AWS rep with, it shouldn't happen again.

ok so nothing much to do on our end? should we just expect improvements happening from Railway and other providers?

Correct, we are always improving something!

we love Railway, but unfortunately these incidents are very costly with our customers

hope Railway is also thinking on some "auto-resolve" systems that after these incidents it goes back to normal rather than relying on manual interventions

really +1 on this, shouldn't have to manually do things to work around incidents