Replica randomly went down and traffic is seemingly still being steered to it
arunesh90
PROOP

a month ago

Creating a seperate help thread as that was recommended to me:

With the current outage one of my replicas randomly went down and never recovered. Additionally it looks as if Railway's LB is still trying to steer towards it as we've been getting a ton of errors since then

image.png
Screenshot_2026-02-11_at_16.01.03.png

Solved

6 Replies

a month ago

Hello,

We had an incident earlier that caused some running workloads to be unexpectedly terminated while the status in the UI didn't properly update. This has since been resolved, but in some cases, you may need to redeploy your services.

You can find the post-mortem for the incident here: https://blog.railway.com/p/incident-report-february-11-2026

Our deepest apologies for the inconvenience this has caused.


arunesh90
PROOP

a month ago

Thanks Brody. Out of curiosity, the Railway LB steering to replicas that are down seems to be somewhat of a seperate issue, still caused by the outage of course, but I was wondering whether it's worth adding extra safeguards to the LB for future incidents?


arunesh90
PROOP

a month ago

I do realise that's easier said than done, but was just curious. If the LB didn't route to the faulty/offline replica, then this outage would have had significantly less impact for us


a month ago

The problem is that we thought they were online.


arunesh90
PROOP

a month ago

aha okay. The dashboard/api did show on my side that the replica was offline so wasn't sure whether the LB had the same picture


arunesh90
PROOP

a month ago

makes sense then, ty for the clarification


Status changed to Solved brody about 1 month ago


Loading...