Unusually Slow Deploys + Internal Network Communication Failures
rbrdigital
HOBBYOP

22 days ago

Hi team,

We’re currently experiencing some abnormal behavior in our Railway infrastructure and wanted to report it.

  1. Very slow deploys:
    One of our frontend servers took more than 30 minutes to build, which is highly unusual for us. Our builds are typically much faster and we haven’t made significant changes that would justify such a delay.

  2. Internal network instability:
    After the build, we started seeing intermittent failures in the Railway internal network.
    We have:

  • A frontend being accessed via reverse proxy from another frontend

  • Communication happening strictly over Railway’s internal network

This setup has been stable, but now requests occasionally fail between services.

  1. Backend database connectivity issues:
    One of our backend microservices (which connects to a MongoDB instance over Railway internal networking) started throwing database connection errors.

When attempting to redeploy the MongoDB service:

  • It has been stuck in “Deploying” for more than 15 minutes

  • Normally, this MongoDB service boots in ~5 seconds

This behavior is clearly outside our normal baseline.

Could you please check if there are:

  • Ongoing infrastructure issues

  • Internal networking degradation

  • Regional cluster instability

  • Or any incidents affecting deploy/build queues?

Thanks in advance.

Project ID: e60124ce-5bb4-41cb-8517-9606dbe5b9f7
Environment ID: 9700af47-ddfe-42ba-96b3-d3323dd60b27
MongoDB Service ID: d5274561-b8e2-4966-9c1c-1522fd37ce01

image.png

Attachments

Solved

18 Replies

rbrdigital
HOBBYOP

22 days ago

Hey @Brody, come here bro..


rbrdigital
HOBBYOP

22 days ago

image.png

Attachments


rbrdigital
HOBBYOP

22 days ago

This guy tried to take me to another server, I think he should be banned.


22 days ago

Done


22 days ago

thx for the report


rbrdigital
HOBBYOP

22 days ago

trying to redeploy again

image.png

Attachments


rbrdigital
HOBBYOP

22 days ago

Now, magically, everything is working and the front-end server is no longer slow; everything seems to be flowing smoothly.


rbrdigital
HOBBYOP

22 days ago

I created a replica in US-West for the frontend, could that have solved the problem?


22 days ago

Not the typical experience, I would say the unusual high platform demand was what affected you here. Railway gained a lot of new users in the last week or so and the team is actively working to make everything steady.


22 days ago

Not sure


22 days ago

Is everything working fine now?


rbrdigital
HOBBYOP

22 days ago

I understand, this happened during an important deployment and caused some stress for the team, but everything seems to be fine now, thank you for your support!


mativm02
PRO

22 days ago

hey, same happened here, our backend was unable to connect to our Mongo (hosted on Railway) for some reason. we fixed this by just redeploying both services, but this costed us 1 hour of downtime and is the second time in one week, we're worried


fredgig
PRO

22 days ago

Looks like one of my services was randomly producing 502's up until I redeployed it just now, I see that there was an automatic deployment done by Railway about an hour ago, unsure if that's what caused it


thelucre
PRO

22 days ago

two of my environments jsut stopped working, third time in a week! is anyone else still affected by the current incident?


22 days ago

Could you share a link to your service on Railway please?


fredgig
PRO

22 days ago


22 days ago

Sorry about that - we had a transient routing issue that has since been resolved


Status changed to Solved brody 21 days ago


Loading...