Production Outage - TWO TIMES IN A WEEK!
jeromevvb
PROOP

2 months ago

HI

Despite your status page indicating a fix was deployed at 10:22 PM, we are still severely impacted by the EU West degraded performance incident

Our production environment has been experiencing extreme slowness for over 2 hours. All our API calls are timing out or taking unacceptably long to respond, making the platform effectively unusable for our users.

This is the second incident of this kind in less than a week. This is completely unacceptable for a production workload. We cannot continue to operate reliably on Railway if this keeps happening.

Solved

3 Replies

jeromevvb
PROOP

2 months ago

This is whats happening!!!! I did not touch anything in my app!!!

Attachments


jeromevvb
PROOP

2 months ago

Its finally up now


2 months ago

Look at your HTTP logs and filter by 502. These errors are all coming from your application, not Railway:

{
  "requestId": "fHhxnTh5QMOBEqy35nX1uw",
  "timestamp": "2026-03-20T16:32:19.633344020Z",
  "method": "POST",
  "path": "[redacted]",
  "host": "[redacted]",
  "httpStatus": 502,
  "upstreamProto": "",
  "downstreamProto": "HTTP/2.0",
  "responseDetails": "Retried single replica",
  "totalDuration": 15000,
  "upstreamAddress": "",
  "clientUa": "[redacted]",
  "upstreamRqDuration": 15000,
  "txBytes": 109,
  "rxBytes": 838,
  "srcIp": "[redacted]",
  "edgeRegion": "europe-west4-drams3a",
  "upstreamErrors": "[{\"deploymentInstanceID\":\"ffe1545c-e3b3-4fb7-93dd-d2997e34419e\",\"error\":\"connection dial timeout\",\"duration\":5000},{\"deploymentInstanceID\":\"ffe1545c-e3b3-4fb7-93dd-d2997e34419e\",\"error\":\"connection dial timeout\",\"duration\":5000},{\"deploymentInstanceID\":\"ffe1545c-e3b3-4fb7-93dd-d2997e34419e\",\"error\":\"connection dial timeout\",\"duration\":5000}]"
}

https://railway.com/project/4792e3f2-240b-4a78-91b4-82cba61609c2/service/e4921e23-73f1-49b3-9b82-0df0404bca72?environmentId=1e85c908-e12a-4e01-a171-b5400f09f347&id=97c7af3b-83e6-4c8e-8647-09ffda6b6b88&permalink=2026-03-20T16%3A32%3A19.633344020Z&start=1774024223329&end=1774024446400#http

Look at the upstreamErrors. "connection dial timeout" means your app did not respond to our proxy.

Your service is also deployed in US East, and thus was not affected by the EU West incident. That particular incident only impacted services deployed on EU West, it did not affect edge network connections from EU West to US East which is the traffic path you took according to your HTTP logs.

This is an issue with your application.


Status changed to Awaiting User Response Railway 2 months ago


Railway
BOT

2 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway about 2 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...