2 months ago
## Description of the issue
I'm experiencing random 503 errors across multiple production services (~10 services) deployed on Railway.
These services were working correctly a few days ago without any issues, but now they are failing intermittently without any code changes.
The failures are non-deterministic:
- Sometimes requests succeed
- Sometimes they return 503
- Sometimes services appear to "freeze" (no logs, no incoming requests)
This is affecting both:
- Frontend (Next.js)
- Backend services (Node.js APIs)
---
## Error messages and descriptions
Common errors observed:
- 503 Service Unavailable
## What I've already tried
- Reduced concurrency (removed Promise.allSettled)
- Added logging to verify if requests reach backend
- Verified services are running (containers are up)
- No crashes or restarts detected
7 Replies
2 months ago
The endpoints that are throwing a 503, would they happen to be proxying anything?
Status changed to Awaiting User Response Railway • about 2 months ago
2 months ago
Yes.
We are using a proxy service that sits between our services.
Example flow:
Service A → Proxy → Service B
And in some cases:
Service A → Proxy → Service B → Proxy → Service C
Status changed to Awaiting Railway Response Railway • about 2 months ago
2 months ago
Noted. In the meantime, you can solve this by proxying via the private network.
Status changed to Awaiting User Response Railway • about 2 months ago
brody
Noted. In the meantime, you can solve this by proxying via the private network.
2 months ago
I´ll try.
Thank you.
Status changed to Awaiting Railway Response Railway • about 2 months ago
Status changed to Awaiting User Response brody • about 2 months ago
2 months ago
Hello,
We have merged a fix for this; no further action is needed on your end.
2 months ago
Hi @brody, jumping in with a related routing issue on our side.
We were getting Cloudflare Error 1000 (DNS points to prohibited IP) , the root cause seems to be proxy/upstream routing, we have two services both with cloudflare domains and one is a proxy for the other.
When the upstream service used our Cloudflare custom domain, we hit Error 1000 intermittently, but after switching upstream to the generated Railway domain, the issue stopped.
Is it possible that a recent change related to the issue here is the cause of this? the cloudflare domains works last I tested, about 24 hours ago, and now I needed to use the Railway domain for it to work.
Status changed to Awaiting Railway Response Railway • about 2 months ago
zvi
Hi @brody, jumping in with a related routing issue on our side. We were getting Cloudflare `Error 1000 (DNS points to prohibited IP)` , the root cause seems to be proxy/upstream routing, we have two services both with cloudflare domains and one is a proxy for the other. When the upstream service used our Cloudflare custom domain, we hit Error 1000 intermittently, but after switching upstream to the generated Railway domain, the issue stopped. Is it possible that a recent change related to the issue here is the cause of this? the cloudflare domains works last I tested, about 24 hours ago, and now I needed to use the Railway domain for it to work.
2 months ago
That isn't related to the OP's issue. Please open your own thread for that.
Status changed to Awaiting User Response Railway • about 2 months ago
Status changed to Solved brody • about 2 months ago