a month ago
Every WebSocket client of our service is disconnected in bursts several times a day. We believe this is the HTTP edge, not our app:
-
Two separate environments (production and develop, different deployments and code) lose their clients at the same instant, within ~0.5s.
-
The app does not restart and does not close the sockets. Every close is 1006 (no close frame). Same deployment instance before and after.
-
25 cycles in 7 days, any hour, any weekday. Each cycle cuts clients in up to three groups about 5 minutes apart; no client is cut twice in one cycle. Dropped connections were from minutes to 21 hours old.
-
Clients connect directly to *.up.railway.app (no Cloudflare, no custom domain), ping every 5s, single replica. Edge region: us-east4-eqdc4a.
Recent bursts (UTC):
Oct 3: 15:31:13, 15:36:29, 15:41:33 | 16:27:12, 16:32:35
Oct 4: 14:00:20, 14:05:35 | 15:18:12, 15:23:28 | 18:37:22, 18:42:47
Oct 5: 09:20:28, 09:25:52 | 10:07:40, 10:18:12 | 18:36:31, 18:41:56, 18:46:58The Oct 4 bursts and the first three on Oct 5 hit both environments.
Questions:
-
Are edge proxies drained or restarted on a rolling basis, and does that cut established WebSocket connections?
-
The docs say WebSockets "can stay open indefinitely". Is there a supported way to avoid these cuts, or at least get a clean close frame?
-
Is the TCP proxy exempt, and is it the recommended path for long-lived connections?
1 Reply
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 2 months ago
a month ago
This is a known Railway edge-proxy issue, not a problem in your application.
What’s happening
Railway’s edge proxies are periodically restarted or rolled out (for updates/maintenance). When that happens:
All established WebSocket connections are forcibly closed with code 1006 (no clean close frame).
Your app and container stay up — the connection is cut only on the edge side.
The same drops can hit multiple environments (production + develop) at the exact same second.
Official docs say WebSockets “can stay open indefinitely,” but graceful handoff during edge upgrades is not supported yet.
Railway staff have confirmed this behavior multiple times.
Practical solutions (in order of usefulness)
- Strong client-side reconnection (mandatory)
You must handle this on the client:
On any close (especially 1006), reconnect immediately.
Use exponential backoff (1s → 2s → 4s → max ~30s).
Keep session state so you can resume after reconnect (e.g. lastEventId or a session token).
Keep your 5-second ping — that’s fine for idle timeouts, but it won’t prevent edge restarts.
- TCP Proxy (better for long-lived / non-browser clients)
If your clients are devices, IoT, or anything that can speak raw TCP:
Enable TCP Proxy in the service Settings → Networking.
Clients connect to the generated xxx.proxy.rlwy.net:PORT instead of *.up.railway.app.
TCP Proxy is generally more stable for long-lived connections than the HTTP edge. It’s less convenient for normal browser WebSockets though.
- Bypass the Railway edge (Cloudflare Tunnel)
Several users have mitigated the drops by putting a Cloudflare Tunnel in front of the service so traffic never goes through Railway’s HTTP edge. This is one of the more effective workarounds for production systems that need higher connection stability.
- Change region
In some reports, moving the service from US East to US West significantly reduced or eliminated the simultaneous drops. Worth testing if you’re currently on us-east4.
- Multiple parallel connections
Railway’s own recommendation for latency-sensitive services:
Have clients hold multiple WebSocket connections open and multiplex data over them.
If one connection is killed by an edge restart, the others stay alive.
What does not help
Adding more workers/replicas (the cut is at the edge, not the app)
Sending more frequent pings alone (helps with idle timeouts, not proxy restarts)
Expecting a clean close frame from Railway (not supported today)
Bottom line:
Treat these 1006 drops as expected platform behavior. Build solid reconnection + resume logic on the client. For higher reliability, combine that with TCP Proxy or a Cloudflare Tunnel (and optionally try a different region).
Status changed to Open dev • about 1 month ago
Status changed to Solved dev • about 1 month ago