Project: binxonia (96cbe6aa-3018-4a09-a206-779b0f566a79)

Service: binxonia-server (49ce556b-08e2-48bb-8b88-2b3d1978d69d)

environment production (9da3b291-893a-4341-a1d4-07190ab84357)

region US East Deployment: 0e629844-0a81-4abc-9669-f144c372f4e7 (SUCCESS, running since 2026-07-17T21:51:10Z with no restarts)

Domains: binxonia-server-production.up.railway.app (Railway domain) and api.binxonia.com (custom, Cloudflare-proxied)

All timestamps below are UTC, 2026-07-18 unless noted.

Summary

This service is a realtime multiplayer game server (Node, ws, long-lived WebSockets). Since at least 2026-07-17 evening, players are disconnected every few minutes. We instrumented both the server and external probes and localized the failure to the Railway edge/ingress -> container leg. Two symptoms, both while the container is demonstrably healthy and serving other traffic in the same second:

1. WebSocket upgrades intermittently rejected with 502 by the Railway edge. Ten occurrences between 15:19Z and 16:22Z from a single probe (one connect attempt every ~5.5s), on BOTH domains — including the plain *.up.railway.app domain with Cloudflare entirely out of the path:

Direct (binxonia-server-production.up.railway.app): 15:19:08, 15:29:37, 15:33:38, 15:34:40, 15:37:14

Via custom domain (Cloudflare passes the origin's 502 through): 15:44:24, 15:47:11, 16:00:19, 16:17:52, 16:22:39

None of these requests appear in the application's logs — they never reached the container. The container was serving other WebSocket handshakes and HTTP successfully at those exact times.

1. Established WebSocket connections killed without close frames (client and server both observe 1006), hitting multiple clients near-simultaneously. Server-side disconnect logs show distinct players, on different residential networks, entering through different Cloudflare edge IPs, dropped within a second of each other:

15:06:51.803Z and 15:06:52.456Z (two different accounts/households, 0.65s apart)

15:25:08.805Z and 15:25:15.937Z (same two accounts, 7s apart)

Sustained churn windows, e.g. 02:37Z-03:29Z: one player involuntarily dropped 13 times (session lengths 19s-799s), every close code 1006 with empty reason

The server process received no signals, logged no errors, and its uptime spans every event above.

Container health during the events (from the app's /health)

uptime ~17.4h, tick p99 = 3ms (50ms budget), snapshotsDropped = 0, RSS 184MB, players 2 — captured 15:11Z, mid-window. CPU/memory nowhere near limits.

Ruled out (to save triage round-trips)

Cloudflare: 502s occur on the raw up.railway.app domain with no Cloudflare involved; simultaneous drops hit clients entering via different CF edges.

The application: the 502'd upgrades never reach it; it stays healthy and serving throughout; no backpressure (0 dropped snapshots), no tick stalls.

Client networks: the simultaneous 1006 drops are recorded server-side across distinct households.

Reproduction

Open a WebSocket to either domain every ~5.5s and log any response other than a successful upgrade (our app closes unauthenticated sockets cleanly with 1008 after 5s, which makes a clean baseline). During bad windows roughly 1 in 40 upgrades gets a 502. Happy to share the probe script and raw logs.

Possibly related prior incident

2026-07-15: the container for this same service was restarted multiple times inside a running deployment (no new deployment; e.g. SIGTERM at 2026-07-15T18:52:45Z with players connected; three restarts in under 2 hours). Deployment IDs available on request.

Asks

Can you check edge/ingress health for this service's upstream in US East during the windows above? The upgrade 502s point at edge->container connection failures while the container is fine.

If a specific host or proxy instance is implicated, can the service be migrated/rescheduled?

Is there a known issue explaining both the in-deployment restarts (07-15) and the ingress instability (07-17/18)?