Intermittent new-connection failures at edge (sin1) since Oct 1 — instance healthy
jimitshah7
HOBBYOP

9 hours ago

Since ~04:40 UTC on 2026-10-01, our production service zone6.in suffers recurring windows where new connections fail to establish at the edge while the instance stays healthy. Fresh connections still show degradation as of 2026-10-05 ~06:43 UTC (details below).

Evidence (2026-10-01): On one held-open keep-alive connection, 36/36 requests succeeded 05:41:05–05:51:10 UTC (request IDs e.g. QbKj4UrXQBeoUmOFmrpb1w, CtQaLn29QReLGtMN-8Y8hA; traces sin1.98a6,hnd1.y5sc/df23/2zrd/4cm0/sx1d and earlier sin1.98a6,ams1.aydy) — while fresh connections to our custom domain AND both of its Railway LB endpoints intermittently timed out at TLS handshake with no x-railway-request-id (05:49:22, 05:49:37, 05:50:14, 05:50:23 UTC; also a 05:37:43–05:39:30 window). A redirect request took 8.9s (05:47:18) and an edge-served 404 took 10.83s (05:48:47) — pure edge/mesh time, no app involvement. Control site railway.app blipped at the same timestamps (05:42:11: 5.64s; 05:47:16: 4.19s) but never hard-failed.

Current status (2026-10-05 06:42–06:44 UTC): no hard TLS failures in an 80s sample, but fresh-connection /r/* redirects take 1.1–2.4s (3/3) and /api/health spikes to 2.3–2.8s (median ~0.5s) — consistent with the same impairment in milder form.

Instance info: service deployed in Southeast Asia (Singapore); database in EU West; serverless disabled on both; no restarts during the windows — the affected deployment's container started 2026-09-30T14:11:59Z and has run continuously since (background jobs logged through Oct 5; 1-month CPU/RAM metrics normal — screenshot attached).

Questions: (1) For a Singapore-deployed service, are hnd1/ams1 handlers expected, or is our instance's mesh/handler placement degraded? (2) Can you check edge/handler logs for dial failures to our instance during those windows? Happy to redeploy for host migration if advised — is a 2nd replica the recommended mitigation?

$10 Bounty

0 Replies

Railway
BOT

9 hours ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway • about 9 hours ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...