a month ago
Hi Railway team,
We're seeing intermittent TLS handshake failures at the edge for services in our project, affecting an external integration calling our auth API. The pattern matches previously reported platform-side edge issues on this forum (e.g. station.railway.com/questions/intermittent-522-connection-timed-out-556e8e65, where Railway staff confirmed "a platform-side edge problem").
Account / project:
Workspace: AI Parser
Project: lively-embrace
Project ID: 6f0532b0-5fc4-478d-a505-9b4717f3c3c1
Environment: production (ID 119b1f73-cec7-4292-aa46-35c2b4ecc0dd)
Region: sfo
Affected services (both in this project, both region sfo):
SecondOpinion — https://secondopinion-production.up.railway.app — resolves to 69.46.46.32
benchmark-admin — https://benchmark-admin-production-c304.up.railway.app — resolves to 69.46.46.82
Symptom:
TCP connection consistently succeeds in ~0.2s. TLS handshake intermittently hangs until client timeout ("SSL connection timeout" / no bytes received), while at other times it completes normally in 0.4–0.7s and the app responds correctly with HTTP 200. Failure rate fluctuates in waves — we measured ~40–50% failures in one test window, then 0/10 failures a few minutes later on the identical endpoint.
Evidence this is not our application or our network:
Application-side logs (SecondOpinion, 2026-07-22) show zero restarts, zero errors, and 1–35ms internal handling time for every request that reaches it — including /health/ready, which touches Postgres, Redis, and object storage (300–500ms). Example from our deploy logs (UTC):
2026-07-22T07:11:54 — Application startup complete, Uvicorn running on http://0.0.0.0:8080
2026-07-22T08:10:00 — GET /health/live -> 200 (33ms internal)
2026-07-22T08:10:10 — GET /health/ready -> 200 (502ms internal, includes DB+Redis+S3 checks)
2026-07-22T08:10:13 — GET /openapi.json -> 200 (83ms internal)
No process restarts occur between these entries and the present. The app is never the bottleneck for requests that arrive.
A second, independent service on the same account/region (benchmark-admin) shows the identical intermittent TLS-hang pattern at the same time, ruling out an app-specific cause:
adm try1: tls=3.19s -> HTTP 401 (reached app, just slow TLS)
adm try2: tls=1.61s -> HTTP 401
adm try3: tls=hang -> 8s timeout, HTTP 000
adm try4: tls=3.27s -> HTTP 401
Control requests over the same network window to https://www.google.com, https://www.cloudflare.com, and https://status.railway.com succeeded 100% of the time (12+ requests, TLS 0.16–0.38s), ruling out a client-side network or DNS issue.
Timing example for secondopinion-production.up.railway.app/health/live (curl -w breakdown, single contiguous test run, 2026-07-22):
try1: tcp_connect=0.46s tls=hang -> 15s timeout, HTTP 000 "SSL connection timeout"
try2: tcp_connect=0.20s tls=0.68s -> HTTP 200, total 1.06s
try3: tcp_connect=0.19s tls=hang -> 15s timeout, HTTP 000
try4: tcp_connect=0.21s tls=hang -> 15s timeout, HTTP 000
try5: tcp_connect=0.21s tls=0.43s -> HTTP 200, total 0.86s
Reproduced independently over both IPv4 and IPv6 against the same hostname on the same date.
Prior occurrence: on 2026-07-21, our integration also observed GET /health/live, GET /health/ready, and GET /openapi.json timing out repeatedly at the 10s client timeout with HTTP 000 (no response received), while background Celery workers in the same project (crm-system, crm-beat) continued executing tasks successfully throughout — consistent with the edge/web path being affected while backend workers on private networking were unaffected.
Request:
Could you check edge/proxy health and network path status for the sfo region around these windows (2026-07-21 and 2026-07-22), for project 6f0532b0-5fc4-478d-a505-9b4717f3c3c1 (services SecondOpinion and benchmark-admin)? We'd also appreciate guidance on:
Whether this matches a known ongoing edge issue in sfo.
Whether moving to a different region and/or adding a second replica would mitigate exposure to this class of edge instability.
Thanks!
1 Replies
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 30 days ago
a month ago
Which edge (not metal) location are you connecting to? On a browser you can view it here: https://status.railway.com/locations
Information about your internet service provider (ISP) would be helpful as well
I personally wouldn't rule out the client network just yet, I'd recommend running HTTP tests to e.g. Google and Railway at the same time for ~5 minutes
Status changed to Solved chivkunovd-bitdenis • 30 days ago