iOS clients lose the connection right after upload; backend completes every request (edge lax1, since Oct 20)

15 hours ago

Service: ctvd-recognizer (49d2fc23-80f1-4069-8ad3-9ed355c82ae9) in project protective-delight (b1c7d2c0-6efd-4128-acd4-4f0975cc6e16), environment production, region us-west2, Hobby plan, public domain api.cubesnap.app. Response headers show server: railway-hikari and x-railway-edge: lax1.

Symptom: since about 2026-10-20, a growing share of requests from our iOS app (Apple URLSession, HTTP/2) to POST /api/recognize (multipart body of 400 to 800 KB, JSON response after about 2 s) fail on the phone with NSURLErrorNetworkConnectionLost (-1005). Our backend receives the full body and processes every one of these requests: we can match each client failure to a backend event 0 to 3 s later, with normal latency (p50 1.9 s, p90 2.7 s; the lost requests are not the slow ones). So the client is cut off right after the upload completes, before the response is written. In two cases the backend got BrokenPipeError while writing the response, at 2026-10-25T00:48:19Z (peer 100.64.0.23:27714 as seen by the container). Android (OkHttp) and browser clients show zero such failures in the same period (19 Android sessions, 56 web devices since 10-25), and the iOS networking code has not changed.

Affected iOS requests per day: 10-15 to 10-19: 0, 1, 1, 2, 0. 10-20: 4. 10-21: 7. 10-22: 8. 10-23: 7. 10-24: 5. 10-25: 12 (8 devices out of 43 sessions). 10-26: 14 by 16:00 UTC (5 devices out of 36 sessions).

Example timestamps (UTC): one device failed 8 consecutive times between 2026-10-26 01:17:58 and 01:20:30, all 8 processed by the backend. 2026-10-25 00:48:14 and 00:48:20 are the pair that produced the broken-pipe tracebacks.

Questions: did the edge in front of this service change around Oct 20 (hikari rollout or incident remediation) in a way that affects HTTP/2 request bodies from Apple's URLSession? Can you check edge logs for this service at the timestamps above for stream resets, GOAWAY, or connection closes toward the client? Happy to provide more timestamps or run a test build.

$10 Bounty

2 Replies

Railway
BOT

15 hours ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway • about 15 hours ago


sundayabasi684-byte
FREE

14 hours ago

Railway edge HTTP/2 RST_STREAM/GOAWAY after multipart upload: backend processes request, client disconnects before response, BrokenPipeError while writing response, Apple URLSession NSURLErrorNetworkConnectionLost (-1005), x-railway-edge lax1, railway-hikari

The request body is fully uploaded and the backend processes it normally, but the client loses the connection immediately after upload completes. The server logs BrokenPipeError while writing the response, which indicates the peer closed the connection before the response was delivered. This matches an edge/proxy HTTP/2 stream reset or connection close rather than a backend business-logic bug.

Affected requests are iOS Apple URLSession over HTTP/2; Android/browser are not seeing the same issue. The issue started around 2026-10-20 and increased afterward. Response headers show server: railway-hikari and x-railway-edge: lax1.

This is consistent with:

HTTP/2 stream reset

RST_STREAM

GOAWAY

edge-side connection close

upstream response write failure after client disconnect

Relevant timestamps:

2026-10-25T00:48:14Z

2026-10-25T00:48:20Z

2026-10-26 01:17:58 to 01:20:30 UTC

Expected: the client receives the response after upload completes. Actual: the connection is lost before the response is delivered.

This is likely an edge / proxy HTTP/2 issue on the Railway path, not an application bug in /api/recognize.


sundayabasi684-byte
FREE

2 hours ago

Have you tried this


Welcome!

Sign in to your Railway account to join the conversation.

Loading...