Outbound HTTPS streams to AWS S3 stall mid-transfer approximately 1/300 requests
matthewsmkim1
PROOP

4 hours ago

Service: Node.js Express + Apollo GraphQL

Issue: Outbound HTTPS streams to AWS S3 stall mid-transfer approximately 1/300 requests

What happens:

  1. GetObjectCommand is sent to S3 — succeeds, headers received with HTTP 200
  2. Stream body begins transferring (~100kb file)
  3. Stream silently stalls mid-drain — no error, no RST, no FIN, just stops delivering data
  4. Hangs indefinitely until something kills the connection (~4 minutes)

What we've ruled out:

  • Not an S3 issue — never happens when the same code runs locally
  • Not a code issue — headers arrive fine, only the body stalls
  • Not a retry issue — the AWS SDK sees a 200 and considers the request successful, so it never retries
  • Only affects S3 (outbound HTTPS streaming) — Postgres and Redis connections on the same service are stable

Key detail:

The failure is completely silent — no error event is raised on the stream, no exception is thrown. The stream just stops emitting data events mid-transfer and never emits end. This suggests something in Railway's network layer is dropping packets mid-stream without closing the connection, rather than actively terminating it.

Suspected cause:

Railway's egress network (NAT gateway or outbound proxy) is silently dropping packets mid-stream on outbound HTTPS connections approximately 1/300 times. Because no RST or FIN is sent, the TCP connection appears open but data stops flowing.

Questions:

  1. Is there a known idle or mid-stream timeout on outbound connections from Railway containers?
  2. Is outbound traffic routed through a proxy or NAT gateway that could be dropping packets?
  3. Are there any known issues with outbound HTTPS streaming to AWS S3 specifically?
Awaiting Railway Response

0 Replies

Status changed to Awaiting Railway Response Railway about 4 hours ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...