HTTPS response-header timeouts to FRED from Singapore region
kimsungho93
PROOP

8 days ago

Hello Railway Support,

Our Python service cannot receive HTTP response headers from FRED on Railway, while the same keyless API request receives an HTTP response from a local Windows PC.

Environment

  • Plan: Pro
  • Region: Singapore (asia-southeast1-eqsg3a)
  • Runtime: Python 3.12, Docker

Observed behavior

Our scheduled FRED collection failed on September 27, 2026, around 04:01-04:03 UTC. All three series failed while waiting for HTTP response headers, with no HTTP status received.

We then tested this URL without an API key:

https://api.stlouisfed.org/fred/series/observations?series_id=DGS10&file_type=json&limit=1

  • Local PC: HTTP 400 received in approximately 0.4 seconds.
  • Railway: TLS connected in approximately 0.06 seconds, but reading the HTTP response timed out after 15 seconds.
  • Explicitly setting ALPN to HTTP/1.1 on Railway produced the same timeout.
  • FRED's public documentation page also timed out after TLS connected:

https://fred.stlouisfed.org/docs/api/fred/series_observations.html

These diagnostic requests used Python's http.client.HTTPSConnection with certificate verification enabled. No API keys were included, and no retry loop was used.

Could you help us check:

  1. Whether there is an outbound routing or connectivity issue from this service to these hosts on port 443.
  2. Whether destination-side filtering of the service's egress IP could explain this behavior, and what evidence we should provide to FRED.
  3. What additional low-impact diagnostic you recommend.

Please do not change the region or outbound IP without confirmation, as other integrations depend on IP allowlists.

Thank you.

$20 Bounty

6 Replies

Railway
BOT

8 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway • 8 days ago


kimsungho93
PROOP

8 days ago

Railway team: This question was accidentally submitted publicly. I intended to contact Pro private support. I have removed the infrastructure IDs from the current post, but creating a private replacement is blocked by the duplicate-thread check.

Could you please move this case to private support and remove any publicly accessible original/edit history? If conversion is not possible, please remove this public thread and advise how I can submit privately. Thank you.


eugenoprea
PRO

8 days ago

A successful TLS handshake narrows this down, but doesn’t prove that FRED is blocking your IP. It also doesn’t rule out a network failure after the handshake.

The next useful check is whether Railway and your local PC reach the same destination IP. Different DNS answers could send them to different FRED/CDN endpoints.

If curl is available, run this once from the affected container:

curl -q -4 --http1.1 --connect-timeout 5 --max-time 15 \
  -sS -o /dev/null \
  -w 'destination=%{remote_ip} status=%{http_code} tcp=%{time_connect} tls=%{time_appconnect} first_byte=%{time_starttransfer} total=%{time_total}\n' \
  'https://api.stlouisfed.org/fred/series/observations?series_id=DGS10&file_type=json&limit=1'

Run the same command locally using curl.exe, on one line, with /dev/null replaced by NUL.

If the destination IPs differ, repeat with this option added to both commands, substituting the IP from the successful local test:

--resolve api.stlouisfed.org:443:DESTINATION_IP

This preserves the hostname, TLS SNI, and certificate verification while fixing the destination address. curl documentation

  • Railway succeeds with the pinned address: that points toward a difference between destination endpoints or the paths to them.
  • Railway still times out while local succeeds against that same address: source-IP filtering or a source-dependent network path remains plausible. Railway/FRED logs would be needed to distinguish them.

Also, if you already use Railway’s static outbound IPs, include all assigned addresses when reporting this privately. Railway can balance outbound connections across multiple IPs, so an IP-check request may show a different source address from the failed FRED request. Railway documentation

These tests require no API key, retries, or changes to your service’s region or outbound-IP configuration.


kimsungho93
PROOP

8 days ago

Thanks eugenoprea and larrybol. A keyless test to the same destination IP returned HTTP 400 from my PC in 0.34s; Railway completed TLS in 0.31s, then timed out waiting for headers at 15s. Different DNS/CDN addresses alone do not explain it. Source-IP filtering or a source-dependent path remain possible. We shared the egress IP privately with FRED. No key, retries, or config changes.


eugenoprea
PRO

8 days ago

That comparison narrows it down: different DNS answers alone don’t explain the failure.

Since you’ve already contacted FRED, the useful next step is to correlate one failed attempt using its exact UTC timestamp, destination IP, request path, and possible egress IPs. Ask them to check their edge/security logs as well as application logs:

  • Did the HTTP request arrive after TLS completed?
  • Was it blocked or dropped by a security rule?
  • Was it forwarded upstream, and was a response sent?

An absent application-log entry wouldn’t by itself establish a Railway networking problem: the request could have stopped at FRED’s edge before reaching the application.

If FRED confirms it sent a response, that gives Railway a specific timestamp and destination to investigate for a delivery failure. If FRED identifies a filtering rule, they can assess an exception for your existing egress addresses.

I’d keep the current configuration unchanged while those logs are checked; another identical timeout test is unlikely to distinguish the remaining possibilities.


eugenoprea

That comparison narrows it down: different DNS answers alone don’t explain the failure. Since you’ve already contacted FRED, the useful next step is to correlate one failed attempt using its exact UTC timestamp, destination IP, request path, and possible egress IPs. Ask them to check their **edge/security logs as well as application logs**: - Did the HTTP request arrive after TLS completed? - Was it blocked or dropped by a security rule? - Was it forwarded upstream, and was a response sent? An absent application-log entry wouldn’t by itself establish a Railway networking problem: the request could have stopped at FRED’s edge before reaching the application. If FRED confirms it sent a response, that gives Railway a specific timestamp and destination to investigate for a delivery failure. If FRED identifies a filtering rule, they can assess an exception for your existing egress addresses. I’d keep the current configuration unchanged while those logs are checked; another identical timeout test is unlikely to distinguish the remaining possibilities.

kimsungho93
PROOP

8 days ago

Thanks, @eugenoprea. I ran one keyless test from a short-lived Railway US West (SFO) Sandbox, pinned to the same destination IP (104.75.39.52). It returned HTTP 400 in 0.738s. The Singapore production container completed TLS but received no HTTP bytes within 15s. We still need the logs to distinguish filtering from a source-dependent network-path issue.

I destroyed the Sandbox and left the production configuration unchanged. I sent FRED the comparison and the earlier UTC failure window through their private contact form, asking them to check the edge/security and application logs you mentioned. Thank you for the careful guidance. It helped us narrow down the next check.


kimsungho93

Thanks, @eugenoprea. I ran one keyless test from a short-lived Railway US West (SFO) Sandbox, pinned to the same destination IP (104.75.39.52). It returned HTTP 400 in 0.738s. The Singapore production container completed TLS but received no HTTP bytes within 15s. We still need the logs to distinguish filtering from a source-dependent network-path issue. I destroyed the Sandbox and left the production configuration unchanged. I sent FRED the comparison and the earlier UTC failure window through their private contact form, asking them to check the edge/security and application logs you mentioned. Thank you for the careful guidance. It helped us narrow down the next check.

eugenoprea
PRO

8 days ago

Thanks for following through. The SFO result confirms that FRED is reachable from at least one Railway location, so this isn’t a blanket failure across Railway. It still leaves the Singapore egress address and network path as variables.

You’ve sent FRED the useful comparison. I’d wait for their findings before running more tests or changing production. If they confirm the Singapore request was answered, that would give Railway a concrete failure window to investigate on the return path.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...