2 months ago
Hello Railway team,
My production service in the sfo region is experiencing outbound TCP connection timeouts to two endpoints of an external provider.
Diagnostic findings:
- Both endpoints time out on TCP port 443 from the Railway container.
- The same endpoints resolve to the same IP addresses and connect successfully from my local computer.
- A control connection to example.com:443 succeeds from the container in approximately 0.02 seconds.
- The failure reproduces with Python's standard socket library, before TLS or authentication.
- No proxy environment variables were found.
- Restarting the existing deployment did not resolve the issue.
- The web application remains online and responds with HTTP 200.
- The application worked two days ago, with no recent code changes or new deployments.
We have not established whether the cause is destination-side filtering, routing, or another network issue.
For confidentiality, I am omitting the provider's identity, destination domains/IPs, and project identifiers from this message.
Could a Railway team member provide a private channel where I can share those details and request a targeted connectivity investigation? Please confirm how to provide them without making them visible to the community.
Thank you.
1 Reply
2 months ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 2 months ago
2 months ago
This looks much more like a destination-specific egress/IP filtering or routing problem than a general Railway outbound-networking failure.
The key evidence is:
- TCP/443 to
example.comsucceeds from the same container. - Only two endpoints belonging to the same external provider fail.
- The failure occurs during
connect(), before TLS, HTTP, or authentication. - The same destination IPs are reachable from a non-Railway network.
That eliminates application code, TLS configuration, credentials, and a blanket Railway TCP/443 restriction.
The next thing I would test is the Railway egress IP/path.
From the affected deployment:
curl -4 https://ifconfig.meor:
python - <<'PY'
import urllib.request
print(urllib.request.urlopen("https://api.ipify.org", timeout=5).read().decode())
PYRecord that IPv4 address, then ask the external provider whether that IP or its ASN/range is blocked, rate-limited, reputation-filtered, or absent from an allowlist.
Railway specifically provides Static Outbound IPs for integrations where external providers filter or allowlist source IPs. If this service is on Pro, I would enable Static Outbound IPs, redeploy, and retest:
railway outbound-network static-ip enable --service <service>
railway outbound-network static-ip status --service <service>Then redeploy the service before testing, since Railway documents that the new outbound IP assignment only takes effect after redeployment.
There are two additional tests that can identify the responsible side very quickly.
1. Test the same destination from another Railway region
Deploy the same minimal socket test temporarily in another region.
import socket, time
targets = [
("DESTINATION_IP_1", 443),
("DESTINATION_IP_2", 443),
("93.184.216.34", 443), # control
]
for host, port in targets:
start = time.monotonic()
try:
with socket.create_connection((host, port), timeout=5):
print(host, "CONNECTED", round(time.monotonic() - start, 3))
except Exception as exc:
print(host, type(exc).__name__, repr(exc),
round(time.monotonic() - start, 3))Interpretation:
- Fails from current region, works from another Railway region: likely path/peering or source-range-specific filtering.
- Fails from multiple Railway regions but works locally: strongly suggests the provider is filtering Railway/cloud-hosting source IPs or ASNs.
- Works after changing to a Static Outbound IP: almost conclusive evidence that the previous Railway egress address/range was being filtered somewhere downstream.
2. Compare TCP behavior against several unrelated destinations
Test perhaps Cloudflare, Google, GitHub, and the affected provider:
nc -vz -w 5 1.1.1.1 443
nc -vz -w 5 github.com 443
nc -vz -w 5 google.com 443
nc -vz -w 5 <affected-ip-1> 443
nc -vz -w 5 <affected-ip-2> 443If unrelated destinations consistently connect while only that provider times out, I would not classify this as a general Railway egress outage.
A timeout rather than ECONNREFUSED is also consistent with a firewall silently dropping SYN packets somewhere between Railway's egress and the destination. It does not by itself tell us which side is dropping them.
For a Railway-side investigation, the most useful private data would be:
- project/service/deployment ID
- Railway region
- exact UTC timestamps of several failed TCP attempts
- destination IPv4 addresses and ports
- current Railway outbound IPv4 address
- results from another Railway region, if possible
- whether Static Outbound IP changes the result
Those data should allow Railway to check the relevant egress/NAT host and upstream path without exposing the provider publicly.
My first practical test would therefore be:
enable a Static Outbound IP → redeploy → retest → give that IP to the external provider for allowlisting/checking.
That test has much more diagnostic value than repeatedly restarting the existing deployment.