Intermittent outbound TCP timeout from Railway egress to 92.223.17.0/24 (Wargaming Asia API)

a month ago

Hi, I'm running a Next.js Node service on Railway (Hobby project, GCP region) and I'm experiencing reproducible intermittent TCP connect timeouts from inside the container to a specific destination subnet, with no equivalent issues for neighboring subnets owned by the same upstream ASN.

The problem is consistent enough to break our production page rendering whenever a fetch lands during a "down" window (about 50% of attempts during the ~5 min outage cycles).

Observed behavior:

Destination affected: 92.223.17.174:443 (api.worldoftanks.asia) and 92.223.29.12-14:443 (asia.wargaming.net)

Failure mode: TCP connect() never completes, undici reports UND_ERR_CONNECT_TIMEOUT after 10s

Cycle: ~2–5 min UP, then ~2–5 min DOWN, repeats

Failure rate during DOWN windows: 100% (5/5 sequential tests timeout)

What I've tested:

Tested with both Standard Static IP 208.77.244.15 and dynamic egress (34.34.54.205, GCP block) — both affected the same way

Same /16 neighbors work fine throughout: 92.223.24.25 (EU), 92.223.101.116 (NA), 92.223.22.65 (RU) — all respond in <100ms

Reproduced via raw net.connect() and undici fetch() from inside the deployed container

Same destinations reachable instantly from my home ISP and other clouds → not a Wargaming-side issue

HA Static IPs (3-IP system) had the same problem, rollback to Standard didn't help, disabling Static IPs entirely didn't help

Conclusion: looks like a BGP route or peering problem between Railway's upstream transit and the AS that announces 92.223.17.0/24 and 92.223.29.0/24 (Wargaming's Asia infrastructure). Other subnets they announce are fine, so it's likely a specific route flap.

Ask: could you check with your network team if there's a known degraded route to those subnets? Reproducible right now with:

curl -sv --max-time 12 https://api.worldoftanks.asia/ -o /dev/null from any of your scheduled hosts.

Happy to provide more traceroute/mtr data if useful.

Solved

9 Replies

a month ago

Could you try to redeploy so you land on our own metal infrastructure?- we can't control internet traffic from the GCP AZs.


a month ago

If you don't land on a Railway machine, please link me your deployment so I can pin it to Railway metal


a month ago

Hi Phineas, thanks! I just triggered a redeploy but I landed on GCP again (egress IP 34.158.188.24, was previously 34.34.54.205). ASIA destination (92.223.17.174:443) still times out 5/5 from this new IP.

Deployment URL: https://railway.com/project/ebe8220e-058c-4d58-ac11-d3d0d26828f4/service/410d6abc-1e46-4b61-a1a9-f4f186fe4cd0?environmentId=a6e158f4-f442-42a3-8b00-68067af6bce1 (@deployment:886a8766-f117-4f30-9322-2bee6ade79a0)

Could you pin it to Railway metal infrastructure please? I'll re-test as soon as it's repinned.


a month ago

Done - you are now on a Railway metal host machine.


a month ago

Thanks for the metal pin Phineas! Did a fresh test:

Egress IP 152.55.184.106 (metal confirmed)

First batch of ASIA fetches: 5/5 OK

30s later: 5/5 timeout to 92.223.17.174:443

Did a traceroute from inside the container: it reaches 92.223.29.1 (WG Asia edge) in 154ms via Arelion (62.115.x.x), with several asymm hops indicating asymmetric return path. From my local ISP (Orange FR → Hurricane Electric), TCP succeeds reliably to the same destination.

So it looks like a return-path/peering issue between Arelion and Wargaming Asia's AS, not Railway-fixable i guess??? Thanks anyway for the investigation!


a month ago

You could try deploying to Singapore, their routing is probably optimized for that region. But it sounds like it's out of our control considering it's happening on both GCP and Railway metal; those are two entirely separate egress paths.


a month ago

ICMP is consistently successful with that address, so it sounds like some sort of rate limiting or blacklisting behavior from their network.

Our view:

drams3a-production-border-sw-002#sh ip bgp 92.223.29.1
BGP routing table information for VRF default
Router identifier 172.20.34.11, local AS number 400940
BGP routing table entry for 92.223.29.0/24
 Paths: 2 available
  1299 199524
    10.255.6.254 from 10.255.6.254 (172.20.34.10)
      Origin IGP, metric 0, localpref 100, IGP metric 1, weight 0, received 195d22h ago, valid, internal, best
      Rx SAFI: Unicast
  2914 3491 199524
    81.20.64.14 from 81.20.64.14 (129.250.0.42)
      Origin IGP, metric 0, localpref 100, IGP metric 1, weight 0, received 96d18h ago, valid, external
      Community: 2914:420 2914:1206 2914:2203 2914:3200 3491:400 3491:412 3491:9080 3491:9081 3491:9084 3491:9087 3491:9089 3491:62210 3491:62220
      Rx SAFI: Unicast```

a month ago

Thanks for digging into the BGP! Very insightful. I noticed your "best" path uses Arelion (AS1299), which is exactly what my traceroute showed (62.115.x.x hops with asymmetric returns). The alternative NTT/PCCW path (AS2914→AS3491) probably wouldn't have the same issue but it's not preferred.

Update from my side: I followed your suggestion and switched the deployment region to US West. It now routes via AWS → China Mobile International (AS58453) → Wargaming, which is a third path entirely. Tested 18 sequential requests over 1 minute = 18/18 success, latency 570-762ms, no flap.

So the issue is definitely the Arelion-to-WG-Asia peering, not anything Railway-fixable on this specific subnet. Just wanted to confirm in case it helps future tickets about Wargaming Asia or similar Asian destinations. Thanks again!


We're aware of intermittent outbound connectivity drops affecting some services, where outbound connections fail while internal networking and DNS keep working, and we're actively investigating. If you hit it, the quickest workaround is to redeploy, which moves your service onto a fresh host and restores connectivity. If it happens again, reply here with the rough time it started and we'll jump on it while it's live. Those real-time reports are exactly what help us track down the cause.


Status changed to Awaiting User Response angelo-railway 24 days ago


Railway
BOT

17 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 17 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...