Outbound connections to googleapis.com failing — egress IP likely blocked by Google
victorabs23
PROOP

18 days ago

Project: OPERACIONAL (ID: 8361d9aa-1bb3-4377-89be-ca0132fee8fe)

Services affected: operacional_backend (prod) AND serene-achievement (homolog), region US East.

Symptom: every outbound call to https://www.googleapis.com/oauth2/v4/token fails with

"Invalid response body while trying to fetch https://www.googleapis.com/oauth2/v4/token: Premature close"

(connection dropped mid-response). ~100% failure since today, on BOTH environments simultaneously.

Evidence it's the egress path, not our app:

  • The exact same request succeeds instantly (HTTP 400) from outside Railway (local machine).
  • All other outbound traffic from the containers works fine (Postgres, object storage, HTTP serving) — ONLY

googleapis.com is affected.

  • No app/dependency change; both environments run different code but fail identically.
  • Forcing IPv4 DNS (NODE_OPTIONS=--dns-result-order=ipv4first) did not help.

This matches the known pattern of Google blocking shared Railway egress IP ranges.

Request: please check our outbound IP reputation and move us to a clean egress IP (or advise on Static Outbound IPs /

proxy).

Solved

5 Replies

Railway
BOT

18 days ago

Since you're on the Pro plan, you can enable Static Outbound IPs on each affected service right now - go to the service's Settings tab, toggle "Enable Static IPs" under Networking, and redeploy. This will route your outbound traffic through dedicated IPs rather than the shared egress pool, which should resolve the Google-side blocking. Note that the static IPs may still be shared with other customers, but in practice they carry far less traffic and a cleaner reputation than the default egress range.


Status changed to Awaiting User Response Railway 18 days ago


victorabs23
PROOP

18 days ago

Static IPs didn't help — still Premature close on googleapis.com from the static IP

pool (162.220.234.x / 152.55.180.x). Need a clean egress IP or guidance.


Status changed to Awaiting Railway Response Railway 18 days ago


joshuachestang
PRO

17 days ago

Having this same issue!


victorabs23

Static IPs didn't help — still Premature close on googleapis.com from the static IP pool (162.220.234.x / 152.55.180.x). Need a clean egress IP or guidance.

joshuachestang
PRO

17 days ago


Good news: this isn't an egress block, and there's nothing to fix on the network side — your outbound path to Google is healthy. Another user hit the identical "Premature close" on the same Google token endpoint and traced it: from inside the container, Node's built-in global fetch reaches https://oauth2.googleapis.com/token fine, while the request fails only when it goes through node-fetch. Firebase Admin / google-auth-library mint their OAuth token via node-fetch, which is why every Google token exchange breaks while your Postgres, storage, and HTTP serving keep working. That also explains why static IPs, IPv4-first DNS, and a region change didn't help — they were aimed at a network problem that wasn't there.

Two things to try:

  1. Mint the token using Node's global fetch (undici) instead of node-fetch. The other thread has a working snippet that keeps firebase-admin's cert() credential and overrides only getAccessToken() to use global fetch: https://station.railway.com/questions/outbound-https-to-oauth2-googleapis-com-086171e4
  2. This likely started suddenly because a base image like node:20-bookworm-slim is a moving tag, so a rebuild pulls the latest Node 20.x patch — a change to Node's HTTP/1.1 handling can break node-fetch while leaving undici unaffected. Pinning a specific Node minor/patch (or a digest) prevents a repeat.

If you'd rather not change the client, upgrade/swap the library doing the token exchange to one built on undici/global fetch. Happy to help wire up the override.


Status changed to Awaiting User Response Railway 17 days ago


Railway
BOT

10 days ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway 10 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...