issue with Node/Undici fetch and Railway internal domains
lpnf213
PROOP

24 days ago

Hi Railway team,

We are deploying a project with two services in the same Railway project and region:

  • Frontend: Next.js / Node 20
  • Backend: Flask / Gunicorn, listening on port 5000

We would like to keep the backend private-only, without exposing a public URL, and have the frontend call it through Railway private networking:

BACKEND_URL=http://backend.railway.internal:5000

However, we are seeing unstable/slow behavior from the frontend container when calling the backend private domain.

Inside the frontend container, we installed curl with:

apt-get update && apt-get install -y curl

Then tested the backend private URL:

curl -4 -w "\nTOTAL=%{time_total}s\n" -o /dev/null -s http://backend.railway.internal:5000/api/auth/me

Results vary a lot. Sometimes IPv4 is fast, but often it takes several seconds:

TOTAL=0.007972s

TOTAL=3.483850s

TOTAL=8.417265s

IPv6 is consistently fast:

curl -6 -w "\nTOTAL=%{time_total}s\n" -o /dev/null -s http://backend.railway.internal:5000/api/auth/me

TOTAL=0.006609s

TOTAL=0.006232s

TOTAL=0.001799s

We also enabled this on the frontend service:

NODE_OPTIONS=--dns-result-order=ipv6first

And confirmed inside the frontend container:

echo $NODE_OPTIONS

--dns-result-order=ipv6first

node -p "process.env.NODE_OPTIONS"

--dns-result-order=ipv6first

DNS resolution is correctly returning IPv6 first:

node -e "const dns=require('node:dns'); dns.lookup('backend.railway.internal',{all:true},console.log)"

Output:

null [

{ address: 'fd12:f284:7247:1:e000:2b:ce14:c87d', family: 6 },

{ address: '10.148.200.125', family: 4 }

]

But Node fetch / Undici still fails or times out when calling the internal backend URL:

node -e "const t=Date.now(); fetch(process.env.BACKEND_URL + '/api/auth/me').then(r=>r.text().then(()=>console.log(r.status, Date.now()-t+'ms'))).catch(console.error)"

Output:

TypeError: fetch failed

[cause]: ConnectTimeoutError: Connect Timeout Error

attempted addresses:

fd12:f284:7247:1:e000:2b:ce14:c87d:5000

10.148.200.125:5000

timeout: 10000ms

Important notes:

  • The backend itself is healthy and responds quickly.
  • Calling the backend through its public Railway URL works correctly and solved the issue.
  • Backend -> Postgres is also fast. A simple SQLAlchemy select 1 test from the backend container gives ~8-12ms after the first connection.
  • The issue appears isolated to frontend -> backend over Railway private networking, especially Node/Undici fetch behavior against backend.railway.internal.

Questions:

  1. Is Railway private networking expected to be IPv6-first or IPv6-only for reliable service-to-service traffic?
  2. Is there a known issue with Node/Undici fetch and Railway internal domains?
  3. Is there a recommended way to call private services from a Node/Next.js service without exposing the backend publicly?
  4. Should the backend bind differently for private networking? Currently Gunicorn binds to 0.0.0.0:5000.

Our goal is to remove the backend public URL and keep backend traffic private, but currently the public URL is the only stable workaround.

Could you help us understand the correct private networking setup for this architecture?

$20 Bounty

2 Replies

Railway
BOT

24 days ago

This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.

Status changed to Open Railway 24 days ago


Your frontend sits on the client machine, meaning it can't reach Railway's private networking. Your frontend can only reach backend via public networking.


lleywyn
PROTop 5% Contributor

23 days ago

ignore the "frontend sits on the client machine" answer, thats wrong if both services are deployed in the same project, your next.js server-side fetch runs ON railway and can absolutely use private networking. the real issue is undici + ipv6. your private dns returns ipv6 first then ipv4, and undici tries to connect and hangs/falls back badly, which is why curl-ipv6 is instant but node fetch times out at 10s. couple fixes:

-undici doesnt respect --dns-result-order the way you'd hope, so force it. set the family explicitly with a custom dispatcher/Agent (new Agent({ connect: { family: 6 } })) so it goes straight to ipv6.

-make sure youre on a node fetch that honors happy-eyeballs; older undici versions had nasty ipv6 fallback bugs, so bump undici/node.

-confirm gunicorn is bound to ipv6 ([::]:5000 / --bind [::]:5000), if its only on ipv4 the ipv6 dns answer points nowhere and you eat the full timeout before fallback. this is very likely the actual culprit.

that last one (gunicorn binding) is what i'd check first honestly.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...