Network healthcheck failure
mickeywgla
HOBBYOP

12 days ago

Hi all,

I've been trying to redeploy my app onto railway but it is constantly failing the network healthcheck.

Attempt #14 failed with service unavailable. Continuing to retry for 25s

This has been happening over the last 24 hours only.

Diggin about the suspicion is the Railway edge routing maybe the issue.

Anyone else having similar issues and a fix?

Tia!

$10 Bounty

5 Replies

Railway
BOT

12 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway 12 days ago


sahandkaramy666-lab
FREE

11 days ago

Ok that's good perefc


lleywyn
PROTop 5% Contributor

11 days ago

Any errors in deploy logs ? Other than build log?


mickeywgla
HOBBYOP

11 days ago

This is what I've worked with the llm's on so far.

No error logs in the build logs apart from healthcheck failed.

We're experiencing a persistent deploy failure on Railway (Hobby plan, EU West / ams1). Every deploy since ~June 25 02:13 UTC fails the platform healthcheck with "Application failed to respond", even though the container builds, starts, and runs healthy.

Setup:

Node.js app (node:22-alpine, Express, multi-stage Docker build)

EXPOSE 3000, PORT=3000 env var, app.listen(PORT, '0.0.0.0')

Healthcheck path: /api/health

No railway.toml or railway.json config file

What happens on every deploy:

Docker image builds successfully ✅

Container starts — runs migrations, seeds, then node server.js ✅

Server is listening on 0.0.0.0:3000 within ~3 seconds ✅

Docker HEALTHCHECK instruction passes inside the container ✅

Railway platform healthcheck fails — "service unavailable" for all 14 retry attempts over a 5-minute window ❌

"1/1 replicas never became healthy!" — deploy marked FAILED ❌

Edge proxy returns 502: {"status":"error","code":502,"message":"Application failed to respond"} with header x-railway-fallback: true ❌

Evidence the app is healthy:

replicas: { running: 1, crashed: 0, exited: 0 }

Container runtime logs show Express serving /api/health requests internally (from Docker HEALTHCHECK wget localhost:3000/api/health)

The last successful deploy (dfd463b7, June 25 02:13 UTC) uses the identical Dockerfile, port config, and healthcheck setup — it worked fine for months

No code changes to server.ts, port binding, or Dockerfile preceded the failures

Edge proxy response:

HTTP/2 502

server: railway-hikari

x-railway-fallback: true

x-railway-edge: ams1

{"status":"error","code":502,"message":"Application failed to respond"}

The x-railway-fallback: true header suggests the proxy can't route to the container at all.

What we've tried (all unsuccessful):

Fix Result

1 Fixed a startup error in seed script Still failed

2 Healthcheck timeout 30s → 60s Still failed

3 Explicitly set PORT=3000 env var Still failed

4 Manual redeploy from Railway dashboard Still failed

5 Healthcheck timeout 60s → 300s (5 min, 14 attempts) Still failed

6 Explicit 0.0.0.0 bind in app.listen() Still failed

7 Created a second identical service from scratch Same failure

8 Set domain targetPort=3000 explicitly Still failed

What we've ruled out:

Slow startup — server ready in ~3s; 300s timeout with 14 attempts still fails

App crash — 0 crashes, Docker HEALTHCHECK passes

Port mismatch — EXPOSE 3000, PORT=3000, listen(3000), domain targetPort=3000 all consistent

IPv6 binding — explicit 0.0.0.0 bind had no effect

Dockerfile changes — last change was before the last successful deploy

Key questions:

Has there been a platform change in EU West (ams1) around June 25 that could affect proxy-to-container routing?

Is there a way to disable Railway's platform healthcheck and rely only on Docker's HEALTHCHECK instruction?

Could this be a stale proxy configuration or routing table issue on the service?

Are there additional diagnostics we can run to pinpoint why the proxy can't reach the container?

Any help or suggestions would be hugely appreciated! 🙏


mickeywgla
HOBBYOP

11 days ago

Further TS shows

The Railway edge proxy is routing traffic to a stale container IPv6 address that no longer exists, while the actual new container is healthy at a different IPv6 address.

The railway logs --network output proves it:

Proxy → container: 54 bytes (TCP reset, no HTTP) — targeting [...95:80c4:9445]:3000

Docker HEALTHCHECK → container: 1.3 KB (real HTTP 200) — targeting [...46:7221:3f94]:3000

Different IPs. The proxy never reaches the new container. Nothing we change in our app matters — the traffic is being sent to a dead address.


mickeywgla

Further TS shows The Railway edge proxy is routing traffic to a stale container IPv6 address that no longer exists, while the actual new container is healthy at a different IPv6 address. The railway logs --network output proves it: Proxy → container: 54 bytes (TCP reset, no HTTP) — targeting [...95:80c4:9445]:3000 Docker HEALTHCHECK → container: 1.3 KB (real HTTP 200) — targeting [...46:7221:3f94]:3000 Different IPs. The proxy never reaches the new container. Nothing we change in our app matters — the traffic is being sent to a dead address.

lleywyn
PROTop 5% Contributor

4 days ago

your network log is the answer but not the way you read it. the proxy getting a raw tcp reset means nothing is listening on that ipv6 address, and your app isn't, app.listen with '0.0.0.0' binds ipv4 only. the docker healthcheck passes because wget localhost goes over ipv4. railway's edge talks to containers over ipv6, and if that flipped on ams1 around jun 25 it'd break exactly like this with zero code changes on your side.

change the bind to app.listen(PORT, '::') and redeploy. that listens on both stacks and should make the platform healthcheck go green on the first attempt.

just answered a similar question to this lol


Welcome!

Sign in to your Railway account to join the conversation.

Loading...