9 days ago
Description:
Project: basic-software-sprint-bridge-app
Environment: production
Issue: sprintbridge-app cannot connect to Postgres despite both being online
Postgres logs show "database system is ready to accept connections"
App fails with: "Error acquiring client: Connection terminated due to connection timeout" and "Connection terminated unexpectedly"
Suspicion: IPv6/IPv4 mismatch on legacy environment's private networking (postgres.railway.internal resolves to IPv6 only, but pg client defaults to IPv4)
All variables correctly configured, credentials valid, both services online
Already tried: variable fixes, Postgres redeployment, app restart, using DATABASE_URL
This was working on Friday of this week without issue. I did not change any settings or deploy any new code but the connection is now failing.
2 Replies
9 days ago
This thread has been opened as a public bounty so the community can help solve it. The thread and any further activity are now visible to everyone.
Status changed to Open Railway • 9 days ago
9 days ago
We are intermittently experiencing the same issue but using the Public URL as well.
2 hours ago
Worth separating two possibilities here, because alex-delia's reply is an important data point that points away from your own diagnosis.
If the failure also happens over DATABASE_PUBLIC_URL, it is not a private networking problem. Public and private take completely different paths — the public URL goes out through the TCP proxy and back in, private stays on the internal network. A fault that hits both is upstream of both, which means the Postgres service itself or the host it's on, not DNS or address families. So the first thing I'd establish is whether your app fails on the public URL too, or only privately. That single answer splits the problem in half and it's ten minutes of work.
If it is private-only, then the most likely cause given "worked Friday, nothing changed" is not the IPv4/IPv6 mismatch you suspected.
Is your app image Alpine-based? Alpine uses musl instead of glibc, and musl queries all configured DNS servers in parallel and takes whichever responds first. Railway's private DNS and the public resolver both get asked, and when the public one wins the race, postgres.railway.internal fails to resolve privately. Because it's a race, it's nondeterministic — which is exactly how you get days of working followed by sudden failure with no deploy. The fix is a service variable on the app, not on Postgres:
ENABLE_ALPINE_PRIVATE_NETWORKING=true
A ~3 second delay before the first connection attempt is also worth adding, since the private network takes a moment to come up in a fresh container and a pool that connects immediately can fail before it's ready. "Connection terminated due to connection timeout" at boot fits that.
On the IPv4/IPv6 framing specifically — if your environment predates October 16th 2025 then yes, the private network is IPv6-only and postgres.railway.internal has AAAA records only. But node-postgres doesn't force IPv4 on its own. When a hostname has only AAAA records, dns.lookup returns the v6 address regardless of result order, so a plain pg client pointed at the hostname resolves fine. That's why I'd look at the resolver (Alpine) or the timing before the address family.
To tell resolution from reachability, from a shell in the running container:
getent hosts postgres.railway.internal
Nothing returned, or a public-looking answer, means DNS — the Alpine variable is your fix. A private IPv6 address returned but connections still timing out means reachability, which is a different problem.
Last thing to rule out: private IPv6 addresses on Railway are ephemeral and change when a service redeploys. Anything that resolved the hostname once and cached the result — a long-lived pool, a connection string assembled at boot from a resolved IP — would break silently after a Postgres redeploy while both services still show as online. Always connect by name, never by a captured address.