Description of the issue: My backend service (speccheck-api) cannot connect to my Postgres service via private networking (postgres.railway.internal:5432) at container boot, consistently, across multiple days and redeploys (Aug 4, 5, 7 2026). Project: Speccheck-backend (ID: 11157f10-97e7-4121-b5c8-30927f0e164a) Service: speccheck-api (ID: 76b1a5c5-727d-448a-93f5-3409958a4f59) Database: Postgres (same project) Your own AI support agent investigated this and found a likely race condition: CONFIGURE_NETWORK reports complete at 10:56:19.135, but my app's first connection attempt at 10:56:19.141 (6ms later) already fails with Connection refused. It concluded the CONFIGURE_NETWORK completion signal appears to fire before the container's network namespace is actually attached to the private mesh. I've since: - Added retry logic with exponential backoff (up to 30s between attempts, indefinite retries in a background task) - Added a healthcheck at /health with a 60s timeout, and made the app start serving requests immediately instead of blocking on the DB connection Despite this, the connection still fails continuously for the full 60-second healthcheck window on every deploy, and the deploy fails. Additional evidence: - Manually connecting to Postgres from an already-running container's Console (raw socket connect, single asyncpg.connect(), and full asyncpg.create_pool()) always succeeds instantly, every time. - Postgres's own logs show zero record of any incoming connection attempt during the failed boot windows — suggesting the connection is being blocked before it reaches Postgres at all, likely at the network layer. - Ruled out: wrong DATABASE_URL, DNS resolution, IPv6 vs IPv4 (tested both), credentials, connection pool vs single connection. Error messages: ConnectionRefusedError: [Errno 111] Connection refused (full tracebacks and deploy logs attached) Could someone from the infrastructure team check whether private networking attachment is delayed or faulty for this service/project? Link to repo: [add your GitHub repo link here if you're comfortable sharing it, e.g. https://github.com/CaptnJayce/spec-check-app]

Private networking: Postgres unreachable at container boot, works fine manually

Anonymous

FREEOP

2 months ago

Description of the issue:

My backend service (speccheck-api) cannot connect to my Postgres service via private networking (postgres.railway.internal:5432) at container boot, consistently, across multiple days and redeploys (Aug 4, 5, 7 2026).

Project: Speccheck-backend (ID: 11157f10-97e7-4121-b5c8-30927f0e164a)

Service: speccheck-api (ID: 76b1a5c5-727d-448a-93f5-3409958a4f59)

Database: Postgres (same project)

Your own AI support agent investigated this and found a likely race condition: CONFIGURE_NETWORK reports complete at 10:56:19.135, but my app's first connection attempt at 10:56:19.141 (6ms later) already fails with Connection refused. It concluded the CONFIGURE_NETWORK completion signal appears to fire before the container's network namespace is actually attached to the private mesh.

I've since:

Added retry logic with exponential backoff (up to 30s between attempts, indefinite retries in a background task)
Added a healthcheck at /health with a 60s timeout, and made the app start serving requests immediately instead of blocking on the DB connection

Despite this, the connection still fails continuously for the full 60-second healthcheck window on every deploy, and the deploy fails.

Additional evidence:

Manually connecting to Postgres from an already-running container's Console (raw socket connect, single asyncpg.connect(), and full asyncpg.create_pool()) always succeeds instantly, every time.
Postgres's own logs show zero record of any incoming connection attempt during the failed boot windows — suggesting the connection is being blocked before it reaches Postgres at all, likely at the network layer.
Ruled out: wrong DATABASE_URL, DNS resolution, IPv6 vs IPv4 (tested both), credentials, connection pool vs single connection.

Error messages:

ConnectionRefusedError: [Errno 111] Connection refused

(full tracebacks and deploy logs attached)

Could someone from the infrastructure team check whether private networking attachment is delayed or faulty for this service/project?

Link to repo: [add your GitHub repo link here if you're comfortable sharing it, e.g. https://github.com/CaptnJayce/spec-check-app]

$10 Bounty

2 Replies

Railway

BOT

2 months ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway • about 2 months ago

0x5b62656e5d

MODERATOR

2 months ago

I assume you're connecting to the database during the pre-deploy and/or deploy phase instead of the build phase?

0x5b62656e5d

I assume you're connecting to the database during the pre-deploy and/or deploy phase instead of the build phase?

Anonymous

FREEOP

2 months ago

It's during runtime, in my app's startup code (FastAPI lifespan function), which runs

when the container process starts. Its not during the build phase, and I don't have any

pre-deploy step configured for this service. The connection attempt happens as soon as

Uvicorn starts the app, before it begins serving any requests.

Welcome!