21 days ago
Project ID: 296a68b7-22dc-4f16-a10d-a35985dd8ea5
Environment: production
Affected services: sixstaraudio (web, id e84a12fd-ebe4-4aff-9a78-8c2f84b1a4ea) ↔ Postgres (id 6fa3f3fa-dc67-40eb-9bd9-b5d2d4545097)
Since ~2026-07-30 18:00 UTC, queries/transactions from our web service to our Postgres instance are intermittently extremely slow or hang for 60-125+ seconds, while Postgres itself shows near-zero CPU (~0.02-0.1%) and normal memory (~70-85MB) the entire time — it is not under load.
Concrete examples:
A simple indexed lookup (SELECT ... FROM users WHERE email = %s LIMIT 1) — sometimes returns in <30ms, other times the client gives up after 60-125s waiting.
SELECT pg_advisory_xact_lock(...) — repeatedly fails with canceling statement due to statement timeout after 15s, on every deploy attempt since the incident started.
Reads that don't touch Postgres (static pages, tool pages) are consistently fast (5-200ms) — the slowness is specific to the Postgres connection path.
I restarted BOTH the Postgres service and the web service (full redeploys) — the issue persists identically afterward, ruling out a stuck transaction/lock or a stale connection pool on our end.
This looks like it may be affecting our project specifically (isolated to our private networking path to Postgres), separate from the broader "Builds and deployments delayed" incidents you resolved on 2026-07-29/30 — could you please check the network path / storage layer for this Postgres instance?
1 Replies
21 days ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 21 days ago
21 days ago
the 60-125s range is the tell here. thats not the db being slow, thats tcp retransmit timeout - your pool has idle connections that something on the network path silently dropped, the client fires a query into a dead socket and waits out the retransmit cycle before giving up. explains everything: db shows idle (query never arrives), intermittent (only stale connections hit it), restarts dont help (new pool goes idle and gets culled again), started at a specific time (railway changed something on the path 07-30).
two things to do:
confirm it: temporarily point your web service at DATABASE_PUBLIC_URL. if the hangs vanish, its the private network path on your host, and thats hard evidence for the railway team.
work around it: enable tcp keepalive on your pool and drop the idle timeout under ~60s so connections get recycled before the network kills them. node-postgres: keepAlive: true, idleTimeoutMillis: 30000. prisma/pgbouncer: add keepalives=1&keepalives_idle=30 to the url. your pg_advisory_xact_lock timeouts on deploy are the same thing, the migration is grabbing a dead connection.
this doesnt excuse the platform side, still worth railway checking the path, but keepalives will make your app immune to it either way