11 days ago
Service: vedic-astro-backend (ID: c7550e5a-4799-4331-9622-624eb2b106e2)
Project: vedicastro-io
Environment: vedicastro-production
My app runs database migrations on startup. A migration got stuck mid-execution and left an advisory lock (lock_key: 147501) held in Postgres.
Every deployment attempt times out trying to acquire this lock after 5 seconds:
[MIGRATION_EVENT] {"event": "migration_lock_timeout", "lock_key": 147501, "strict_mode": false, "waited_ms": 5022}
The database is responsive, but this stuck lock prevents the app from starting.
Current deployment: aaa5bb3a-3e74-42d6-bd7f-3ea9ad4fa0db (stuck on healthcheck)
Can you please manually clear the advisory lock in my Postgres database (lock_key: 147501)? A SELECT pg_advisory_unlock_all(); should do it.
Once cleared, my deployment should proceed normally.
2 Replies
11 days ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • 11 days ago
11 days ago
Try going to the Console tab of your database, run psql, then run:
SELECT pid, usename, state, age(clock_timestamp(), query_start), query
FROM pg_stat_activity
WHERE query LIKE '%147501%'
OR query LIKE '%pg_advisory%'
OR state = 'active';
Then: SELECT pg_terminate_backend(<PID_NUMBER>); where <PID_NUMBER> is the PID of that query
11 days ago
The suggested pg_advisory_unlock(147501) / pg_advisory_unlock_all() from a fresh Console session will not clear a lock held by another backend: advisory locks are session-owned, so those functions only release locks held by the session executing them.
First identify the actual holder (and confirm it is a stale migration, not a migration still doing useful work):
SELECT l.pid, l.mode, l.granted, l.classid, l.objid, l.objsubid,
a.usename, a.application_name, a.state,
a.backend_start, a.query_start, a.query
FROM pg_locks AS l
JOIN pg_stat_activity AS a ON a.pid = l.pid
WHERE l.locktype = 'advisory'
ORDER BY l.granted DESC, a.query_start;For a single-bigint lock such as pg_advisory_lock(147501), the low 32-bit portion commonly appears as objid = 147501; inspect the lock tuple rather than assuming the representation.
After verifying the PID is the abandoned migration connection, release its session by terminating that backend:
SELECT pg_terminate_backend(<stale_migration_pid>);That automatically releases its advisory locks. It can also roll back that session's uncommitted transaction, so check the migration table/logs and take a backup or snapshot first; do not terminate an active migration. If your Railway DB role cannot call pg_terminate_backend, ask Railway support to end the identified backend, or restart Postgres only as a last resort because it disconnects all clients.
For prevention: run migrations as a single release/one-off job rather than in every replica's startup path, use a bounded advisory-lock wait/retry, and ensure the migration process closes its DB connection cleanly on SIGTERM.