postgres volume wiped on update
jback3030
HOBBYOP

5 months ago

# Auto-update redeployment assigned new volume, causing complete data loss

**Environment:** `0a35acf5-8ca0-46ca-b407-6905fa8b7e48`
**Service:** `68c5b8cb-dadc-458c-bad9-d5ed0fb5f12a`
**Postgres image:** `ghcr.io/railwayapp-templates/postgres-ssl:17`

## Summary

On Feb 17, our dev Postgres service was redeployed — we believe by an auto-update to 17.7. The new deployment was assigned a **different volume** than the previous one, resulting in a fresh `initdb` and complete data loss. Auto-updates were enabled on this service.

## Evidence

### Old deployment (`f4cf7117`) — data intact

18:05:29 Mounting volume on: .../vol_g3lemx2e53qr2p68

18:05:31 PostgreSQL Database directory appears to contain a database; Skipping initialization

18:05:31 database system was interrupted; last known up at 2026-01-30 23:03:38 UTC

18:05:31 database system is ready to accept connections

18:10:49 sending signal SIGTERM to container

18:11:52 Stopping Container


### New deployment (`988dc6f4`) — empty volume, 4 seconds later

18:11:56 Mounting volume on: .../vol_rp0xx9ceadyrzqql <-- different volume ID

18:11:58 selecting default "max_connections" ... 100 <-- initdb running on empty dir

18:12:01 CREATE DATABASE

18:12:18 PostgreSQL init process complete; ready for start up.


The old deployment mounted `vol_g3lemx2e53qr2p68` with all existing data. The new deployment received `vol_rp0xx9ceadyrzqql`, which was empty, so Postgres ran `initdb` and created a fresh cluster.

Is this expected behavior when auto-updates trigger a redeployment? The [volume docs](https://docs.railway.com/volumes/reference) state data persists across redeploys.

## Impact

The dev data loss is recoverable (re-run migrations), but this would be bad on a production database. We've disabled auto-updates on prod as a precaution.
Solved

2 Replies

sam-a
EMPLOYEE

5 months ago

Sorry for the confusion here. Looking at the workflow timeline, this wasn't caused by an auto-update - a volume wipe was triggered on this service at 18:00:11 UTC on Feb 17, about 11 minutes before the deployment you referenced. The volume wipe feature intentionally creates a new empty volume and redeploys the service to use it, which is why you saw a different volume ID. The old volume data is no longer recoverable since the wipe process queues deletion of the previous volume. For your dev environment, re-running migrations as you mentioned should get you back up. Going forward, if you need to clear data but want a safety net, consider creating a manual backup before any destructive operation - that way you have a restore point if needed.


Status changed to Awaiting User Response Railway 5 months ago


sam-a

Sorry for the confusion here. Looking at the workflow timeline, this wasn't caused by an auto-update - a volume wipe was triggered on this service at 18:00:11 UTC on Feb 17, about 11 minutes before the deployment you referenced. The volume wipe feature intentionally creates a new empty volume and redeploys the service to use it, which is why you saw a different volume ID. The old volume data is no longer recoverable since the wipe process queues deletion of the previous volume. For your dev environment, re-running migrations as you mentioned should get you back up. Going forward, if you need to clear data but want a safety net, consider creating a manual backup before any destructive operation - that way you have a restore point if needed.

winter100004
PRO

5 months ago


Status changed to Awaiting Railway Response Railway 5 months ago


Status changed to Solved sam-a 5 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...