postgres volume wiped on update
jback3030
HOBBYOP

24 days ago

# Auto-update redeployment assigned new volume, causing complete data loss

**Environment:** `0a35acf5-8ca0-46ca-b407-6905fa8b7e48`
**Service:** `68c5b8cb-dadc-458c-bad9-d5ed0fb5f12a`
**Postgres image:** `ghcr.io/railwayapp-templates/postgres-ssl:17`

## Summary

On Feb 17, our dev Postgres service was redeployed — we believe by an auto-update to 17.7. The new deployment was assigned a **different volume** than the previous one, resulting in a fresh `initdb` and complete data loss. Auto-updates were enabled on this service.

## Evidence

### Old deployment (`f4cf7117`) — data intact

```
18:05:29  Mounting volume on: .../vol_g3lemx2e53qr2p68
18:05:31  PostgreSQL Database directory appears to contain a database; Skipping initialization
18:05:31  database system was interrupted; last known up at 2026-01-30 23:03:38 UTC
18:05:31  database system is ready to accept connections
18:10:49  sending signal SIGTERM to container
18:11:52  Stopping Container
```

### New deployment (`988dc6f4`) — empty volume, 4 seconds later

```
18:11:56  Mounting volume on: .../vol_rp0xx9ceadyrzqql    <-- different volume ID
18:11:58  selecting default "max_connections" ... 100      <-- initdb running on empty dir
18:12:01  CREATE DATABASE
18:12:18  PostgreSQL init process complete; ready for start up.
```

The old deployment mounted `vol_g3lemx2e53qr2p68` with all existing data. The new deployment received `vol_rp0xx9ceadyrzqql`, which was empty, so Postgres ran `initdb` and created a fresh cluster.

Is this expected behavior when auto-updates trigger a redeployment? The [volume docs](https://docs.railway.com/volumes/reference) state data persists across redeploys.

## Impact

The dev data loss is recoverable (re-run migrations), but this would be bad on a production database. We've disabled auto-updates on prod as a precaution.
Solved

2 Replies

sam-a
EMPLOYEE

24 days ago

Sorry for the confusion here. Looking at the workflow timeline, this wasn't caused by an auto-update - a volume wipe was triggered on this service at 18:00:11 UTC on Feb 17, about 11 minutes before the deployment you referenced. The volume wipe feature intentionally creates a new empty volume and redeploys the service to use it, which is why you saw a different volume ID. The old volume data is no longer recoverable since the wipe process queues deletion of the previous volume. For your dev environment, re-running migrations as you mentioned should get you back up. Going forward, if you need to clear data but want a safety net, consider creating a manual backup before any destructive operation - that way you have a restore point if needed.


Status changed to Awaiting User Response Railway 24 days ago


sam-a

Sorry for the confusion here. Looking at the workflow timeline, this wasn't caused by an auto-update - a volume wipe was triggered on this service at 18:00:11 UTC on Feb 17, about 11 minutes before the deployment you referenced. The volume wipe feature intentionally creates a new empty volume and redeploys the service to use it, which is why you saw a different volume ID. The old volume data is no longer recoverable since the wipe process queues deletion of the previous volume. For your dev environment, re-running migrations as you mentioned should get you back up. Going forward, if you need to clear data but want a safety net, consider creating a manual backup before any destructive operation - that way you have a restore point if needed.

winter100004
PRO

24 days ago


Status changed to Awaiting Railway Response Railway 24 days ago


Status changed to Solved sam-a 24 days ago


Loading...