2 months ago
Hi Railway team,
On March 30, 2026, my PostgreSQL service was redeployed with the postgres-ssl:18 Docker image. After this redeployment, all data in my database was lost.
Evidence:
- pg_stat_user_tables shows n_tup_del = 0 for ALL tables — no DELETE/TRUNCATE was ever executed.
- My app has not restarted since Mar 30, and no code-level DROP or TRUNCATE exists.
- A postgres-volume is attached, but data was not preserved through the redeployment.
This coincides with the CDN caching incident on March 30. Could the infrastructure changes that day have also affected PostgreSQL volume persistence?
I see similar threads ("postgres volume wiped on update") — is this a known issue?
Questions:
1. Why was the volume not preserved during the redeployment?
2. Does Railway have any internal snapshots that could help recover data?
3. How can we prevent this going forward?
Thank you.
3 Replies
Status changed to Awaiting Railway Response Railway • about 2 months ago
Status changed to Open Railway • about 2 months ago
2 months ago
Root cause
When Railway redeployed with the new image, it likely recreated the container with a shifted PGDATA path. Postgres initialized a fresh cluster and ignored your existing volume data entirely. The n_tup_del = 0 confirms this,data wasn't deleted, it was abandoned.
Recovery options
- Exec into the container and check the volume manually. Your old data directory may still be sitting there untouched under /var/lib/postgresql/
- Contact Railway support and ask explicitly about infrastructure-level volume snapshots, they sometimes exist even if not exposed in the UI
- Check any app-level caches (Redis etc.) that survived the redeployment
Prevention going forward
- Pin PGDATA explicitly in your env vars to match your exact volume mount path
- Set up automated pg_dump to S3/Backblaze via a cron job
- Always run a manual dump before any image upgrade
- Consider a managed Postgres provider (Supabase, Neon, RDS) if this is production,Railway isn't built for stateful reliability
On the CDN incident
Unrelated. A CDN caching issue doesn't touch volume persistence. Timing was coincidental.
2 months ago
This can happen if you've updated your postgres db to a new major version. When you change from an older postgres version to a newer one, the new engine cannot read the existing disk data because the internal architecture has changed. Your data is likely still safe on the volume. To recover it, you should revert your version back.
If you want to upgrade your version and retain your data, you typically have to dump your data first, upgrade to the new version, then restore the data to the new db.
2 months ago
This isn’t actual data loss — it’s likely a Postgres version issue.
When Railway redeployed with postgres-ssl:18, it probably upgraded Postgres. New major versions can’t read old data directories, so Postgres created a fresh empty database instead of using your existing volume.
That’s why nothing shows as deleted, but your data is “gone.”
Your old data is likely still on the volume.
What to do:
- Roll back to the previous Postgres version (fastest fix)
- Check the container for the old data directory / PGDATA mismatch
- Ask Railway for any volume snapshots
also always pg_dump before upgrading Postgres.