17 days ago
Hi Railway team,
Earlier today my Postgres service had its public schema destroyed via
DROP SCHEMA CASCADE. Dashboard does not show any backups available.
This is production data for an active business — bookings, customers,
financial records.
Is there ANY recovery path on your side? WAL archive, internal snapshots,
volume-level backup, anything?
Time of incident: 8:45AM AST - 20 mins ago.
Thanks — happy to provide whatever IDs/info you need.
Pinned Solution
17 days ago
Unfortunately, that's two very different scenarios.
In the PocketOS incident the agent hit a Railway API endpoint that deleted the volume, that's an infrastructure level event and Railway maintains safeguards in case the user wants to recover from those kinds of actions (a snapshot, in this case).
In your case, the volume and service are intact and working as expected, the query you ran was a SQL operation within your Postgres service, that's a valid user initiated action and not an infrastructure level event, thus not having the same safeguards from the PocketOS incident.
Because of that difference, Railway doesn't have the ability to rollback individual queries unless you did automated and/or manual backups. It’s not about willingness to help, it's just a completely different scenario that doesn't have the same safeguards as the PocketOS case.
I hope this made things clearer for you!
4 Replies
17 days ago
This thread has been marked as public for community involvement, as it does not contain any sensitive or personal information. Any further activity in this thread will be visible to everyone.
Status changed to Open Railway • 17 days ago
17 days ago
Unless you've done manual backups yourself and/or enabled backups for that volume, there's nothing Railway can do.
17 days ago
Thanks for the response. I'd like to request engineering escalation on this rather than closing the thread.
Per Railway's April 29, 2026 blog post on the PocketOS incident (https://blog.railway.com/p/your-ai-wants-to-nuke-your-database), Railway maintains offsite "disaster backups" for Postgres volumes regardless of whether user-configured backups are enabled. That post explicitly notes Railway recovered PocketOS's data via these disaster backups even when the user had no backups configured.
This is a paid Pro plan production database for an active SMB (children's science education franchise serving Atlantic Canada). I have shell access to the container, the snapshot is preserved, and I have an open ticket. I'm asking for the same disaster-snapshot check Jake described in that blog post.
Happy to provide service ID, project ID, exact UTC timestamp of the DROP, or any other identifying info via private channel.
17 days ago
Unfortunately, that's two very different scenarios.
In the PocketOS incident the agent hit a Railway API endpoint that deleted the volume, that's an infrastructure level event and Railway maintains safeguards in case the user wants to recover from those kinds of actions (a snapshot, in this case).
In your case, the volume and service are intact and working as expected, the query you ran was a SQL operation within your Postgres service, that's a valid user initiated action and not an infrastructure level event, thus not having the same safeguards from the PocketOS incident.
Because of that difference, Railway doesn't have the ability to rollback individual queries unless you did automated and/or manual backups. It’s not about willingness to help, it's just a completely different scenario that doesn't have the same safeguards as the PocketOS case.
I hope this made things clearer for you!
17 days ago
Thanks for the detailed explanation — the distinction between infrastructure-layer and application-layer events makes sense, and I appreciate the honest reply rather than a generic "no backups, sorry." I have a snapshot of the pgdata captured before any restart and an active recovery effort underway via WAL decode. Closing this thread.
Feedback for the team: it would significantly help SMB customers if Railway's Postgres template shipped with at least one of (a) a default destructive-DDL event trigger, (b) a default pg_dump-to-S3 sidecar service, or (c) clearer language in the docs that the Backups feature does NOT protect against in-database DROP operations. The PocketOS blog post arguably created the impression that Railway protects against more than it does. Happy to provide more detail if useful.
Status changed to Solved brody • 17 days ago