Your MySQL service in US East (GCP) was directly affected by the ongoing service disruption. The logs show connection errors and "packets out of order" starting on 2026-05-19, and no logs at all after 21:34 UTC that day, indicating the MySQL process is likely stuck. Redeploying the service is safe - your volume and all data on it persist across redeploys, as the volume is simply re-mounted to the new container. Please trigger a redeploy from the dashboard or CLI to restart the MySQL process with your existing data intact.

URGENT - MySQL redeploy wiped all user data despite support assurance that volume persists

We experienced a significant data loss incident today following advice from Railway support.

Our project is zizzle-backend and the service is MySQL. We are urgently requesting any available snapshot, backup, or volume state from before today's redeploy.

Please escalate this as high-priority - this involves paying customers and was caused by following Railway support's explicit guidance.

Comment: We've tried our best to recover things now on our end. There are still missing pieces. Please reach out if you have a snapshot we could use. The main thing would be selectively recovering user progress data.

I recommend in the future doing offsite backups. There are many tools available to automate this process too.

Thanks for the recommendation. Much appreciated. I've had a backup set up but it missed some crucial tables/columns. Will fix this going forward. Do you think getting a snapshot is likely/realistic?

We've recovered as much as possible and set up better backup processes now. We'd still appreciate any update on whether a snapshot might be available. Let us know!

Hey, we looked into the MySQL logs from the redeploy and can confirm the volume itself was not wiped. InnoDB loaded existing data files and redo logs on startup, which means the persistent storage survived the redeploy. The data loss appears to have happened at the database engine level, likely due to corruption from the abrupt shutdown during the GCP outage, or from a migration/initialization process in your application that ran when MySQL came back up.

Unfortunately for MySQL services we don't have volume snapshots or point-in-time recovery, so there isn't a pre-existing backup we can restore from.

Going forward, we'd recommend enabling volume backups on your MySQL service so you have a restore point for situations like this.

Okay, thanks for checking!