I have the same issue.

This is not an official response, but what I would do in your position is mount the volume to another instance where you can pull the data off, then load it back into a fresh postgres instance.

Again, not an official response, just my instinct from experience. I lost my DB as well, but I didn't have anything production on it so I just re-created and re-ran migrations.

I thought of doing this.. but I dont seem to have an option to detach the volume..

Again, please take with a grain of salt, but when I resigned myself to re-building I triggered a deletion of postgres, and it just detached the volume for me and left it floating.

I don't see anywhere in the settings to detach the volume, though there probably should be. Maybe they don't offer that to preserve data integrity?

I started up a new instance, and it built correctly. Odd.

I will not delete anything bc im a little hesitant, but maybe I can point to this for the interim.

The team should design a recovery option, if I can simply re-instantiate a new DB... but I can't restart my existing one, then I should have the option to detach the volume, and attach it to my new instance. I don't see the risk in this, as you are not going to lose data. It seems strange that my DB has been down for like 12 hours... but If I could just move my volume over it wouldve been up and running by now. I do not have the capability.

I have the same issue, also looking for the fix, customers started calling me hours ago for the most recent data

If you have a good backup outside of Railway, you would be able to deploy a new DB and restore the backup. I had luck with just deleting my existing postgres and Railway automatically detached the volume and left it floating. This is just my experience, ymmv

Regrettably, I don't see a way to simply download the backup from Railway. That would have made this whole process easier; being able to download your backup.

I have same issue, and mounting a new container does not work either. The logs say the europe-west4 is not available, so it's more a infrastructure problem that makes impossible to set up services. Anyone on this problem? Known issue on the support?

Apologies for this canned message but in an effort to help all our customers get back up and running, we are sending this bulk message. As you may know, we had a major interruption to our services yesterday. We've published a post-mortem if you'd like more information on the incident. It describes what happened and what we are doing to prevent it in the future. We are deeply sorry for the impact that it has had on you.

It is taking some time to bring everything back up, but we are working on it as fast as we can. In general, a redeployment should fix most service issues. Due to the volume of customers redeploying right now, builds and deploys may take longer than normal to process.

You can track recovery status here: https://status.railway.com/incident/KVZ1Z8GY

If you are still having other issues that might be related to the incident you can read more here: https://station.railway.com/community/road-to-recovery-post-gcp-outage-builds-d362e48c

Feel free to respond if your question has not been addressed.

Hello - you mention in the attached link:

Q: Why is my Postgres DB still offline?

If your Postgres (or other) service is stuck in a crash loop with catatonit: failed to exec pid1: No such file or directory: try one redeploy first. If it does not recover within a few minutes, reply with your project ID and service ID. The volume may need to be moved to a healthy node.

My postgres is still offline and has been for roughly 24 hours not recovering. I tried restarting, same issue with pid as I've seen.

Project ID: 6426ad69-898f-41d2-b895-59054a04ee2d

Service ID: 51628ecb-5366-4fcc-ab3b-b4a712312756

Please help.

Hey, your Postgres is crash-looping with the catatonit pid1 error, which means the container image pulled during the outage is broken. Your data is safe on the volume.

The fix: go to your Claim DB service Settings and update the Postgres version (e.g. if you're on 16, change to 16.8 or 17). This forces a fresh image pull while keeping your existing volume and data intact. Once Postgres is back, redeploy any dependent services.

This is related to the May 19 GCP outage. Sorry about the long downtime. Let us know if it works.

I also would like a download option..

Interesting. I believe I already was on 17.. I just went and changed it to 16.8... failed.. changed back to 17.. Finally deployed successfully! - Thank you!

Glad the version toggle worked - switching versions forces a fresh image pull, which is exactly the fix for the corrupted container image. Your data was safe on the volume the whole time.

Regarding downloading backups, those are volume snapshots and aren't usable off-platform. To export your data, use pg_dump now that Postgres is back up. We hear the feedback on volume detach/reattach, you can submit that as a feature request at railway.com/feedback.