Hey there! We've found the following might help you get unblocked faster:

• 🧵 I want to pause a service within a project

• 🧵 Read-only filesystem preventing volume mount

• 🧵 Volume Migration Failed Post-Metal Migration – Risk of Data Loss

If you find the answer from one of these, please let us know by solving the thread!

Currently experiencing a similar issue but with a Postgres service: https://station.railway.com/questions/unable-to-connect-to-postgres-database-1dcf1a1a

I have the same issue this morning on my volume, it's down for no reason. And I was underneath 5GB, and around 9:30 am eastern time this morning my volume is down, and I can't reach any information about it as well.

I am at least able to ssh into the Postgres server machine and connect to the database, and the data appears to be intact, so there is hope agustintiseira.

Is your volume down as well?

It looks empty:

And my database never connects. I can't connect to it from my services or my local machine, but if I ssh in I can connect. This has taken down our product

I have a dev and a production on railway, the exact same code, same amount of volume, one is down and not the other:


No more metrics in my volume

Ok so you do have the exact same issue, did it happen around 1-2 hours ago?

It looks like it started last night, and it may have been intermittent at first. Need to investigate more though.

Hello all.
We are investigating this issue currently! Thank you for bringing it up.

Hello!

We've escalated your issue to our engineering team.

We aim to provide an update within 1 business day.

Please reply to this thread if you have any questions!

Ok Yeah, it seems like I can't see the volume 10h ago on my side. Stats on the volume from 10h ago.

Thank you, Noah. Do you have any suggested workarounds? This has completely taken our business down.

Same for me 12 employees can't work.

I don't have any workarounds as of right now. We are investigating this issue right now and are working towards resolution.

Hello,
We have called an incident for this issue and it is actively being worked on by our engineers.
You can follow along here: https://status.railway.com/cmhdlky9r003e44y4p4lyortz

Looks like we're back up, thank you!

Hello,
Everything should be back and working. Are you able to verify that your services and data are intact? In my checks it looks like your volume is displaying 28 gigs of data!

I confirm that the service is now working again and that the volume has been restored with the data it contained up until the time of the outage. I would like to know what caused the issue and whether there is any possibility of it happening again.

Additionally, I would like to ask if it’s possible to merge the restored volume with the one I was using during the downtime, since if I start using the old volume again, I would lose the media files created over the past few hours while the issue was being resolved.

Thank you in advance for your help.

Unfortunately I do not know of any easy method of merging the volumes other than manual data transfer.
I am glad to hear that the issue is resolved!