Corrupt PG volume after abrupt maintenance notice
dotlouis
PROOP

2 months ago

Hey there, I'm very frustrated by what happened today.

I received an immediate notice of maintenance this morning saying:

Your Railway service will be affected by an infrastructure maintenance event.

There is no action required on your part. Your service will automatically resume once maintenance completes. We apologize for the inconvenience.

Right now, I can't restart or redeploy my Postgres service because it's stuck on "Creating containers" for 7+ minutes. I don't see anything on build and deploy logs.

I fire up the agent to help me out, which states the volume might be corrupted because the PG service might not have been terminated properly. No shit!

I need help getting back on my PG service with its volume data. But come on, could you at least let us know beforehand when you're doing maintenance during the day (btw, why isn't it running at night in UE regions)?

And to add fuel to the fire, I almost had my volume wiped out by your agent after it found out it might be corrupted. (Thankfully, I discarded the changes just in time). Please don't let the agent do this so lightly.

Thanks for your help,

I love and trust Railway, but this kind of thing erodes trust fast.

Please fix.

Attachments

Solved

2 Replies

2 months ago

Hey! Sorry about this. We're currently dealing with a suspected hardware failure affecting a subset of services in EU West: https://status.railway.com/incident/cmmui0c7z012icp7ebcd1a3zv


Status changed to Awaiting User Response Railway 2 months ago


chandrika
EMPLOYEE

2 months ago

We've resolved the incident https://status.railway.com/cmmui0c7z012icp7ebcd1a3zv. If your service has not automatically recovered, please try redeploying. If you're still experiencing issues after that, please let us know here and we'll help. Again, sorry for the disruption.


Railway
BOT

2 months ago

This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!

Status changed to Solved Railway about 2 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...