Postgres database missing after auto-migration to metal
grantcuster
HOBBYOP

10 months ago

Hello - after auto-migration to metal it looks like my postgres database is no longer showing up under the data tab or when i try and connect to it through the API. I'm hoping maybe it's just an issue with it mounting. Please advice.

Solved

5 Replies


Status changed to Awaiting User Response Railway 10 months ago


10 months ago

I'm looking into this with the team but, just wanted to see what the DB is used for an if there's a possibility some kinda of external job fired (like, what's upstream of this Postgres?)


10 months ago

Figured it out. Wow that's a doozie

So, when we create the migration, we don't actually move the old instance. We make a copy, streaming replicated, verify integrity, and then cut over.

It appears your volume was migrated during a high traffic time, which meant the migration timed out. Which, would have been fine, except it was retried. And when it retried, it tried to copy from new (empty) volume. We performed this migration 10-20k times yesterday, and it looks like < 10 instances right now were affected.

We've done the following:

- Fixed the bug
- Reset the migration, restoring your data
- Applied credits to your account for the inconvenience

Again, very sorry this happened. I can assure you the data is there, and that we take your trust in us, and your data, VERY seriously.

Best,
Jake Cooper
Founder @ Railway


grantcuster
HOBBYOP

10 months ago

So glad it's back! Thanks for figuring it out and the explanation.


Status changed to Awaiting Railway Response Railway 10 months ago


echohack
EMPLOYEE

10 months ago

Thanks for sticking with us through the migration!


Status changed to Awaiting User Response Railway 10 months ago


Status changed to Solved echohack 10 months ago


Loading...