9 months ago
We migrated to Metal ages ago but last night Railway automatically attempted the migration again. It's left our database in permanent 'waiting to migrate…' state.
14 Replies
9 months ago
It also seemed to redeploy our Redis instance. It's a bit frustrating because we had planned downtime 2 months ago to make the switch, but the persistent services all seem to have been redeployed (Or at least attempted to be redeployed)
9 months ago
!t
9 months ago
This thread has been escalated to the Railway team.
Status changed to Awaiting Railway Response adam • 9 months ago
9 months ago
Hello all,
Railway will occasionally initiate mandatory deployments to migrate your service between hosts for security patches and platform upgrades.
Did the same for my Redis Instance and still not back up, which means I have been down for almost 14 hours. Railway said they had the issue resolved within the hour. Doesn't seem to be the case.
9 months ago
Wait so at any point Railway might take our DB down without any notice? For security patches
9 months ago
It is rare, but sometimes a redeploy is needed to keep the underlying systems up to date and running smoothly.
9 months ago
Totally makes sense, but are these going to be without warning?
Sorry, I don’t mean to be annoying, just trying to get a full picture of what I can expect 😅
9 months ago
We do not have a pre-warning system in place as there would be negligible impact, what Tuck ran into was an issue in its own right.
Status changed to Awaiting User Response Railway • 9 months ago
9 months ago
Thank you both for the responses, yep it seems to have resolved, it was on the Postgres volume. I'm not sure if it was a manual resolution or not :)
"We do not have a pre-warning system in place as there would be negligible impact" Just to double check, even on persistent workloads there would be minimal impact?
Status changed to Awaiting Railway Response Railway • 9 months ago
9 months ago
Correct, when migrating stateful workloads, we will snapshot the volume, but keep the deployment online while we transfer the data in the initial snapshot, then we will shut the deployment down, take another snapshot, and transfer the small amount of new data that was created since the initial snapshot, so the amount of downtime is quite small.
Status changed to Awaiting User Response Railway • 9 months ago
Status changed to Awaiting Railway Response Railway • 9 months ago
Status changed to Solved jake • 9 months ago
