Already migrated to Metal but Railway has tried to migrate again
theoht
PROOP

8 months ago

We migrated to Metal ages ago but last night Railway automatically attempted the migration again. It's left our database in permanent 'waiting to migrate…' state.

Solved

14 Replies

theoht
PROOP

8 months ago

b7ffa58d-4fc3-417f-8ad8-ffc29078d847


theoht
PROOP

8 months ago

Also we are Railway Pro members if that's relevant 🙂


jasperchess
PRO

8 months ago

It also seemed to redeploy our Redis instance. It's a bit frustrating because we had planned downtime 2 months ago to make the switch, but the persistent services all seem to have been redeployed (Or at least attempted to be redeployed)


adam
MODERATOR

8 months ago

!t


adam
MODERATOR

8 months ago

This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response adam 8 months ago


brody
EMPLOYEE

8 months ago

Hello all,

Railway will occasionally initiate mandatory deployments to migrate your service between hosts for security patches and platform upgrades.


jerrodtuck
PRO

8 months ago

Did the same for my Redis Instance and still not back up, which means I have been down for almost 14 hours. Railway said they had the issue resolved within the hour. Doesn't seem to be the case.


jasperchess
PRO

8 months ago

Wait so at any point Railway might take our DB down without any notice? For security patches


brody
EMPLOYEE

8 months ago

It is rare, but sometimes a redeploy is needed to keep the underlying systems up to date and running smoothly.


jasperchess
PRO

8 months ago

Totally makes sense, but are these going to be without warning?

Sorry, I don’t mean to be annoying, just trying to get a full picture of what I can expect 😅


brody
EMPLOYEE

8 months ago

We do not have a pre-warning system in place as there would be negligible impact, what Tuck ran into was an issue in its own right.


jake
EMPLOYEE

8 months ago

FYI I don't see a "Waiting on migration" in that project


Status changed to Awaiting User Response Railway 8 months ago


jasperchess
PRO

8 months ago

Thank you both for the responses, yep it seems to have resolved, it was on the Postgres volume. I'm not sure if it was a manual resolution or not :)

"We do not have a pre-warning system in place as there would be negligible impact" Just to double check, even on persistent workloads there would be minimal impact?


Status changed to Awaiting Railway Response Railway 8 months ago


brody
EMPLOYEE

8 months ago

Correct, when migrating stateful workloads, we will snapshot the volume, but keep the deployment online while we transfer the data in the initial snapshot, then we will shut the deployment down, take another snapshot, and transfer the small amount of new data that was created since the initial snapshot, so the amount of downtime is quite small.


Status changed to Awaiting User Response Railway 8 months ago


jasperchess
PRO

8 months ago

Thanks Brody slightly_smiling_face emoji


Status changed to Awaiting Railway Response Railway 8 months ago


Status changed to Solved jake 8 months ago


Loading...