My applications automatically redeployed this morning and crashed with: Volume migration failed
btnalexandre
PROOP

8 months ago

Hello,

This morning, two services automatically redeployed, but crashed with this error:

Pre-deploy ' Migrate volumes

(00:00)

Volume migration failed

Could you please investigate and correct this error as I think it is related to the automatic migration of volumes to Metal. I'm on EU Amsterdam.

The two services concerned :

Environment : Production

Services :
- WordPress (Failed)

- MySQL (Failed)

Thanks in advance.

Solved

18 Replies

btnalexandre
PROOP

8 months ago


Railway
BOT

8 months ago

Hello!

We're acknowledging your issue and attaching a ticket to this thread.

We don't have an ETA for it, but, our engineering team will take a look and you will be updated as we update the ticket.

Please reply to this thread if you have any questions!


8 months ago

Heya, sorry about that, looks like you're back up now. I've flagged this w/the team so they can take a look at what happened.

Did you experience any downtime from these ?


Status changed to Awaiting User Response Railway 9 months ago


itsrems

Heya, sorry about that, looks like you're back up now. I've flagged this w/the team so they can take a look at what happened.Did you experience any downtime from these ?

btnalexandre
PROOP

8 months ago

Hello !

No downtime now, have you redeployed the services ?
The migration to metal is completed for me ?

Thanks !


Status changed to Awaiting Railway Response Railway 9 months ago


8 months ago

The migration failed so unfortunately we'll need to run it again. it should be smooth and happen in the next few days slightly_smiling_face emoji


Status changed to Awaiting User Response Railway 9 months ago


itsrems

The migration failed so unfortunately we'll need to run it again. it should be smooth and happen in the next few days

btnalexandre
PROOP

8 months ago

Okay thanks. Let me know when it’s done slightly_smiling_face emoji


Status changed to Awaiting Railway Response Railway 9 months ago


Status changed to Solved itsrems 8 months ago


btnalexandre
PROOP

8 months ago

@itsrems you mark this thread as solved but are the services migrated ?


Status changed to Awaiting Railway Response Railway 8 months ago



btnalexandre
PROOP

8 months ago

And we've had poor performance on WordPress for the past 2 weeks. Could this be due to an incomplete migration?


8 months ago

Are you still running into performance issues this week / today ? we had some performance issues with metal volumes, but applied fixes across our fleet earlier this week.


Status changed to Awaiting User Response Railway 8 months ago


btnalexandre
PROOP

8 months ago

Hello, today this seems better. But I need some days after a redeployment to see if performance issues have gone. I'll let you know in a few days.

Have you some news regarding the migration issue that occurred a second time this morning for this service : https://railway.com/project/ad2fe3e4-05ab-4d67-a5da-87f5bb5bf1ee/service/932c4061-a767-45c1-b821-592845724010?environmentId=71e28411-a28b-4e00-9928-0d3e929d942c

Thanks in advance !


Status changed to Awaiting Railway Response Railway 8 months ago


8 months ago

We're moving some workloads between our hosts, and a subset of migrations fail. However these don't affect your service, and shouldn't show up in your dashboard (as we retry them as needed). Apologies for the confusion !


Status changed to Awaiting User Response Railway 8 months ago


btnalexandre
PROOP

8 months ago

Okay thanks for this information ! From what you can see, have I migrated everything to Metal (I did it manually before Railway's intervention)?


Status changed to Awaiting Railway Response Railway 8 months ago


8 months ago

Yup all on metal ! The failed migration you saw was from us moving some workloads from metal to metal.


Status changed to Awaiting User Response Railway 8 months ago


itsrems

Yup all on metal ! The failed migration you saw was from us moving some workloads from metal to metal.

btnalexandre
PROOP

8 months ago

Thanks for your quick reply. I will reopen this thread if I encounter an another issue or if performance issues persist slightly_smiling_face emoji


Status changed to Awaiting Railway Response Railway 8 months ago


Status changed to Solved btnalexandre 8 months ago


itsrems

Yup all on metal ! The failed migration you saw was from us moving some workloads from metal to metal.

btnalexandre
PROOP

8 months ago

Hello, another news this night the service WordPress (on production) was down between 04:15:39 AM CEST and 04:50:19 AM CEST, today.

On the service I see the error again "Volume migration failed."
Could you investigate how to get rid of down due to a failed migration?

Thanks


Status changed to Awaiting Railway Response Railway 8 months ago


Railway
BOT

8 months ago

Hello!

We've escalated your issue to our engineering team.

We aim to provide an update within 1 business day.

Please reply to this thread if you have any questions!

Status changed to Awaiting User Response Railway 8 months ago


btnalexandre

Hello, another news this night the service WordPress (on production) was down between 04:15:39 AM CEST and 04:50:19 AM CEST, today.On the service I see the error again "Volume migration failed."Could you investigate how to get rid of down due to a failed migration?Thanks

chandrika
EMPLOYEE

8 months ago

Hi there, I've fixed your volume migration — you should be all set raised_hands emoji! Could you please confirm that everything is working smoothly now?


btnalexandre
PROOP

8 months ago

Thanks that seems good ! Have a nice day


Status changed to Awaiting Railway Response Railway 8 months ago


Status changed to Solved btnalexandre 8 months ago


Loading...