My applications automatically redeployed this morning and crashed with: Volume migration failed

btnalexandre
PRO

2 months ago

Hello,

This morning, two services automatically redeployed, but crashed with this error:

Pre-deploy ' Migrate volumes

(00:00)

Volume migration failed

Could you please investigate and correct this error as I think it is related to the automatic migration of volumes to Metal. I'm on EU Amsterdam.

The two services concerned :

Environment : Production

Services :
- WordPress (Failed)

- MySQL (Failed)

Thanks in advance.

Solved

18 Replies

btnalexandre
PRO

2 months ago


Railway
BOT

2 months ago

Hello!

We're acknowledging your issue and attaching a ticket to this thread.

We don't have an ETA for it, but, our engineering team will take a look and you will be updated as we update the ticket.

Please reply to this thread if you have any questions!


2 months ago

Heya, sorry about that, looks like you're back up now. I've flagged this w/the team so they can take a look at what happened.

Did you experience any downtime from these ?


Status changed to Awaiting User Response Railway 2 months ago


itsrems

Heya, sorry about that, looks like you're back up now. I've flagged this w/the team so they can take a look at what happened.Did you experience any downtime from these ?

btnalexandre
PRO

2 months ago

Hello !

No downtime now, have you redeployed the services ?
The migration to metal is completed for me ?

Thanks !


Status changed to Awaiting Railway Response Railway 2 months ago


2 months ago

The migration failed so unfortunately we'll need to run it again. it should be smooth and happen in the next few days


Status changed to Awaiting User Response Railway 2 months ago


itsrems

The migration failed so unfortunately we'll need to run it again. it should be smooth and happen in the next few days

btnalexandre
PRO

2 months ago

Okay thanks. Let me know when it’s done


Status changed to Awaiting Railway Response Railway 2 months ago


Status changed to Solved itsrems 2 months ago


btnalexandre
PRO

2 months ago

@itsrems you mark this thread as solved but are the services migrated ?


Status changed to Awaiting Railway Response Railway 2 months ago



btnalexandre
PRO

2 months ago

And we've had poor performance on WordPress for the past 2 weeks. Could this be due to an incomplete migration?


2 months ago

Are you still running into performance issues this week / today ? we had some performance issues with metal volumes, but applied fixes across our fleet earlier this week.


Status changed to Awaiting User Response Railway 2 months ago


btnalexandre
PRO

2 months ago

Hello, today this seems better. But I need some days after a redeployment to see if performance issues have gone. I'll let you know in a few days.

Have you some news regarding the migration issue that occurred a second time this morning for this service : https://railway.com/project/ad2fe3e4-05ab-4d67-a5da-87f5bb5bf1ee/service/932c4061-a767-45c1-b821-592845724010?environmentId=71e28411-a28b-4e00-9928-0d3e929d942c

Thanks in advance !


Status changed to Awaiting Railway Response Railway 2 months ago


2 months ago

We're moving some workloads between our hosts, and a subset of migrations fail. However these don't affect your service, and shouldn't show up in your dashboard (as we retry them as needed). Apologies for the confusion !


Status changed to Awaiting User Response Railway 2 months ago


btnalexandre
PRO

2 months ago

Okay thanks for this information ! From what you can see, have I migrated everything to Metal (I did it manually before Railway's intervention)?


Status changed to Awaiting Railway Response Railway 2 months ago


2 months ago

Yup all on metal ! The failed migration you saw was from us moving some workloads from metal to metal.


Status changed to Awaiting User Response Railway 2 months ago


itsrems

Yup all on metal ! The failed migration you saw was from us moving some workloads from metal to metal.

btnalexandre
PRO

2 months ago

Thanks for your quick reply. I will reopen this thread if I encounter an another issue or if performance issues persist


Status changed to Awaiting Railway Response Railway 2 months ago


Status changed to Solved btnalexandre 2 months ago


itsrems

Yup all on metal ! The failed migration you saw was from us moving some workloads from metal to metal.

btnalexandre
PRO

2 months ago

Hello, another news this night the service WordPress (on production) was down between 04:15:39 AM CEST and 04:50:19 AM CEST, today.

On the service I see the error again "Volume migration failed."
Could you investigate how to get rid of down due to a failed migration?

Thanks


Status changed to Awaiting Railway Response Railway about 2 months ago


Railway
BOT

2 months ago

Hello!

We've escalated your issue to our engineering team.

We aim to provide an update within 1 business day.

Please reply to this thread if you have any questions!

Status changed to Awaiting User Response Railway about 2 months ago


btnalexandre

Hello, another news this night the service WordPress (on production) was down between 04:15:39 AM CEST and 04:50:19 AM CEST, today.On the service I see the error again "Volume migration failed."Could you investigate how to get rid of down due to a failed migration?Thanks

chandrika
EMPLOYEE

2 months ago

Hi there, I've fixed your volume migration — you should be all set ! Could you please confirm that everything is working smoothly now?


btnalexandre
PRO

2 months ago

Thanks that seems good ! Have a nice day


Status changed to Awaiting Railway Response Railway about 2 months ago


Status changed to Solved btnalexandre about 2 months ago


My applications automatically redeployed this morning and crashed with: Volume migration failed - Railway Help Station