9 months ago
Project ID: 1ad2e8d9-d8df-4cd2-aca2-4be43f7219d3
Our API service started failing deployment on our production environment with a strange error.
It apparently thinks it's in Region "0" instead of the EU Metal region where the current active deployment is.
It also randomy started to ignore the config file in our repo (switched back to Nixpacks instead of Docker) and throwing errors like "Cannot create code snapshot right now" on further deployments.
I guess this service is in a broken state but as it is on our production environment, I cannot just delete and recreate it and wait for a DNS propagation to happen on the API's domain.
0 Replies
9 months ago
Happened to me too, very weird behaviour
I'll go ask about it
9 months ago
Team's working on a fix
9 months ago
Sorry for the trouble
9 months ago
Angelo from the team explained what happened here if you're curious
9 months ago
Hey @alex73630 -
We just got the fix in, we are going to be doing a few things.
Railway sparingly moves workloads around as a function of our ability to handle capacity. Box gets full, we move 'em. Box needs an upgrade? Move 'em.
This time, we did a type change on how we handle multi-region deployments because we need to keep track of regions across cloud providers GCP <-> Railway Metal
However, because we didn't account for that, we sent them to a new, un-working region: Region 0
Moving forward, we will:
Put a Railway banner whenever the platform takes an action on their behalf. Avoiding confusion if we do a restart.
Not send them to Region0
In your case, your existing workload is fine an operable yes?
9 months ago
New deploys should have gone out just fine.
Hey Angelo,
Thanks for the response.
Our current deployment is working as expected, only the newer deployments have failed to even build.
Luckily they didn't contain any new backend code and so it wasn't a huge issue.
I'll try to do trigger a deployment to see if it's fixed.
9 months ago
Thanks for using Railway
(and reporting)
ty @Fragly for escalating it to us
My level of trust in Railway is the same as my current Metal usage, 99% 😄
Keep up the great work!

9 months ago
!s
Status changed to Solved dev • 9 months ago