313 Replies
4 hours ago
Team is aware and looking into it
I noticed this in my mysql db's logs:
> 2026-02-11T14:55:06.361425Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user . Shutting down mysqld (Version: 9.4.0).
Is this mainly related to internal networking, otherwise I will start rerouting via public network?
4 hours ago
It seems related to processes being terminated, team is looking into it, but no harm in trying.
4 hours ago
Can you share your service URL? The one that appears in the URL bar (the one that starts with railway.com…) when you have the service in focus.
4 hours ago
Seeing this too with my Caddy server. Keeps getting shutdown
This MongoDB indeed seems to have been shutdown: https://railway.com/project/e843ddc7-071d-430c-be13-52ab4fe102c0/service/d130e10f-dd87-45a2-aa5e-cc3ce0cd8798?groupId=6e585a96-f33a-482d-817b-417a7dd8eb42&environmentId=40e37c2c-937f-4965-89a2-76389262b957&id=7d81b9d3-3be6-40bd-9517-db468ec6a018#deploy
I've tried redeployment, restart, different instances, nothing fixes it
4 hours ago
Same here, the same day we launched a campaign :(
4 hours ago
Attachments
if I restart my mysql db it seems to work for a minute or two, then it's shut down again
I'm getting massive response times suddenly, nothing seems to connect to my services either
4 hours ago
Also having issues
networking looks bust and we've randomly got containers that have been terminated.
4 hours ago
same problem here.
4 hours ago
🚨|incidents has been updated, we just have to be patient while the team fixes it
4 hours ago
None yet, and for updates be sure to look into the incident: https://status.railway.com/cmli5y9xt056zsdts5ngslbmp.
I have an uptime kuma app with sqlite db for monitoring and also this is going down. so it looks like it is related to volume mounting.
4 hours ago
For reference, the issue seems related to processes being killed randomly, not related to volumes.
4 hours ago
None at the moment, sorry.
4 hours ago
all our databases are down and our clients cant acces systems
Each time i stick with railway cuz i like the service but i can't fucking trust it if it keep going down every month
We're experiencing similar issues with our internal networking when connecting to both MongoDB and Postgres. All instances were running without any problems until approximately two hours ago, when we started encountering connection timeouts.
Same here!
Attachments
4 hours ago
It's not related to internal networking.
Is your postgres server actually running though? In all of my services it doesn't seem to be networking related, it's that the services are receiving a shutdown command after a few minutes of being up.
now the services is just crashed randomly
for some reason, it become crashed, just crashed
without any logs
4 hours ago
The team will post updates on here: https://status.railway.com/cmli5y9xt056zsdts5ngslbmp. Really sorry for that.
4 hours ago
It truly is, again really sorry for that
Yeah, the status update is deceptive. All our services are hard down, period.
fair. i would say that builds failing doesn't fully describe the issue. some existing services went down as well
which is why i saw the status page and assumed it didn't impact us
Yeah this is not just deployments this is a major service outage effecting my customers
4 hours ago
Can agree here, told the team if they can update it
4 hours ago
There's an incident called: https://status.railway.com/cmli5y9xt056zsdts5ngslbmp. Updates will be posted there.
4 hours ago
A new update has been posted, they're still looking into that.
since June last year, after migrate to the METAL stuff,
it start on and off having the outrage
average 1 month per incident
4 hours ago
From the issues I've been observing, it seems related to the new influx of users to the platform. Been awhile since we got a major outage that affected running workloads (not builds).
4 hours ago
i just restarted my MySQL db and it appears to be online again.
yeah this one is different, it affects any services, even the already-running ones.
A bit more than 1 month ago, all my services went offline, node js mongo etc etc
Internal routing is down as notionally my app is up but can't route to the DB. Nothing to do with deployments
4 hours ago
Team is trying their hardest to fix it and I can agree that it's unacceptable, they'll be posting a post-mortem on this too.
I believe the services have been restored. Please 👍🏻 or 👎🏻 this message if it works for you or not
Things happen, as long as it gets fixed and there is an understanding on it and how to avoid it again, all is good - at least from my pov
4 hours ago
It's not related to internal networking, the processes themselves seem to be killed randomly.
What a joke. Responding quickly is not enough. It's one outage a month at this point. Not at all usable.
Ah! correct, my db was showing online but clicking into it shows deployment is offline - my node service then can't route to it for obvious reasons!
Can you please update the status. "We're investigating issues impacting deployments" is very incorrect. This is a complete outage. It's deceptive to write it this way when it's clear to everyone the impact is far wider than deployments.
O problema aparentemente é apenas no Build com Metal, desabilitei o uso e meus sistemas voltaram a funcionar normalmente, apenas ao desabilitar o Use Metal Build.
been thinking is it worth it for migrate to the METAL thing. before this is just fine. lolz
4 hours ago
I've passed this down to the team to update it, they're looking into it!
it has become a major outage
Attachments
Railway's postmortems are normally pretty good! Certainly more detailed than many other providers cough AWS cough.
December 16 outage was exactly the same @ThallesComH 'minor outage' - when everything was down. This is systematic dishonesty.
if it helps anybody, my servers are on Legacy, not on Metal! And they're no longer dying every 5 mins.
What GPT had spotted in the logs was that 270GB of physical storage were assigned to the service, so I do believe that as all our projects grow, once in a while Railway will be running out of storage
4 hours ago
I remember Railway having an issue with elevated networking latency, but nothing as critical as this, but I can understand your frustation if that affected your workload, sorry for that.
damn, i still remember that,
the connection is suddenly go for 3s after migrate,
super lagging, damn scary
I have an email from December 16 saying the following, and it definitely wasn't just build failures, perhaps because it was European time but everything was down
"Hello,
We recently experienced a major outage that impacted your Railway workloads. We're very sorry for the disruption this caused.
Our goal at Railway is to provide a best-in-class experience, and this incident fell short of that standard. To help make things right, we've applied a credit based on your average bill from the last three months (20) to your workspace, Projects. You can view your available credits here.
For a detailed breakdown of what happened and the steps we're taking to prevent this in the future, you can read the full post-mortem here."
allow to download database backups, we need an anternative to get back to work
4 hours ago
For reference, we run our entire company on Railway and we weren't affected by that issue (to the point of our services going down). I apologize if I gave a different impression, it was not my intention to be dishonest.
really, it dont have that. sad
Attachments
can this be changed at least? certainly not caused by the application, it's the fault in hosting provider. 🙁
Attachments
4 hours ago
Every workload on Railway runs on metal, there's no "non-metal" option.
4 hours ago
That's a great feedback, will check with the team!
Idt we even received credits even though all our workspaces were down then
when there's a major outage like this, it would be great to at least be notified by railway so I can at least push messaging to my client notifying paying users about our service outage
my services are back now, i'm not using metal, but my apps werent connecting to my database, now they are able
yes please, a link pointing to the status page would be great. that will at least mitigate the damage for end users a bit, at least they know that the fault is happening on the hosting provider rather than the developers.
4 hours ago
You can subscribe to incidents on and, in case you use Discord, you can follow the #🚨|incidents channel.
We've got clients calling and are pretty annoyed that it's the second time in the past months. It's looking like we might have to change provider in good faith.
Even subscribing to it wouldn't do much, clients will probably call before that
4 hours ago
This is a monthly occurrence now. Your customers deserve a clear write up of what happened, and what steps you're taking to fix it.
4 hours ago
I can understand your frustration. Our company also runs on Railway, and we're just as upset as you are about this situation. However, we also recognize that outages are inevitable in any service and we love Railway way too much to leave.
We genuinely don't care if some stuff on your timeline takes longer to get added, we just don't want to have a major outage each month
yes, I did that once I found out about the incident. but I only found out about the incident 30min after it started because I happened to open my site to test something
3 hours ago
I can guarantee that the team will prioritize stability over features at any time. For reference, the build issues we were having was their top one priority.
Do you have a plan for resolving the problem? Will the data be accessible? Answers are needed; several critical systems are currently down. I need to know if the data is secure.
The benefit to Railway as a developer is amazing, to the client, they just don't want their website to go down and we provide client services so our opinions are less important. It makes it difficult to justify to client why to continue using an unstable service just because I personally like it.
3 hours ago
Our redis service shutdown and therefor got wiped.
3 hours ago
Can agree here, but I don't think we would have gotten our product to the point it is now without Railway. Every time we don't have to worry about infrastructure is time we get to ship. I can guarantee you that Railway is actively working on improving stability.
RESOLUTION turn this OFF.
DONE. 🫰
Attachments
it seems to be working again, now I have to deal with a lot of angry customers :|
3 hours ago
I don't think that's guaranteed, since some folks (including myself) saw improvement turning it on. In general, I'd use the Metal build environment
3 hours ago
Have you restarted it?
2026-02-11T15:52:23.923886Z 1 [ERROR] [MY-012574] [InnoDB] Unable to lock ./ibdata1 error: 11
2026-02-11T15:52:24.081102Z 1 [ERROR] [MY-012574] [InnoDB] Unable to lock ./ibdata1 error: 11
3 hours ago
Pretty sure that disabling metal build isn't doing much here, as your service received a redeploy which seems to be temporarily fixing the issues for some users. Please keep metal build enabled, it's much faster.
I don't doubt that the team is working on fixing issues on Railway but the fact is that major outages are occurring every month
Railway can of course fail, like every service. But at least they hear out and actively fix our concerns. AWS would just put all of us on voicemail…
3 hours ago
I can assure you they're aware. We have an internal chat with the team and receive updates on any issues they're actively fixing. One of the examples was the GitHub authentication problem where they told what was happening and the steps they're taking to fix it.
And when it does it's so bad that our clients are more accepting of it not being our fault
That's what I did in my database and applications, and all 6 had Metal enabled for testing. After disabling it, they all started working again right now. I'm telling you what I did and it worked. What you think based on guesswork without testing is just guesswork.
This can be re-enabled again after the problem is solved, and the problem is with Metal; it's on the resolution page!
Attachments
3 hours ago
Every workload runs on Railway Metal, which is Railway's own server infrastructure. Service builds were previously running on GCP, and now they're experimenting with running these builds on their own servers as well. Whether your server is built on Metal or GCP should not affect the outage.
3 hours ago
Unfortunately we've to wait until the incident is fixed.
Excuse me, I don't understand. I don't have any Metal services; they're all legacy, and my MySQL database wont starts. Is anyone else having problems with the database?
3 hours ago
A one-hour outage is unacceptable...when will the problem be fixed and will a detailed incident report be released?
3 hours ago
Have you run railway services on GCP? Metal's much more stable.
3 hours ago
"Legacy" services are still running on Metal iirc, it's just the runtime that's legacy
What I did worked, and I shared it. I'm fine, do whatever you want. For me, it's already resolved. I don't care if it's just guesswork.
Yes, but only the applications on metal were affected. I disabled them until they fix it; it's a temporary solution. It doesn't matter what's best right now, what matters is what's working.
3 hours ago
Hey, just got confirmation from the team that redeploying the affected services should fix the issue for some users (this explains why @Gandalf service is working, which is not related to Metal Builds). Just doing a redeployment on each service for example your API and database should fix the issue for some.
3 hours ago
Not a great solution, I can agree, but the team is still working on a fix for everyone else
I did a simple "restart" of the already deployed SQL service and everything came back for me
3 hours ago
Yep, restart also works
I redeployed however I am still running into 502's with my caddy proxy to my services
3 hours ago
after restart mysql service it started working again
3 hours ago
All my apps are down, servers and job applications are out. @Railway address promptly.
3 hours ago
Hey, can you try redeploying each of your services as mentioned above? That should fix for some users.
I cant redeploy the database, it would erase it… can only restart it I presume.
I did restart it
Attachments
3 hours ago
same here. databses down
3 hours ago
If you've a volume attached to your service, that shouldn't happen. If you don't know what a volume is, it's this small "card" displayed below your service.
Attachments
3 hours ago
Hey, can you try redeploying each of your services as mentioned above? That should fix for some users.
for me the data wasn't corrupted or removed. I'm still checking the integrity, but based on what I see - data seems to be okay
this? @ThallesComH
Attachments
3 hours ago
Stuck with development postgres server not being able to be connected to via backend. Prod still up.
Database Connection
Attempting to connect to the database...
3 hours ago
Yep, you can redeploy it no problem.
3 hours ago
Hey, can you try redeploying each of your services as mentioned above? That should fix for some users.
Ok let's go
Attachments
Whoever has a database and the data in a single deployment should be able to SSH in to it right? Though no one should be doing that ever.
3 hours ago
Hey, can you try redeploying each of your services as mentioned above? That should fix for some users.
3 hours ago
Maybe, but I don't see why you would need to SSH now though.
Just had three client websites crash and the automatic restart policies didn't work at all.
Just an idea but people who use Railway are people who are not comfortable with infrastructure or understand it completely. Railway creates this abstraction and makes it visually easy to understand. But this doesn't mean you can have completely no knowledge of infrastructure like Vercel, It's still involved.
https://svelte.dev/tutorial/svelte/welcome-to-svelte
Svelte has this really nice interactive tutorial… Imagine if Railway had this.
3 hours ago
For people still having issues after restart/redeploying, please open a #✋|help thread and I'll escalate it to the team.
Confirming that redeploying my Postgres and app services brought them back online. Thanks @ThallesComH 👍
I don't think https://discord.com/channels/713503345364697088/1471157274314539098/1471171555592638698 is the resolution at all.
Can you please confirm we can safely redeploy a Postgres DB without affecting it?
3 hours ago
Railway has an SSH option by using the CLI, just type railway ssh.
3 hours ago
Can you share a screenshot of your service on the canvas?
I just did and it's fine - the service mounts a persistent volume that survives the redeploy.
3 hours ago
You can blur it out but shouldn't be a problem.
3 hours ago
or temporarily renaming it to another name
redeploying the web app part
Attachments
3 hours ago
You can redeploy it, no problem.
3 hours ago
Yep
3 hours ago
Error de aplicación: se produjo una excepción del lado del cliente al cargar station.railway.com (consulte la consola del navegador para obtener más información) .
Help, i can't redeploy
Attachments
3 hours ago
Hey, can you try switching to using railpack instead of nixpacks? You can change by that going to your service settings
Still
Attachments
This is really bad, it's effecting all our services that use MYSQL, tried restarting MYSQL, which seems to fix the problem. Any updates? Please focus on reliability, I really love Railway and don't want to migrate to another platform. Client's are starting to notice that things are not working
can i turn on my redis again already? it was really slow/refusing connection.
But it shows that the service is still online
Attachments
It has been more than 10' since I hit redeploy. Aborting and trying again. What a mess.
3 hours ago
Something like this is unacceptable, I've been offline for 2 hours
@ThallesComH Still down, cannot redeploy the DB
Attachments
3 hours ago
Ya estoy perdiendo clientes, muy enojados. Que va hacer railway al respecto? ahora como vuelvo a convener a los clientes?
I'm already losing customers, and they're very angry. What is Railway going to do about this? How am I going to win them back?
3 hours ago
Ya estoy perdiendo clientes, muy enojados. Que va hacer railway al respecto? ahora como vuelvo a convener a los clientes?
I'm already losing customers, and they're very angry. What is Railway going to do about this? How am I going to win them back?
3 hours ago
Hey folks! Can you all try triggering redeploys on your services or anything dependent that's running into this? Say you have an API that depends on Redis and Postgres—try triggering a redeploy on Postgres, Redis, and the API service. That should fix it.
To make sure we get back to all of you, can you please create your own station thread? Want to make sure we miss nobdoy and everyone gets in a good place
It has been like that for ages
Attachments
3 hours ago
For reference, Noah will be replying to this thread from now on, I really need to get back to work, sorry if I didn't reply to someone before!
Cannot redeploy a mysql DB located here
Attachments
Attachments
2 hours ago
I am still down. My POSTGRESS DB is un reachable in 2 of my 3 environments. A major client of mine is offline..... Restart doesnt work and a redeploy just hangs..
My services are backup but I'm getting, which isn't true, it's back up. Though that's just a UI thing.
Attachments
Attachments
I dont have a mysql db myself, only a redis service. I managed to redeploy it fine.
2 hours ago
Ya, lots stuck for me too (stuck ones seem to be ones with a pre-deploy script)
Attachments
Is there some official rep from Railway in this thread who can update us on the actual issue?
im having the same problem, both services are online but unable to communicate each other and there's no log nothing only in the db says
Attachments
Attachments
2 hours ago
on "Creating Containers"? That normally means a busy queue from people redeploying after the incident, it will eventually deploy.
Attachments
any help with this?
https://discord.com/channels/713503345364697088/1471181106668765204
2 hours ago
Yep, sorry for the incovenience, this can happen after an incident, people spam builds and the queue gets crowded
2 hours ago
Please use restart, not redeploy.
Yeah, i a restarting redebploying for like 2 and a half hours. Still not working
Anyone can help me i have pro plan
Attachments
2 hours ago
Restart and Redeploy not working for my MySQL. + 2 hour offline!!!!!
FAILED after 17'
Attachments
2 hours ago
Nope no need to delete.
Can you please select the stuck/offline service, abort any running "redeploys" or "builds" and press "command/ctrl + k" and select "Deploy latest commit" or if a docker image, "redeploy source image"
2 hours ago
That will deploy a fresh variation.
2 hours ago
can you link?
2 hours ago
restarting the affected services worked for me. Thanks for the fix.
2 hours ago
If you select that deploy and copy the URL that is in your browsers search bar that'd be what I'm looking for
2 hours ago
So sorry y'all hit this. We'll be providing a very detailed post mortem of what happened
2 hours ago
2 hours ago
i tried to redeploy and restart, but nothing happened
2 hours ago
Restart and redeploy not working for MYSQL!
3 hours offline.. bye bye railway
2 hours ago
Frustrating. When is it going to be fixed?
This may be the thing to push people over the edge. This is the second such outage in 12 weeks. Especially frustrating that Railway have recently raised £100 MILLION dollars Series B funding. I can't build a business around these kinds of failures. Appreciate all Railway are doing to mitigate but this is costing folks livelihoods.
2 hours ago
regardless of the ReDeploy apps dont work. This S4cks. Cannot be happenning. I need to launch something extremely important and you guys are down.
2 hours ago
Nothing has been working for three hours now, and this situation occurs consistently once a month. What is the problem? I am losing customers.
2 hours ago
Restart and redeploy isnt working.. My clients are all DOWN. This isnt resolved.
2 hours ago
At least one of my services is still down, and cant redeploy (have tried restart multiple times).
It fails whenever it needs to talk to its Postgress
https://railway.com/project/5dfc53a8-dea9-46af-b0a0-536e51368af3/service/eca68ad2-7ea4-4e75-9918-6597e05b4fca/database?groupId=b412cd06-ef5e-487f-82a1-e12ff2af0268&environmentId=2d2473e5-9346-4932-9ab3-7c2c014f8ba6
2 hours ago
People that are still having this issue, please open your own separated help thread and provide affected service links.
