Production MySQL down on Railway. Service restart hung 10+ min on "Mounting volume". DB unreachable.
vonsa
PROOP

a month ago

Hi,

ProjectId: d9ac9779-d4c6-45b2-83e6-ee4a5c70b424

Service: MySQL 8.4.3, Pro plan, 50GB volume

Started: ~11:30 UTC, 4 May 2026

We started experiencing issues with the DB today:

All INSERT/UPDATE queries hung 30s–1min+ ("waiting for handler commit")

GET/SELECT queries worked normally

SHOW ENGINE INNODB STATUS showed only 0.66 fsyncs/sec and 5 writes/sec

Current state (I attempted restart twice through the UI):

MySQL shutdown completed at 11:49:05 UTC

After first restart, nothing mounting. After second restart, deploy log stuck on "Mounting volume on: /var/lib/containers/railwayapp/bind-mounts/..." for 10+ minutes

DB unreachable, production app down

Email support 48h isn't viable for this prod outage

Changes I made today (in order):

SET PERSIST max_connections = 400 (was 151)

SET PERSIST innodb_buffer_pool_size = 4GB

SET GLOBAL sync_binlog = 100 (was 1)

SET GLOBAL innodb_flush_log_at_trx_commit = 2 (was 1)

Triggered service Restart at 11:49 UTC

8 Replies

vonsa
PROOP

a month ago

We have to resolve this ASAP. I'm already looking into migrating to an alternative DB solution. Not sure why 48 hour response time for a prod issue? I don't think that's viable for any professional usage.


laurentcompania
PRO

a month ago

Same exact issue here, started 05:18 UTC today on a MySQL service in europe-west4 — waiting for handler commit blocking all UPDATEs, SELECTs fine. My app went down too.

Restart got stuck on Mounting volume on: /var/lib/containers/railwayapp/bind-mounts/... for ~30 min before unblocking on its own (probably Railway intervened on the host). Then InnoDB crash recovery took another 12 min before MySQL accepted connections again. So total ~45 min of downtime, but it did come back without data loss.

Two services with identical symptoms on the same day → this looks like a host-level storage incident, not anything we did wrong. Hang in there, and definitely push Railway to confirm and migrate you to a healthy host.


lonasozo
PRO

a month ago

Yea, same here, unable to mount the mysql database!


marcellov7
PRO

a month ago

We're having the same problem here. He's been trying to mount a volume for 3 hours without success.

No response from support regarding something this serious!

We don't know what to do, and we're under pressure from our customers.


iyimiyiz-admin
PRO

a month ago

same here.


vonsa
PROOP

a month ago

Hi all,

This worked for me:

1. Create a new backup

2. Restore the backup

After creating a new backup, I also created a new volume, not sure if that matters. I didn't use it.

And as it seems to be a bit broken, I hope all data was backed up correctly.


mativm02
PRO

a month ago

I'm also having some issues with two different services that connect to two different Redis instances (getting some timeouts, started 4 hours ago)


a month ago

Team has been made aware and incident has been called: https://status.railway.com/incident/QK1OHYXB


Welcome!

Sign in to your Railway account to join the conversation.

Loading...