MySQL service attached to persistent volume is unavailable.
cleitonsanches
HOBBYOP

14 days ago

Facts:

Volume still exists and contains approximately 315 MB of data.

Service shows Active deployment but Console reports container status as "stopping".

Database UI cannot connect.

External clients (DBeaver) cannot connect.

Application returns HTTP 503 because database connection fails.

Recent redeploys are being reported as manually cancelled, although no explicit cancellation was performed.

Network Flow Logs still show TCP activity on port 3306.

No deployment/runtime logs are available.

Can you verify the container status and whether the volume is healthy and still attached correctly? We need to recover access to the existing database data stored on the volume.

you can also verify the deployment ID (b54ef9a4-93d1-4d7d-8cda-7604a187a5e0). The repeated CREATE_CONTAINER hangs on the same MySQL service suggest an infrastructure-level problem that needs platform investigation. My config is valid; this is not something you can fix on your end.

$10 Bounty

5 Replies

Status changed to Awaiting Railway Response Railway 14 days ago


13 days ago

Does a re-deploy of your database help?


Status changed to Awaiting User Response Railway 13 days ago


ray-chen

Does a re-deploy of your database help?

cleitonsanches
HOBBYOP

13 days ago

No. I've tried to re-deploy twice and, as i said, they are being reported as manually canceled.

SNAPSHOT_CODE completes instantly ✓

CREATE_CONTAINER enters pending state and never completes

No build logs, no deploy logs, no error recorded

No failure stage or failure reason recorded

This pattern repeats across multiple redeploy attempts

What I've already tried:

Redeployed the service twice

Waited 12+ minutes for container creation to complete

Verified service configuration is valid (volume mount, TCP proxy, env vars all correct)

Confirmed the image (mysql:9.4) resolves successfully

Why this isn't a config issue:

The last successful deployment (April 27) used identical configuration

SNAPSHOT_CODE completing proves the image pulls successfully

No configuration errors are reported

Current status: The deployment is stuck indefinitely in CREATE_CONTAINER. Redeploying again will likely result in the same hang.

Request: Please investigate why CREATE_CONTAINER is hanging on this MySQL service. This appears to be a platform-side infrastructure issue, not a user configuration problem.


Status changed to Awaiting Railway Response Railway 13 days ago


Railway
BOT

12 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway 12 days ago


cleitonsanches
HOBBYOP

12 days ago

Thank you.

I appreciate opening the thread to the community, but I believe this issue requires investigation by the Railway infrastructure team rather than community troubleshooting.

Here's why:

  • The MySQL service had been running normally for months without configuration changes.
  • The persistent volume is still attached and contains approximately 315 MB of data.
  • The application deployment is healthy (Vercel deployment succeeds), but the application cannot connect because the database is unavailable.
  • The Railway Database UI cannot connect to the service.
  • External clients (DBeaver) also cannot establish a connection.
  • The service console reports "container stopping" / "connection lost", while the deployment appears as Active.
  • There are no MySQL startup logs available to diagnose the issue from the user side.

At this point I don't have any additional actions available. The database is inaccessible through every available interface.

Could someone from the Railway team please verify the health of this service internally, including:

  • the container lifecycle;
  • the persistent volume attachment;
  • the internal MySQL process status;
  • and any infrastructure events affecting the us-west2 region?

This database is the production environment for our application, so any assistance would be greatly appreciated.

Thank you.


ray-chen

Does a re-deploy of your database help?

cleitonsanches
HOBBYOP

12 days ago

Hi @ray-chen,

Just following up on my reply from yesterday. I'm not sure if you had a chance to read it yet.

Unfortunately, the issue is still ongoing and the database remains completely inaccessible. I have already tried multiple re-deploys, but they did not resolve the problem.

This is our production database, and our team is currently unable to use the application because of this incident.

Could you please let me know if someone from the Railway team is investigating this internally, or if there is any update you can share?

I would really appreciate any assistance, as this has become quite urgent for our operation.

Thank you.


ankitgelda8
HOBBY

7 days ago

Create a new instance of mysql and see if Railway allows you to attach the existing volume , not sure if this help if not Migration will be the only thing you can do check with railway support if they can help for backup the data


Welcome!

Sign in to your Railway account to join the conversation.

Loading...