2 months ago
I have a Railway-managed MySQL database on US West (California).
The database is healthy:
- Service status: Running
- Logs show "ready for connections"
- Private networking works (mysql.railway.internal)
- My website connecting from within Railway via MYSQL_PUBLIC_URL works
But ALL public TCP proxies time out from external networks:
- Original proxy: shortline.proxy.rlwy.net:44654 → timed out
- Deleted and re-created → sakura.proxy.rlwy.net:25690 → timed out
- Deleted and re-created again → tokaido.proxy.rlwy.net:51392 → timed out
Error: 2003 (HY000): Can't connect to MySQL server (10060 - connection timed out)
To confirm it's not my network or firewall:
- I created a NEW MySQL service in the same project
- Got kodama.proxy.rlwy.net:57604
- Connected successfully in < 2 seconds
So the issue is specific to this one MySQL service's TCP proxy routing.
What I've tried:
- Redeploy MySQL service
- Delete + re-add Public Access (3 times, all different addresses)
- Multiple VPNs from different regions
- Railway CLI tunnel (railway connect MySQL --tunnel-only) → also timed out
- All fail on the old service; new service works perfectly
Could you please investigate the TCP proxy routing for this specific MySQL service? My data is on this service and I cannot access it externally.
1 Replies
2 months ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 2 months ago
2 months ago
This isn't your network, your client, or your config — you've already proven that with the new service working instantly. What you're hitting is a stuck routing binding for that one MySQL container that survives proxy recreation, because recreating "Public Access" just gives you a new edge hostname — it doesn't touch the actual backend mapping between the proxy layer and your container. That's why three different proxies (shortline → sakura → tokaido) all fail identically, and why the CLI tunnel fails too (it rides the same backend path, not just DNS).
Two things to try, in order:
- One more redeploy — but only after the current proxy exists.
There's a known quirk where a freshly-added TCP proxy doesn't actually bind until the service does a fresh deploy on top of it. If your "redeploy" happened before you added the current (tokaido) proxy, do it again now, with the proxy already attached. If you already did it in the right order, skip this.
- If that doesn't fix it, this is a Railway-side stuck proxy, not something you can self-serve.
There's precedent for exactly this shape of bug — proxy/routing breaks after an infra event, private networking stays fine, and recreating the proxy does nothing because the fix has to happen on Railway's backend (restarting the binding at the host level), not from your dashboard. Same root cause as regional "unhealthy proxy" incidents Railway has had before, where a specific proxy node gets pulled from rotation by staff — user-side actions don't touch it.
Don't wait on that ticket to get your data, though — you don't need public access for this:
Since private networking already works fine and your new MySQL is in the same project, you can migrate directly over the internal network without ever touching the broken proxy:
Deploy a throwaway one-off service in that project (any image with mysql-client, or just a minimal Docker service)
Run this from it, pointing at both private hostnames:
mysqldump -h mysql.railway.internal -u root -p<OLD_PASS> railway | \
mysql -h mysql-new.railway.internal -u root -p<NEW_PASS> railway
Swap your app's MYSQL* env vars to the new service's values
Verify data, then delete the throwaway service and the broken MySQL service
Then file the ticket on Central Station with: project ID, the broken MySQL service ID, and all three proxy hostnames you cycled through, so their infra team can actually root-cause why the binding is stuck — you'll be unblocked already by the time they get to it.