7 hours ago
Since ~14:10 UTC on Oct 3, my Postgres service shows "Online" and its logs say "database system is ready to accept connections", but it refuses every connection:
Private network (postgres.railway.internal:5432) from my web service: connection timed out.
Public TCP proxy (metro.proxy.rlwy.net:45484): "server closed the connection unexpectedly".
The dashboard Data tab is stuck on "Attempting to connect to the database…".
Earlier the DB was interrupted at 13:45 UTC and recovered cleanly at 13:55 UTC (logs show redo done + checkpoint complete). Clicking Restart returns "Problem processing request", and a Redeploy has been stuck for 15+ minutes. Please check the host/volume — the app is down for all users.
App Request ID: nRRgO7uEQLOYLPUP2hOiww
2 Replies
Status changed to Awaiting Railway Response Railway • about 7 hours ago
5 hours ago
Your original Postgres volume is on a host in US West that became unresponsive at about 14:15 UTC and was restarted at about 16:30 UTC. That's why it showed Online but refused every connection, why Restart errored, and why the redeploy hung: a database has to start on the same host as its volume, and that host was down.
The host is back now, and the original volume is still attached to your Postgres service. Redeploy that service once to bring it back on its existing data.
We can see you've also created a second database, Postgres---4a, which has its own separate volume. Before switching your web service back, decide which of the two it should point at, since anything written to Postgres---4a in the meantime won't be in the original.
It's covered by this incident: https://status.railway.com/incident/72DDHCC1
Status changed to Awaiting User Response Railway • about 5 hours ago
nico
Your original Postgres volume is on a host in US West that became unresponsive at about 14:15 UTC and was restarted at about 16:30 UTC. That's why it showed Online but refused every connection, why Restart errored, and why the redeploy hung: a database has to start on the same host as its volume, and that host was down. The host is back now, and the original volume is still attached to your Postgres service. Redeploy that service once to bring it back on its existing data. We can see you've also created a second database, Postgres---4a, which has its own separate volume. Before switching your web service back, decide which of the two it should point at, since anything written to Postgres---4a in the meantime won't be in the original. It's covered by this incident: https://status.railway.com/incident/72DDHCC1
5 hours ago
Hi Nico, thanks for the clear explanation and for confirming the incident.
I'll keep the web service on the new database (Postgres---4a). It was restored from a backup taken from the original database right after it recovered (~14:12 UTC), and my app couldn't write anything after ~13:45 UTC, so the original has no data that the new one is missing. Since then, all new data has gone only to Postgres---4a.
So I won't redeploy the original Postgres. I'll delete it (and its volume) in a few days, once I'm sure everything is stable.
Thanks again for the help!
Status changed to Awaiting Railway Response Railway • about 5 hours ago
Status changed to Solved Railway • about 5 hours ago