database system is shut down
peterjkim83
HOBBYOP

3 months ago

postgres crashing with data is corrupted error, on the hobby account so no backup to work with, any ideas how I can resolve this issue?

2026-03-03 10:21:56.513 UTC [2] LOG: starting PostgreSQL 16.11 (Debian 16.11-1.pgdg13+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 14.2.0-19) 14.2.0, 64-bit

2026-03-03 10:21:56.513 UTC [2] LOG: listening on IPv4 address "0.0.0.0", port 5432

2026-03-03 10:21:56.514 UTC [2] LOG: listening on IPv6 address "::", port 5432

2026-03-03 10:21:56.527 UTC [2] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

2026-03-03 10:21:56.546 UTC [25] LOG: database system was interrupted while in recovery at 2026-03-03 10:21:44 UTC

2026-03-03 10:21:56.546 UTC [25] HINT: This probably means that some data is corrupted and you will have to use the last backup for recovery.

2026-03-03 10:21:57.075 UTC [25] LOG: database system was not properly shut down; automatic recovery in progress

2026-03-03 10:21:57.085 UTC [25] LOG: redo starts at 0/7E535D10

2026-03-03 10:21:59.671 UTC [26] FATAL: the database system is not yet accepting connections

2026-03-03 10:21:59.671 UTC [26] DETAIL: Consistent recovery state has not been yet reached.

2026-03-03 10:22:05.030 UTC [25] LOG: redo done at 0/87FFFA90 system usage: CPU: user: 0.16 s, system: 0.28 s, elapsed: 7.94 s

2026-03-03 10:22:05.076 UTC [25] FATAL: could not write to file "pg_wal/xlogtemp.25": No space left on device

2026-03-03 10:22:05.083 UTC [2] LOG: startup process (PID 25) exited with exit code 1

2026-03-03 10:22:05.083 UTC [2] LOG: terminating any other active server processes

2026-03-03 10:22:05.083 UTC [2] LOG: shutting down due to startup process failure

2026-03-03 10:22:05.101 UTC [2] LOG: database system is shut down

Solved

7 Replies

Status changed to Awaiting Railway Response Railway 3 months ago


gadatos
PRO

3 months ago

i have same. shitty hosting


You need to click on the volume mounted to your Postgres service, go to the settings tab, and increase the capacity of the volume (resize).

Make sure to redeploy your database after doing this.


sam-a
EMPLOYEE

3 months ago

We've identified the issue. You resized your volume from 500 MB to 1 GB, but the underlying filesystem wasn't expanded to match. That's why Postgres is running out of space even though the volume shows 1 GB.

We can fix this on our end by expanding the filesystem directly. We'll need to briefly remove the current deployment (it's already crash-looping), resize the filesystem, and redeploy. Your data is safe on the volume.

I'll confirm once fixed.


sam-a
EMPLOYEE

3 months ago

All set now - let me know if you still have an issue.


gadatos

i have same. shitty hosting

sam-a
EMPLOYEE

3 months ago

If you were unable to resize your volume, please open a separate thread so we can investigate and see if your issues is the same.


Status changed to Solved sam-a 3 months ago


peterjkim83
HOBBYOP

2 months ago

it was resolved for a bit and now it's repeating the same issue, the volume has been resized to 5GB, there's no way the database could have possibly grown that much in a matter of two weeks...


Status changed to Awaiting Railway Response Railway 2 months ago


2 months ago

Your volume was resized to 5 GB, but the additional space wasn't fully applied. We've corrected this on our end and redeployed your Postgres service. You should now have ~4.7 GB of available disk space.

We've also shipped a fix to prevent this from happening on future volume resizes.


Status changed to Awaiting User Response Railway 2 months ago


Status changed to Solved brody 2 months ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...