Postgres crash-looping: PANIC/FATAL "No space left on device" writing pg_wal (Railway volume postgres-volume)
alejoras
PROOP

25 days ago

Summary

- Railway Postgres service is crash-looping and app deployment fails because DB is unreachable.

- Postgres logs show repeated recovery attempts followed by FATAL/PANIC due to disk full while writing WAL.

What I’m seeing

- Postgres startup:

- "database system was interrupted while in recovery"

- "automatic recovery in progress"

- redo completes, then:

- FATAL: could not write to file "pg_wal/xlogtemp.<pid>": No space left on device

- Postgres shuts down and repeats.

- This started after heavy app activity; project metrics show disk usage around ~494 MB for postgres-volume.

Relevant log excerpts

- 2026-02-09 02:17:26 UTC FATAL: could not write to file "pg_wal/xlogtemp.29": No space left on device

- Earlier also: PANIC: could not write to file "pg_wal/xlogtemp.4544": No space left on device

- Also saw errors writing tmp files: base/pgsql_tmp/... No space left on device

Impact

- Postgres never reaches “accepting connections”.

- App gudink-production) fails with ECONNREFUSED / failed to create database pool.

What I need help with

- How to increase/resize the Postgres disk/volume for this service (volume is named postgres-volume) OR the fastest recovery path (restore backup to a new Postgres instance with larger storage).

- If volume resize is not possible in my plan, confirm the correct migration/recovery steps.

Project details

- App repo: https://github.com/alejoras/gudink-app

- App already has an emergency kill switch to reduce load (DISABLE_BACKGROUND_JOBS=true) and won’t crash-loop when Postgres is down, but Postgres itself cannot boot until disk space is available.

Solved$10 Bounty

Pinned Solution

Click on the volume mounted to your Postgres service, go to Settings, and increase the volume size. Keep in mind that the max volume capacity for Pro users is 1TB.

Deploy Postgres after the resize.

1 Replies

Click on the volume mounted to your Postgres service, go to Settings, and increase the volume size. Keep in mind that the max volume capacity for Pro users is 1TB.

Deploy Postgres after the resize.


Status changed to Solved brody 25 days ago


Loading...