a month ago
Our Postgres service has near-unusable disk I/O since approximately
2026-08-29 00:05 UTC. Checkpoints that normally complete in 5-14 seconds
are now taking 250-750 seconds while writing the same tiny amount of data.
Individual fsync calls are taking up to 80 seconds.
Project: balanced-ambition (825f2804-8ddb-4620-9203-0588af900a76)
Environment: production
Service: Postgres (f1c5fd75-cc6b-4c97-9249-b98295d25b5e)
Volume: postgres-volume, 288MB of 1000MB used (not full)
Normal behaviour, before the issue:
2026-08-28 23:50:06 UTC checkpoint complete: wrote 48 buffers (0.3%);
write=4.759 s, sync=0.080 s, total=4.975 s; distance=268 kB
After it started:
2026-08-29 00:15:24 UTC checkpoint complete: wrote 154 buffers (0.9%);
write=238.864 s, sync=222.851 s, total=622.848 s; longest=40.430 s; distance=1034 kB
2026-08-29 00:43:48 UTC checkpoint complete: wrote 107 buffers (0.7%);
write=30.622 s, sync=47.920 s, total=242.462 s; longest=19.839 s; distance=569 kB
2026-08-29 00:57:17 UTC checkpoint complete: wrote 56 buffers (0.3%);
write=204.301 s, sync=391.017 s, total=750.633 s; longest=80.633 s; distance=489 kB
Nothing changed on our side: no deploy since the issue started, the write
volume per checkpoint is unchanged (~500 kB), and the volume has plenty of
free space. This looks like degraded I/O on the host backing the volume.
Impact: our app (App Service, same project) returns 504s because every
request that touches the database blocks. Public pages that hit the DB go
from 1s to 60s at random.
We noticed the ongoing incident 8GL2R2U5 ("Deployments slow to start",
network issue on an individual server) and wonder if this is related.
Could you check the host backing postgres-volume, or move it to healthy
hardware?
1 Replies
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
We've isolated the cause to our storage layer -- https://status.railway.com/incident/Z5Y3WO06. Apologies for the disruption!
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • about 1 month ago