18 days ago
Hello Railway Support,
We are experiencing intermittent PostgreSQL latency in production.
Project: Bober Travel
Environment: production
Region: europe-west4-drams3a
Services:
- Backend: 790333bb-3999-42bc-8ec7-628b6c537ee6
- PostgreSQL: c0c3ced9-ac84-4f07-9739-6aef32e13816
Affected request:
- Time window: 2026-06-19 12:07:35–12:08:34 UTC
- Method/path: POST /route
- Status: 201
- Railway request ID: vEq6_lXBTZKmYccv63j4Ww
- Backend deployment: 18b78625-e39d-46a5-8518-5d80ed7c7f6b
- Deployment instance: 11b3b600-bf9b-4617-a08b-9e87e6430a61
- Sentry trace ID: 4c0c27d79d7b49d5a5a30429535c3d05
Timing:
- Railway totalDuration: 59,152 ms
- Railway upstreamRqDuration: 59,148 ms
- Application duration: 59,122 ms
- Prisma/DB duration: 59,103 ms across 6 sequential queries
The same endpoint normally completes in 38–189 ms.
The services were not under meaningful CPU or memory load. PostgreSQL had 4 connections out of max_connections=100 when inspected, no recorded
deadlocks, and the volume was approximately 320 MB / 500 MB.
PostgreSQL logs also show unusually long checkpoint/fsync operations. For example:
2026-06-19 10:51:30.008 UTC
checkpoint complete: wrote 6 buffers; write=0.536 s, sync=110.643 s, total=114.366 s
Other checkpoints today took approximately 30–105 seconds despite writing only 5–22 buffers.
This appears consistent with intermittent volume/WAL fsync latency rather than application CPU load or connection-pool exhaustion. No checkpoint
was logged during the exact affected request, so we cannot prove that its specific wait was an fsync; however, 59.103 seconds were spent inside
the database client and the storage stalls are independently visible in PostgreSQL logs.
Could you please:
- Inspect storage/volume latency for the PostgreSQL service during 2026-06-19 12:07:35–12:08:34 UTC.
- Check whether the underlying host or volume experienced an I/O incident.
- Advise whether the database should be migrated/reprovisioned onto a new volume or host.
- Confirm whether similar latency is visible in Railway’s internal telemetry.
Thank you.
6 Replies
Status changed to Awaiting Railway Response Railway • 18 days ago
17 days ago
Hi Ivan, thanks for the unusually thorough report, it made this quick to confirm. We can independently see the storage stalls in our telemetry for your Postgres service: checkpoints are completing with sub-second write and sync phases but total durations of 30-105 seconds (one this morning took 104.7s total with only 0.22s of actual sync). That gap is time the database spends blocked at the volume/storage layer, not CPU, connections, or your application, so your diagnosis is correct. This is consistent with a degraded storage host rather than anything in your project. Our infrastructure team is locating the underlying host for your Postgres volume in europe-west4-drams3a to migrate it onto healthy storage, and we'll update this thread once that's underway. No action needed on your side, and a redeploy on its own won't reliably move you off the affected host.
Status changed to Awaiting User Response Railway • 17 days ago
Status changed to Awaiting Railway Response angelo-railway • 17 days ago
14 days ago
Quick status update: this is still being handled as a storage-host issue, not an application or connection-pool issue on your side.
Your report and the telemetry both point at the volume/storage layer, and a normal redeploy is not expected to reliably fix it. The pending action is still to identify the underlying host and move the database volume off degraded storage. We will update this thread once that work is complete or if we need anything from you.
Status changed to Awaiting User Response Railway • 14 days ago
angelo-railway
Quick status update: this is still being handled as a storage-host issue, not an application or connection-pool issue on your side. Your report and the telemetry both point at the volume/storage layer, and a normal redeploy is not expected to reliably fix it. The pending action is still to identify the underlying host and move the database volume off degraded storage. We will update this thread once that work is complete or if we need anything from you.
10 days ago
Hi! Thank you! Will wait for update
Status changed to Awaiting Railway Response Railway • 10 days ago
Status changed to Awaiting User Response Railway • 8 days ago
noahd
Hey! Are you still seeing this high latency?
8 days ago
Hi! Didn't noticed it for a few days, everything should be resolved?
Status changed to Awaiting Railway Response Railway • 8 days ago
8 days ago
It should be resolved yes! If you run into issues like this again can you please let me know? Thanks!
Status changed to Awaiting User Response Railway • 8 days ago
15 hours ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • about 15 hours ago