8 days ago
Description: Self-hosted CouchDB for Obsidian LiveSync
Category: Storage
1 Replies
2 hours ago
Recurring extreme COMMIT/fsync latency on Postgres (volume-backed) — up to 43s
Hi @feelnopain,
This is the exact same root cause that another Railway user hit and had resolved back in May 2026. Your symptoms — multi-second bare COMMIT stalls, synchronous_commit=on, no replica, no application-level lock contention — are identical to that case, which Railway traced to a degraded storage host under the Postgres volume.
Railway's own conclusion (from employee chandrika, May 2026):
"Your Postgres instance is on a storage host that's experiencing degraded I/O performance. We've staged a migration to a newer, healthier host. Trigger a redeploy and the volume will migrate — storage latency should return to normal."
Solved thread: https://station.railway.com/questions/severe-postgres-disk-i-o-latency-38s-c-22bb5414
Immediate fix (stop the bleeding now — 1 minute, no downtime)
Set synchronous_commit to off so COMMITs stop waiting on the slow disk:
ALTER SYSTEM SET synchronous_commit = off;
SELECT pg_reload_conf();What this does: WAL writes still happen, but COMMIT returns immediately before the fsync completes. In the event of a crash, you could lose the last transaction (typically <1 second of data). PostgreSQL itself is crash-safe — WAL integrity is preserved, only the timing of the flush changes. This is a widely-used setting in production. Flip it back to on once the storage host is migrated.
SHOW synchronous_commit;
-- should return: offPermanent fix (needs Railway support)
Ask Railway to migrate your Postgres volume to a healthy storage host. Include:
- Your project/service IDs
- The worsening trend data you documented (20s → 28.8s → 43s)
- A link to the solved thread: https://station.railway.com/questions/severe-postgres-disk-i-o-latency-38s-c-22bb5414
- The
railway logs --jsonexcerpts showing the stalled COMMITs
Railway's chandrika confirmed the process: they stage the migration, you trigger a redeploy, the volume migrates along with the service. A few minutes of downtime.
Temporary workaround (no support needed)
Until Railway migrates your volume, redeploying to a different region can land you on a healthier storage host. Railway's ray-chen recommended this in the May case. Note: database downtime while the volume syncs to the new region.
Proactive monitoring (catch it early next time)
Run these periodically to catch fsync stalls before users do:
-- Recent slow statements (>1s)
SELECT
query,
calls,
mean_exec_time::numeric(10,2) AS avg_ms,
max_exec_time::numeric(10,2) AS max_ms
FROM pg_stat_statements
WHERE max_exec_time > 1000
ORDER BY max_exec_time DESC
LIMIT 10;
-- Checkpoint stats (high sync_time = storage issue)
SELECT
checkpoints_timed,
checkpoints_req,
checkpoint_write_time::numeric(10,2) AS write_s,
checkpoint_sync_time::numeric(10,2) AS sync_s
FROM pg_stat_bgwriter;If checkpoint_sync_time spikes, or max_exec_time on trivial queries jumps to 10s+, your storage host is degrading again.
TL;DR
Right now: ALTER SYSTEM SET synchronous_commit = off; SELECT pg_reload_conf();
Then: Ask Railway to migrate your volume to a healthy host (same fix as May 2026).
While waiting: Redeploy to a different region as temporary relief.
Disclaimer: I don't work for Railway — just cross-referencing a previously resolved case with the same root cause.