20 days ago
Project: BOA-OpenClaw-Orchestrator (b4c783c0-497d-4eba-9c60-ca3d617e74db)
Environment: production (d9c1df35-176c-45ce-a235-26148d58364c)
Service: openclaw-gateway (b8f27bcd-86a5-4186-ac21-aff7030402d7)
Volume ID: d97bf160-afad-4c9b-b903-1573945e949c
Mount Point: /data
ISSUE SUMMARY:
Persistent volume resize from 5 GB to 10 GB was initiated but never applied to storage. The service is crashing in production.
CURRENT STATE:
- Railway API reports: 5000 MB (unchanged)
- Actual filesystem: 4.6 GB / 5 GB used (100% FULL)
- Intended size: 10 GB (Pro plan allows up to 1000 GB)
- Latest deployment (a17f8115-f85a-4823-9d4c-edaba2356cf4, 2026-09-16 11:49Z) FAILED with error: "Gateway failed to start: database or disk is full"
EVIDENCE:
- Commit 866b9d5 message states: "Railway volume headroom confirmed increased by owner"
- Deployment proceeded expecting larger volume capacity
- Container startup failed immediately with disk-full error
- No resize error/rejection returned; resize simply never propagated
- Service config shows no staged/pending changes
- Pro plan supports resize to 10 GB (well within 1000 GB limit)
ROOT CAUSE:
Railway control-plane or storage backend did NOT apply the resize request. The resize was either:
- Never received by control-plane
- Queued but stuck/in-progress
- Failed silently without returning an error
- Succeeded in control-plane but never propagated to API or storage backend
RAILWAY MUST VERIFY:
- Control-plane state: Does control-plane show 5000 MB or 10000 MB for this volume?
- Storage backend: What is the actual block device size on the host?
- Resize operation log: What is the status of the resize-to-10GB request?
- If control-plane ≠ API: There is a propagation layer issue
REQUIRED FIX:
- Apply the resize to 10000 MB on the existing volume (no data loss)
- Verify block device is resized on storage host
- Propagate update to Railway API so service config reports 10000 MB
- Restart openclaw-gateway so container sees new capacity
CRITICAL CONSTRAINT:
Do NOT delete, replace, or reset the volume. All OpenClaw state data in /data must be preserved.
SIMILAR RESOLVED CASES:
- postgres-crash-loop-volume-resize-not-3d7bd89d (April 2026): PostgreSQL 500MB→1GB resize failed, Railway agent sam-a fixed via control-plane
- volume-resize-500mb-5gb-not-applied-6eb51c0f (August 2026): Similar resize propagation failure, fixed by Railway infrastructure team
This is a known Railway infrastructure issue with volume resize propagation.
1 Replies
20 days ago
The volume is currently 5000 MB with 4999 MB used, which is causing the startup failure. Your Pro plan supports volumes up to 1 TB, so you can resize it to 10 GB directly from the volume settings in the dashboard using Live Resize. Since the disk is at capacity, the resize will restart the service briefly, but all data is preserved. After the resize completes, redeploy.
Status changed to Awaiting User Response Railway • 20 days ago
13 days ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • 13 days ago