a month ago
Subject: Build failures due to builder disk space exhaustion
Hi Railway Support,
My service RunUp (Project ID: 76bc2482-a893-4727-b81c-ca2a3ddd99b2, Environment: production) has been experiencing consecutive build failures since 2026-05-20 15:45 UTC.
Issue: All deployments are failing instantly during the build phase with the error:
ResourceExhausted: failed to read dockerfile: failed to create lease: write /var/lib/buildkit/runc-overlayfs/containerdmeta.db: no space left on device
The builder node assigned to this service (production-builderv3-us-west1-417g) appears to be out of disk space. This has caused 8+ consecutive build failures, while the last successful deployment was at 12:24 UTC the same day.
Details:
Service: RunUp
Repository: nealoneal/RunUp (main branch)
Last successful deployment: 2026-05-20 12:24 UTC
First failure: 2026-05-20 15:45 UTC
Failure count: 8+ consecutive deployments
Builder: production-builderv3-us-west1-417g
The error occurs before any Dockerfile parsing or build steps can begin, indicating the issue is with the builder infrastructure, not the application code.
Request: Could you please either clean up the builder node's disk space or reassign this service to a healthy builder node so deployments can resume?
5 Replies
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
Apologies for this canned message but in an effort to help all our customers get back up and running, we are sending this bulk message. As you may know, we had a major interruption to our services yesterday. We've published a post-mortem if you'd like more information on the incident. It describes what happened and what we are doing to prevent it in the future. We are deeply sorry for the impact that it has had on you.
It is taking some time to bring everything back up, but we are working on it as fast as we can. In general, a redeployment should fix most service issues. Due to the volume of customers redeploying right now, builds and deploys may take longer than normal to process.
You can track recovery status here: https://status.railway.com/incident/KVZ1Z8GY
If you are still having other issues that might be related to the incident you can read more here: https://station.railway.com/community/road-to-recovery-post-gcp-outage-builds-d362e48c
Feel free to respond if your question has not been addressed.
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
I am also having this issue
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
same problem here, i get this error, any workarround?
a month ago
The "no space left on device" error here is fallout from yesterday's incident, where GCP brought our builders back up in an incorrect state. That's been fixed on our end, so canceling and retrying the failed builds should get you unblocked. Sorry for the disruption this caused.
Status changed to Awaiting User Response Railway • about 1 month ago
a month ago
Hey there,
We had a builder disk pressure issue across our V3 fleet (us-west1 region) following the post-outage recovery on May 20. The build queue is fully healthy again now; end-to-end deploys across the platform are completing in under a minute.
I just kicked a fresh redeploy on your service from my side. It should land on a clean builder.
If you're still hitting the same "no space left on device" error after this, reply with the latest deployment ID and I'll route it for direct attention.
Sorry for the disruption,
Angelo
Status changed to Solved Railway • 28 days ago