17 days ago
Hi Railway team,
My service kite-seo-dashboard has failed 4 consecutive deploys today with the same error, while all other services in this same PRO workspace deploy successfully. Looks like this service is stuck on a builder whose disk is full.
Workspace + Service
- Workspace: barry6's Projects (PRO plan, upgraded ~30 min ago from Hobby)
-
- Project: kite-seo-dashboard (794e938a-2977-49c5-807d-6c241b383e57)
-
- Service ID: 7726a70e-df34-4f07-96eb-14f9cd3b0fe4
-
- Environment: production (0ec6adfa-64c1-42a7-bbde-28b4f1af90ab)
Builder pinning
All 4 failed builds today were scheduled on the same builder:
production-builderv3-us-west1-7lx0
Even after upgrading to PRO and waiting between retries.
Error (identical across all 4 builds)
Build Failed: build daemon returned an error < failed to solve: ResourceExhausted: failed to read dockerfile: failed to prepare ... failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/buildkit/runc-overlayfs/snapshots/snapshots/new-... : no space left on device >
Failed deploy IDs (today, all on us-west1-7lx0)
- 5633c853-ccb4-499f-9f87-04b14bbb91f1
-
- 32e34d8c-091a-40a2-b428-ddf00a3f8236
-
- 268e86f5-e9f8-4384-b0bd-a17b5fee17f5
-
- 7ab988b4-c6cc-46de-8e04-0cdb4ff6b71a
Evidence this is NOT a global incident
In the same workspace, the following services deploy fine within minutes of my failed builds:
- whoop-report -> 00139f2a SUCCESS @ 2026-05-20 16:40 UTC
-
- kite-support-hub -> fd15f1fe SUCCESS @ 2026-05-20 16:44 UTC, 86816e44 INITIALIZING @ 16:53
-
- 17 other services in this workspace last-deploy=SUCCESS
Only kite-seo-dashboard is failing, and it always lands on us-west1-7lx0.
Last successful deploy
bc0c5baf @ 2026-05-19 22:45 UTC (v0.12.1). Every deploy attempted since then has failed on this builder.
Request
Please reassign kite-seo-dashboard to a healthy builder (or clear disk on us-west1-7lx0). I've already paid an unpaid invoice ($14.62) and upgraded to PRO to get this unstuck, but the builder pinning seems independent of those changes.
Thanks!
2 Replies
Status changed to Awaiting Railway Response Railway • 17 days ago
17 days ago
Apologies for this canned message but in an effort to help all our customers get back up and running, we are sending this bulk message. As you may know, we had a major interruption to our services yesterday. We've published a post-mortem if you'd like more information on the incident. It describes what happened and what we are doing to prevent it in the future. We are deeply sorry for the impact that it has had on you.
It is taking some time to bring everything back up, but we are working on it as fast as we can. In general, a redeployment should fix most service issues. Due to the volume of customers redeploying right now, builds and deploys may take longer than normal to process.
You can track recovery status here: https://status.railway.com/incident/KVZ1Z8GY
If you are still having other issues that might be related to the incident you can read more here: https://station.railway.com/community/road-to-recovery-post-gcp-outage-builds-d362e48c
Feel free to respond if your question has not been addressed.
Status changed to Awaiting User Response Railway • 17 days ago
17 days ago
Apologies for this canned message but in an effort to help all our customers get back up and running, we are sending this bulk message. As you may know, we had a major interruption to our services yesterday. We've published a post-mortem if you'd like more information on the incident. It describes what happened and what we are doing to prevent it in the future. We are deeply sorry for the impact that it has had on you.
It is taking some time to bring everything back up, but we are working on it as fast as we can. In general, a redeployment should fix most service issues. Due to the volume of customers redeploying right now, builds and deploys may take longer than normal to process.
You can track recovery status here: https://status.railway.com/incident/KVZ1Z8GY
If you are still having other issues that might be related to the incident you can read more here: https://station.railway.com/community/road-to-recovery-post-gcp-outage-builds-d362e48c
Feel free to respond if your question has not been addressed.
9 days ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • 9 days ago