a month ago
Hi Railway team,
Since 2026-09-03 ~10:50 UTC, every deployment in our workspace "arthurveisseire's Projects" (workspace id 2bfa6aae-2a08-4bb1-a1e2-84cc74315599) has been sitting in QUEUED with queuedReason "Waiting for build slot". Nothing has entered BUILDING for more than 2 hours, in both of our projects (project "courteous-communication", id a427f141-2af5-4f53-9acd-84cf3aba13ba, environment "production" 986d4c9c-fea0-48e2-80db-be7aecc30099, and our other project as well).
What happened before: around 10:40-10:50 UTC we merged ~16 pull requests in a few minutes, which triggered roughly 150 GitHub deployments across the 11 services linked to the repo. We cancelled the superseded queued deployments through the API (deploymentCancel, ~97 of them), keeping one per service. Since then the build queue has not moved at all.
What we tried:
- Upgraded the workspace to Pro at ~12:50 UTC. The two deployments recreated afterwards (meta.plan = "pro") are still "Waiting for build slot".
-
- deploymentRedeploy on the queued deployments -> "Cannot redeploy without a snapshot".
-
- No incident on status.railway.com.
Current queued deployment ids (all QUEUED, "Waiting for build slot"):
- concerts.paris (backend): 2fc9ff7b-fbc3-4ee2-a4f2-467aa103988e (created 12:52 UTC, plan pro)
-
- celebrated-mercy (frontend): 5a6c13bc-080e-4f07-ae3b-12ab6f12b3fc (12:52 UTC, plan pro)
-
- kpi-dashboard: 28403e5f-9852-403e-b4b9-8b1318083c82 (11:10 UTC)
-
- import-runner: 5fdfef51-6a2b-431f-a848-7d190f7ef2b6 (11:10 UTC)
-
- backend-staging: 9368dc90-1157-430e-b28e-52119b1cab12 / frontend-staging: eacdb0f8-fcb4-4398-9884-f21f6ea0dc66
-
- revenue-cron cf5925e5-f8fd-43e7-ba96-d64647d90070 / newsletter-cron 2eea2bfc-a8bd-4eb6-be50-a88c5e14bcaa / fnac-availability-refresh 92c960ee-e3bf-4bd9-8a23-956533acd1eb / sale-watch-cron f6399abc-1061-4c6d-8de9-499a32ec3902
This is our production website (concerts.paris) and we have a merged release waiting to go out. Could you please unblok the build queue for this workspace (or lift whatever rate limit / stuck lock the burst of deployments triggered)? Happy to provide anything else you need.
Thanks a lot,
Arthur
2 Replies
Status changed to Awaiting Railway Response Railway • about 1 month ago
a month ago
Update 13:40 UTC. The queue partially drained at 13:22 UTC: every deployment created before the Pro upgrade (meta.plan = "hobby", created 11:10 UTC: kpi-dashboard, import-runner, backend-staging, frontend-staging and the 4 cron services) built and deployed within about 10 minutes. The deployments created after the upgrade (meta.plan = "pro") never start, even with an empty queue: 2fc9ff7b and 5a6c13bc stayed QUEUED, so I cancelled them and re-triggered both at 13:34 UTC. The new ones are still "Waiting for build slot" with nothing else building:
- concerts.paris (backend): aed892c8-cef6-460d-9ca2-c2eb8c9b0bdc
- celebrated-mercy (frontend): b8409140-cc3f-4090-a0ca-07817904d4dd
Both are Railpack builds, build environment V3, skipBuildCache true. So this looks specific to builds tagged plan=pro for this workspace rather than a general queue problem. Could you check the Pro builder assignment for workspace 2bfa6aae-2a08-4bb1-a1e2-84cc74315599? Those two services are the production site.
a month ago
Resolved. Both builds started at 13:44 UTC (about 10 minutes after my previous update) and deployed fine: concerts.paris backend up at 13:46 UTC, celebrated-mercy frontend built 13:45 UTC, both in the new EU West region. Nothing changed on our side in between, so it was either fixed on your end or the Pro build pool caught up on its own. If you did find a cause for the ~3 hours of "Waiting for build slot" after the plan upgrade, I would still be glad to know, but the thread can be closed. Thanks!
Status changed to Solved arthurveisseire • about 1 month ago