a month ago
We are creating new environments in Github Action via CLI by cloning existing staging environment:
railway environment new "$ENV_NAME" -d staging -v <redacted> SKIP=ME
The command errors out with
Failed to fetch: error sending request for url (https://backboard.railway.com/graphql/v2)
Caused by:
0: error sending request for url (https://backboard.railway.com/graphql/v2)
1: operation timed out
This has been happening on Wednesday, it was fine on Thursday but now it's timing out again (Friday).
10 Replies
Status changed to Awaiting Railway Response Railway • 26 days ago
a month ago
The timeout is on our side, not your CLI command or flags. Duplicating your staging environment is intermittently taking longer than the CLI's request window allows, so it reports a timeout reaching our API even though the duplication often finishes on our end a little later. That's also why a retry can come back with "an environment with that name already exists" - the first attempt actually created it despite the timeout. Before retrying, check whether the new environment was created; if it's there, the clone succeeded.
I'll forward this up to our CLI team to see if they can see why this is happening as well
Status changed to Awaiting User Response Railway • 26 days ago
a month ago
Yes, I can confirm that the environment is always created successfully but it's also always empty. This breaks our pipeline on 2 fronts
- it marks 1st run of pipeline as failed (not great, not terrible) but
- rerun of the pipeline also fails because the environment exists (we check and we don't create a new one) but without any services, meaning that any deployment will fail, regardless how many times we retry the pipeline.
Status changed to Awaiting Railway Response Railway • 26 days ago
a month ago
If the command doesn't time out, all services are duplicated successfully. If it does, they are not. The timeouts and failed duplications started happening on Wednesday, it worked before for many months.
a month ago
Temporary workaround is to manually sync empty environment from staging to create empty services, then re-run the pipeline.
a month ago
Hey! just wanted you to know I escalated this up to our team to look into.
We'll follow up when we have a resolution. Sorry about the impact in the meantime.
Status changed to Awaiting User Response Railway • 26 days ago
13 days ago
Just letting you know that today the failure mode was even worse. CLI command succeeded but created environment was empty.
Status changed to Awaiting Railway Response Railway • 13 days ago
13 days ago
This is a Railway API timeout error — your app tried to call Railway's internal GraphQL API and it timed out before getting a response.
Most common causes:
Railway backend is temporarily slow — check status.railway.com for any ongoing incidents
Your request has no timeout set — the call waits forever and eventually fails
Network issue between your service and Railway's API — especially common on private networking
Rate limiting — too many API calls in a short period
Fix:
Check status.railway.com first — if there's an incident, just wait it out
Add retry logic to your code:
jsconst response = await fetch(url, {
timeout: 10000, // 10 second timeout
retry: 3
})
If it's happening constantly — go to railway.com/help and report it with your project ID
13 days ago
Thanks for the update, and sorry this is still hitting your pipeline. This is a known issue on our side that our team is actively working on. Under load, our API can take longer than the CLI's request window to finish duplicating an environment, and in some cases the environment is created while the service duplication does not complete, leaving it empty. It is not caused by your command or flags.
Until this is fully resolved, the safest guard for your pipeline is to treat an environment that exists but has no services as a failed clone rather than a success, and to delete and recreate it (or run a sync from staging) before deploying, since deploying into the empty environment will always fail. We will follow up here as soon as we have a resolution.
Status changed to Awaiting User Response Railway • 13 days ago
6 days ago
This thread has been marked as solved automatically due to a lack of recent activity. Please re-open this thread or create a new one if you require further assistance. Thank you!
Status changed to Solved Railway • 6 days ago
5 days ago
@Bot, how would we get the update on this issue? I think the thread should stay opened.
Status changed to Awaiting Railway Response Railway • 5 days ago
4 days ago
Hey Matic — thanks for the nudge, and yes we'll keep this open until it's fully resolved. Quick status: this is the same known platform-side issue we flagged earlier. Under load our API can take longer than the CLI's request window to finish duplicating an environment, and in the worst case the environment gets created while the service duplication never completes, leaving it empty. It's not your command or flags, and the team is actively working the underlying control-plane performance problem. Until it's fully fixed, the safest pipeline guard is to treat an environment that exists but has zero services as a failed clone (not a success) — delete + recreate it (or sync from staging to populate services) before deploying. I'll post here as soon as we ship the fix. (One note: the earlier reply in this thread suggesting timeout/retry tweaks was from another community member, not staff — client-side retries won't help since the failure is server-side.)
Status changed to Awaiting User Response Railway • 4 days ago