Skip to content
Cloudflare 1015 on service mutations through the Public API (Pro workspace)
dsgh
OP

an hour ago

Hi,

We're on the Pro plan. Since this afternoon, my attempted mutations on newly created services in one of our projects get a Cloudflare rate-limit error from the Public API, and it blocks them for about 17 hours.

Setup

  • For context, I've got a script that runs a one-off maintenance job on a short-lived service of its own (outside CI deploys, so a deploy never kills it). One run calls https://backboard.railway.com/graphql/v2 in this order:

    a. serviceCreate

    b. serviceDelete of its instance in review

    c. variableCollectionUpsert

    d. serviceInstanceUpdate (Docker image from GHCR with registry credentials, start command, region)

    e. serviceInstanceLimitsUpdate

    f. serviceInstanceDeployV2

    Later it calls deploymentRemove and eventually serviceDelete.

  • Today we created about seven services this way.

  • I also made a bunch of queries to logs, in quick succession.

What happens

  • Response: HTTP 429, body error code: 1015, a Retry-After header, and no X-RateLimit-* headers.
  • First time: the serviceCreate succeeded, then the serviceDelete of its review instance got 1015 with Retry-After: 64474. A serviceDelete of the same service in production a moment later succeeded.
  • Second time: serviceCreate, the review delete and variableCollectionUpsert all succeeded, then serviceInstanceUpdate got 1015 with a Retry-After until about 09:56 UTC the next day.
  • Both times we were nowhere near the API limit. A Workspace token's normal responses show ratelimit-policy: "default";q=10000;w=3600 with about 9,970 calls left. Reads keep working throughout.
  • Earlier runs using a railway login token also got 429s, but we didn't record whether those were 1015 or the hourly limit.
  • The same mutations sent with a nonexistent service id pass, get a normal GraphQL "Not Authorized" error, and no 1015.

Questions

  1. What rule is this (which operations it counts, the threshold and the window), and is it per user, token, workspace or project?
  2. Can it be lifted for this workspace, now and going forward?
  3. Is there a recommended pattern for short-lived job services, for example keeping a few services and redeploying them with a new image, that stays within it?
  4. Does the dashboard share this limit with the API?

Thanks!

Awaiting User Response

3 Replies

Railway
BOT

an hour ago

The documented Public API limits for Pro are 10,000 requests per hour and 50 requests per second, and the ratelimit-policy and X-RateLimit-* headers describe only those. A 429 carrying error code: 1015 without those headers is not that hourly allowance, which matches what you saw with plenty of calls remaining. The documented handling for any 429 is to back off and honor Retry-After rather than retrying immediately.

Redeploying an existing service with a new image is a supported flow: serviceInstanceUpdate can change a service instance's image and start command, and serviceInstanceDeployV2 deploys it, so a run doesn't need to create and delete a service each time.

The dashboard runs on the same GraphQL API as the Public API.


Status changed to Awaiting User Response Railway • about 1 hour ago


dsgh
OP

an hour ago

My issue is, it seems that your cloudflare limits are tighter than the API limits.


Status changed to Awaiting Railway Response Railway • about 1 hour ago


Railway
BOT

an hour ago

We don't have a published threshold, window, or scope for the error code: 1015 block. As you saw, it is separate from the hourly Public API allowance, which is why your remaining count and the ratelimit-policy header don't predict it.

Until it lifts, honor the Retry-After value rather than retrying the same mutations.

For the job pattern, you don't need a new service per run: serviceInstanceUpdate changes an existing service's start command, region and other instance settings, and serviceInstanceDeployV2 deploys it. That lets you keep a small set of job services instead of creating and deleting them each run.


Status changed to Awaiting User Response Railway • about 1 hour ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...