a month ago
Hello Railway team,
We need to safely scale a staging Worker from one replica to zero and back using Railway CLI 5.63.1.
Our environment already has five unrelated Backend SMTP variables saved as STAGED changes. Those changes must remain staged and unapplied throughout Worker maintenance.
We reviewed the CLI's railway scale implementation, including its explicit environmentPatchCommit(environmentId, patch, commitMessage) call, but could not establish the server-side interaction with existing pending changes.
Could Railway staff confirm:
- Does
environmentPatchCommitwith an explicit Worker-only replica patch preserve an existing unrelated staged changeset, without applying, merging, overwriting or discarding it? - If that isolation is not guaranteed, what supported operation can scale only the Worker to zero while preserving those pending Backend changes?
- For scale-down from one replica to zero, is a configured
drainingSeconds=60honored by the terminating Worker, and how can we verify it is effective before scaling?
We need a documented or Railway-supported procedure rather than using a live mutation to test these assumptions.
No changes are authorized by this support request. Please do not modify the environment, apply or discard staged variables, or trigger a deployment.
We can provide project, environment, service and patch identifiers privately if necessary.
Thank you.
2 Replies
a month ago
This thread has been opened as a bounty so the community can help solve it.
Status changed to Open Railway • about 1 month ago
a month ago
Hi,
I reviewed the Railway CLI scaling implementation and the relevant documentation.
- Worker-only scaling and staged-change isolation
The CLI constructs a service-scoped multiRegionConfig patch using build_multi_region_patch(service_id, region_data) and submits it through environmentPatchCommit(environmentId, patch, commitMessage).
The source also contains a unit test confirming that the generated patch targets only the selected service.
Source:
https://github.com/railwayapp/cli/blob/master/src/commands/scale.rs
Railway's CLI documentation confirms that railway scale commits a single environment patch and applies the replica changes without a separate redeploy.
https://docs.railway.com/cli/scale
However, the documentation does not explicitly guarantee that an existing, unrelated staged changeset remains unchanged when environmentPatchCommit receives an explicit patch.
Therefore, I would not recommend performing the scale operation until Railway confirms this behavior.
- Supported alternative
The dashboard also supports changing replicas through staged changes, but the normal Deploy action applies all pending changes.
Since the five Backend SMTP changes must remain staged, the dashboard's standard Deploy action is not a safe substitute.
A Railway-supported service-specific operation that guarantees staged-change isolation is needed. If no such operation exists, Railway should confirm that limitation.
- Draining behavior
Railway documents RAILWAY_DEPLOYMENT_DRAINING_SECONDS=60 as a configurable SIGTERM-to-SIGKILL grace period for deployment teardown.
The scaling documentation also states that removed replicas are drained gracefully.
However, the published documentation does not explicitly guarantee that this configured grace period applies unchanged when scaling the final Worker replica from one to zero.
The application's SIGTERM handler should stop accepting new work and allow active jobs to finish. A 60-second grace period is an upper limit for graceful shutdown, not a guarantee that all jobs will complete.
- Non-mutating verification
Before maintenance, the operator can inspect the configured draining period, deployed Worker configuration, and current staged-change diff without applying changes.
For authoritative validation, Railway staff should confirm:
- Whether explicit
environmentPatchCommitpreserves unrelated staged changes. - Whether there is a supported Worker-only scaling operation with that guarantee.
- Whether scale-to-zero honors the configured draining interval.
- Which deployment events or logs can confirm the grace period used by the terminated replica.
Given the requirement to preserve the five pending SMTP changes, a confirmed server-side contract is necessary before recommending a production-safe procedure.
24 days ago
Answering the three questions as far as the public source and docs allow.
-
Preservation of the staged changeset: I couldn't confirm this from the source. The CLI's
railway scalebuilds a service-scopedmultiRegionConfigpatch and commits it withenvironmentPatchCommit, whose schema description is "Commit the provided patch to the environment." The CLI's managed-database commands useenvironmentPatchCommitStagedinstead, and warn that pre-existing staged changes will be applied with them (template_apply.rs). That suggests the two paths are separate, but the server-side behaviour is not documented. -
Supported alternative: I didn't find a documented Worker-only operation that leaves other staged changes alone.
railway scalesays the change "is applied without a separate redeploy step." It doesn't mention staged changes. -
Draining on scale-to-zero: the teardown docs describe
RAILWAY_DEPLOYMENT_DRAINING_SECONDSas the SIGTERM-to-SIGKILL window for a new deployment replacing the old one. They don't say whether it applies when replicas are removed, and the scale docs say that setting a region to0"removes all replicas from that region" without mentioning draining.
Staff should confirm points 1 and 3, and say whether a Worker-only scale that keeps unrelated staged changes exists.