Staged variable changes report success but are silently skipped when CI checks fail
russhaskett
HOBBYOP

14 days ago

Project: college-assistant (7066080d-6630-4160-83b7-c56d4dd4f21e), service college-assistant (af6689ea-b076-4269-a4dd-2f36a31c9bec), environment: development. Hobby plan.

Symptom (about two weeks): after editing environment variables, clicking the staged-changes "Deploy" pill reports success, but no new deployment starts and the old deployment keeps serving. A manual Redeploy is required every time for the change to actually apply.

Today we found the cause of the sister symptom on git pushes: our GitHub check suite was failing, and push-triggered deployments were correctly created and marked SKIPPED with "CI check suite failed". That part is working as designed and was our own CI's fault.

The issue I am reporting is the staged-changes flow: it appears to be gated by the same wait-for-CI logic (or otherwise silently no-ops), but the UI reports the changes as applied, with no skip indicator and no reason shown. From the dashboard, a variable change looks live while the service never restarted. That cost us days of debugging stale behavior.

Requests:

  1. When a staged-changes deploy is skipped, surface it in the UI instead of reporting success - ideally with the skip reason and a "deploy anyway" option, like Deploy commit already offers on a skipped push deployment.
  2. Confirm whether staged variable changes are expected to be gated by wait-for-CI. If so, a note in the staged-changes panel would save others this hunt.

Happy to provide specific deployment IDs and timestamps on request.

Awaiting Conductor Response$10 Bounty

2 Replies

Railway
BOT

14 days ago

This thread has been opened as a bounty so the community can help solve it.

Status changed to Open Railway 14 days ago


dat007
PRO

14 days ago

The Root Cause

When "Wait for CI" is enabled in your service settings, Railway will hold any deployment—whether triggered by a git push or by applying staged changes (like environment variable edits)—until the associated GitHub CI check suite passes.

Here's the sequence that's happening in your case:

You edit an environment variable, creating staged changes.

You click the "Deploy" button in the staged changes panel.

Railway attempts to create a deployment but sees the "Wait for CI" gate and holds it pending the CI check suite.

Critically: The UI incorrectly reports success, with no indication that the deployment is waiting or has been skipped.

This is why your old deployment keeps serving—the new deployment was never actually created due to the CI gate. The manual "Redeploy" bypasses the "Wait for CI" check, which is why it works.

Request #1: Surface Skipped Status in UI

This is a significant UX bug. The UI currently treats a staged-changes deploy as successful even when it's blocked by the "Wait for CI" gate, with no skip indicator or reason shown. The "Show Skipped" filter in the Deployments tab only surfaces skipped git-push deployments, not staged-changes deployments.

Recommendation: The staged-changes panel should either:

Show a clear "Waiting for CI" status with the skip reason displayed.

Offer a "Deploy Anyway" button (similar to what's available for skipped push deployments) that bypasses the CI gate.

Request #2: Confirm Whether Staged Changes Are Gated by "Wait for CI"

Yes, they are. When "Wait for CI" is enabled, all deployments—whether triggered by git pushes or by applying staged changes (environment variables, service settings, etc.)—are gated by the CI check suite. Railway's staged changes workflow acts as a buffer for all configuration updates. Changes to environment variables create staged changes that must be reviewed and deployed, and if "Wait for CI" is on, that deploy will wait for CI to pass.

Recommendation: A note in the staged-changes panel explaining that "Wait for CI" is enabled and will gate this deployment would save other users from this confusion. The official documentation already states that "When 'Wait for CI' is enabled, the deployment is skipped if your GitHub Actions workflow fails (or doesn't run on push for that branch)", but this isn't surfaced in the staged-changes UI.

Workarounds Until This Is Fixed

Disable "Wait for CI" for services where you need immediate environment variable updates. You can find this toggle in your service settings.

Use "Redeploy Latest Commit" (CMD+K → "Deploy Latest Commit") which bypasses the CI gate.

If the staged changes are stuck indefinitely, try a hard refresh (Ctrl/Cmd+Shift+R) of the page to clear any cached UI state.

From the CLI, you can use railway redeploy --from-source --yes as a reliable workaround to force a deployment.

Recommendations for Railway Team

Staged-changes deploy flow should surface skip status—display a clear indicator when the deploy is held by "Wait for CI," with the reason shown.

Add a "Deploy Anyway" button to staged-changes panel, similar to what's available for skipped push deployments.

Add a note in the staged-changes panel explaining that "Wait for CI" is enabled and will gate the deployment.

Fix the UI bug where clicking "Deploy" in staged changes reports success even when the deployment is actually skipped or held.

If you'd like, I can help you escalate this to the Railway team with specific deployment IDs and timestamps.


dat007

The Root Cause When "Wait for CI" is enabled in your service settings, Railway will hold any deployment—whether triggered by a git push or by applying staged changes (like environment variable edits)—until the associated GitHub CI check suite passes. Here's the sequence that's happening in your case: You edit an environment variable, creating staged changes. You click the "Deploy" button in the staged changes panel. Railway attempts to create a deployment but sees the "Wait for CI" gate and holds it pending the CI check suite. Critically: The UI incorrectly reports success, with no indication that the deployment is waiting or has been skipped. This is why your old deployment keeps serving—the new deployment was never actually created due to the CI gate. The manual "Redeploy" bypasses the "Wait for CI" check, which is why it works. Request #1: Surface Skipped Status in UI This is a significant UX bug. The UI currently treats a staged-changes deploy as successful even when it's blocked by the "Wait for CI" gate, with no skip indicator or reason shown. The "Show Skipped" filter in the Deployments tab only surfaces skipped git-push deployments, not staged-changes deployments. Recommendation: The staged-changes panel should either: Show a clear "Waiting for CI" status with the skip reason displayed. Offer a "Deploy Anyway" button (similar to what's available for skipped push deployments) that bypasses the CI gate. Request #2: Confirm Whether Staged Changes Are Gated by "Wait for CI" Yes, they are. When "Wait for CI" is enabled, all deployments—whether triggered by git pushes or by applying staged changes (environment variables, service settings, etc.)—are gated by the CI check suite. Railway's staged changes workflow acts as a buffer for all configuration updates. Changes to environment variables create staged changes that must be reviewed and deployed, and if "Wait for CI" is on, that deploy will wait for CI to pass. Recommendation: A note in the staged-changes panel explaining that "Wait for CI" is enabled and will gate this deployment would save other users from this confusion. The official documentation already states that "When 'Wait for CI' is enabled, the deployment is skipped if your GitHub Actions workflow fails (or doesn't run on push for that branch)", but this isn't surfaced in the staged-changes UI. Workarounds Until This Is Fixed Disable "Wait for CI" for services where you need immediate environment variable updates. You can find this toggle in your service settings. Use "Redeploy Latest Commit" (CMD+K → "Deploy Latest Commit") which bypasses the CI gate. If the staged changes are stuck indefinitely, try a hard refresh (Ctrl/Cmd+Shift+R) of the page to clear any cached UI state. From the CLI, you can use railway redeploy --from-source --yes as a reliable workaround to force a deployment. Recommendations for Railway Team Staged-changes deploy flow should surface skip status—display a clear indicator when the deploy is held by "Wait for CI," with the reason shown. Add a "Deploy Anyway" button to staged-changes panel, similar to what's available for skipped push deployments. Add a note in the staged-changes panel explaining that "Wait for CI" is enabled and will gate the deployment. Fix the UI bug where clicking "Deploy" in staged changes reports success even when the deployment is actually skipped or held. If you'd like, I can help you escalate this to the Railway team with specific deployment IDs and timestamps.

russhaskett
HOBBYOP

14 days ago

Thanks dat007! I appreciate the help. -r


Welcome!

Sign in to your Railway account to join the conversation.

Loading...