4 hours ago
I need help safely recovering a stuck environment apply for the selected telegram-bot service.
The environment is named staging, but it runs our production service. The pending changeset was created on 2026-11-02 at 19:12:30 UTC. At 19:57 UTC it was still APPLYING, with five pending changes: a service update and four non-secret bootstrap-code variable updates. No new deployment had been created. The most recent deployment remains the successful release from 15:02 UTC, with one running replica and no recent failed deployments in the status response.
I checked Railway's staged-changes documentation, Deployment Actions, slow-deployment troubleshooting and status page. The status page reported Fully Operational. The documented Abort deployment action appears to apply to an actual in-progress deployment; this operation is still an environment-patch commit, before a new deployment exists. I have not aborted, removed or restarted the healthy active deployment.
There is an important possible cause: our current release bootstrap passes compressed application code in 27 numbered environment-variable chunks plus two metadata entries. The local prepared payload totals approximately 676,830 UTF-8 bytes; the largest entry is 26,000 bytes. These are source-code payloads, not credentials. I found similar Central Station reports warning that bulk file/code payloads in variables can prevent an apply from finishing. I understand that code should be moved to a repository, image or file-based delivery instead; please confirm whether the variable payload is causing this operation to stall and advise on the safe migration/unblocking sequence.
Could you inspect the selected environment's commitChanges workflow and last apply error, safely unblock or cancel the pending apply if needed, and confirm which configuration writes, if any, have already landed? Please preserve the service, the running deployment, persistent volume/database, current saved configuration and unrelated variables. Please do not delete/recreate the service or detach/delete the volume. We can then replace the code-in-variables mechanism through a supported path.
No environment-variable values, source code, credentials, customer records or private application logs are included. Internal IDs can be supplied privately if the service selector does not provide enough context.
Relevant docs: https://docs.railway.com/deployments/troubleshooting/slow-deployments#when-to-contact-support
Similar reports: https://station.railway.com/questions/railway-deployment-stuck-on-applying-20-afd5ef90 and https://station.railway.com/questions/railway-function-source-accepted-but-not-8b82ce36
Thank you.
4 Replies
4 hours ago
Yes, the variable payload is what is stopping this. Your environment holds several sets of numbered chunks that carry encoded application code, and a start command that decodes and runs them. Storing code in environment variables is not a supported way to ship code on Railway. Once an environment holds that much encoded data, applying staged changes cannot finish.
The pending change still shows as applying, but the attempt to apply it has already stopped and is not running. The variable writes in that changeset are already in your saved configuration, so do not retry, restage, or re-submit it. Each attempt adds more data and makes the next apply fail sooner. Your current deployment is not affected by the stalled apply. The stuck status does not have to clear before you migrate.
To get the environment applying again, move the code out of variables and deliver it another way. You can deploy from a connected repo, build and deploy an image, keep large files on a volume, or fetch them at runtime from object storage. After that, set a short start command that runs the deployed files. Then remove every encoded chunk set and its metadata variables, and leave your regular configuration and secrets as they are.
Status changed to Awaiting User Response Railway • about 4 hours ago
Railway
Yes, the variable payload is what is stopping this. Your environment holds several sets of numbered chunks that carry encoded application code, and a start command that decodes and runs them. Storing code in environment variables is not a supported way to ship code on Railway. Once an environment holds that much encoded data, applying staged changes cannot finish. The pending change still shows as applying, but the attempt to apply it has already stopped and is not running. The variable writes in that changeset are already in your saved configuration, so do not retry, restage, or re-submit it. Each attempt adds more data and makes the next apply fail sooner. Your current deployment is not affected by the stalled apply. The stuck status does not have to clear before you migrate. To get the environment applying again, move the code out of variables and deliver it another way. You can deploy from a connected repo, build and deploy an image, keep large files on a volume, or fetch them at runtime from object storage. After that, set a short start command that runs the deployed files. Then remove every encoded chunk set and its metadata variables, and leave your regular configuration and secrets as they are.
4 hours ago
Thank you. I understand the code-in-variables pattern is unsupported and I will not retry, restage or resubmit the current apply. I have not changed or restarted the live service.
I still need platform-level clarification before the migration: can you confirm whether every write in the stalled changeset has finished, and whether the stale APPLYING workflow/marker can be safely cleared? Is the supported route to replace the start command and delete the encoded chunk sets in a single new configuration change after the code files/image are ready, despite the existing APPLYING marker?
Please have a Railway engineer verify the safe sequence for this environment, especially the risk of partially saved configuration being used by any restart. We are preparing a file/image-based code delivery, not asking Railway to rewrite the application. Please preserve the current live deployment, service, persistent volume/database and unrelated configuration/secrets. No cleanup or deletion has been attempted.
Status changed to Awaiting Railway Response Railway • about 4 hours ago
4 hours ago
The saved configuration for telegram-bot already has the stalled changeset's variable writes. The committed variables include CIRCE_LIVE_BOOT_25, CIRCE_LIVE_BOOT_26, CIRCE_LIVE_BOOT_PARTS and CIRCE_LIVE_BOOT_SHA256. The saved start command also already decodes the CIRCE_LIVE_BOOT_* set. So the change is not half-written on the configuration side. What never ran is the deployment step. Your running deployment is still the 15:02 UTC release, but any new deployment of this service would start from the saved configuration as it is now.
The change still shows as applying, but its apply process has ended and nothing is running behind it. Automatic reconciliation can later mark a stalled apply like this as failed. We can't give a time for when that will happen, and clearing that status doesn't fix the underlying size problem.
The environment holds three encoded chunk sets, APP_SOURCE_*, CIRCE_LIVE_BOOT_* and CIRCE_PATCH_*, each with its own parts and checksum variables. Taken together they are what keeps the environment from applying. Once your repo or image delivery is ready, the order is: switch the source and start command to the deployed files, and remove all three chunk sets and their metadata in that same change. Keep your regular configuration and secrets. Removing the encoded data is what lets the environment apply again. We can't promise how a change submitted while the old one still shows as applying will behave until that status has settled.
Status changed to Awaiting User Response Railway • about 4 hours ago
Railway
The saved configuration for telegram-bot already has the stalled changeset's variable writes. The committed variables include `CIRCE_LIVE_BOOT_25`, `CIRCE_LIVE_BOOT_26`, `CIRCE_LIVE_BOOT_PARTS` and `CIRCE_LIVE_BOOT_SHA256`. The saved start command also already decodes the `CIRCE_LIVE_BOOT_*` set. So the change is not half-written on the configuration side. What never ran is the deployment step. Your running deployment is still the 15:02 UTC release, but any new deployment of this service would start from the saved configuration as it is now. The change still shows as applying, but its apply process has ended and nothing is running behind it. Automatic reconciliation can later mark a stalled apply like this as failed. We can't give a time for when that will happen, and clearing that status doesn't fix the underlying size problem. The environment holds three encoded chunk sets, `APP_SOURCE_*`, `CIRCE_LIVE_BOOT_*` and `CIRCE_PATCH_*`, each with its own parts and checksum variables. Taken together they are what keeps the environment from applying. Once your repo or image delivery is ready, the order is: switch the source and start command to the deployed files, and remove all three chunk sets and their metadata in that same change. Keep your regular configuration and secrets. Removing the encoded data is what lets the environment apply again. We can't promise how a change submitted while the old one still shows as applying will behave until that status has settled.
2 hours ago
I still need help with the platform-side blocker. The latest automated answer says the apply process has ended, but also says it cannot confirm how a replacement change will behave until the APPLYING status settles. Please route this thread to a Railway engineer to verify or clear the stale apply marker and confirm the safe migration sequence. We will move code to supported file/image delivery and remove the encoded variables in the same change; we are not retrying the existing changeset. Please preserve the running deployment, service and persistent volume/database. No configuration cleanup or restart has been performed.
Status changed to Awaiting Railway Response Railway • about 2 hours ago