22 days ago
Hi Railway team,
We deploy a public GHCR image, ghcr.io/oldbonestherapy/prometheus:stable, with image auto updates on and the maintenance window set to "As soon as ready". When we release, we move the stable tag to a new digest. The service doesn't pick it up.
The service:
- Project: olympus-acme-old-af
- Service: olympus
- Environment: production
What happened:
- At 14:21 UTC on Oct 29 we ran "Redeploy source image" on the service, so its active deployment came from :stable.
- At 18:15 UTC we moved :stable to a new digest. GHCR shows the tag updated.
- As of 19:40 UTC, about an hour and a half later, the service is still on the old digest. Settings, Source shows no update available either.
Earlier: on Oct 29 at 01:20 UTC we moved :stable once before. Six services had auto updates on: this one "As soon as ready", five with a custom 08:00–10:00 UTC window. None updated over the next 13 hours. Their active deployments had been created from :canary before we switched Settings, Source to :stable, which may be relevant.
Questions:
- How often does Railway actually check a mutable tag for a new digest with "As soon as ready"? Your docs say detection is cached "up to a few hours".
- Does auto update compare the digest of the tag in Settings, Source, or of the image the active deployment was created from?
- Can we trigger the check ourselves, for example with a registry webhook or an API call, so a release reaches services within minutes?
Each service lives in our customers' own Railway accounts, so we can't redeploy them for them. Auto updates are how they get fixes, which makes this important for us.
Thanks,
Brandon Fields
Olympus by Old Bones
P.S. I'll update this thread with how long it takes.
2 Replies
22 days ago
With "As soon as ready", the update is applied once we detect the new digest, not when it is pushed. Detection results for a tag are cached for up to a few hours to avoid overloading registries, so a gap of about 1.5 hours after moving :stable is still inside that window. On this service, auto updates are on for :stable, the window covers every hour of every day, nothing is snoozed, and no registry credentials are needed for a public GHCR image. Our latest cached check for :stable has not picked up a newer digest yet, which matches the "no update available" you see in Settings, Source.
For non-semver tags like :stable, we watch for a new image pushed to the configured tag (same tag, different SHA) and redeploy when one appears. The tag set in Settings, Source is the one that is checked and kept.
The 13 hours with no update after the 01:20 UTC move on Oct 29 is longer than the caching window explains, including for the services with an 08:00-10:00 UTC window, so caching alone does not account for that. We can't yet say whether starting from a deployment built from :canary played a part.
We can't offer a registry webhook or API call to trigger the update check on demand right now. Detection runs on our schedule.
Status changed to Awaiting User Response Railway • 22 days ago
22 days ago
Update:
It worked!!! Our next push to :stable deployed on its own about 4 h 20 min later, with "As soon as ready" on.
That's a bit longer than we expected, but it fits the "up to a few hours" detection caching. For anything urgent we'll use Redeploy source image (⌘K).
Thanks for the quick answer Railway bot ;)
Status changed to Awaiting Railway Response Railway • 22 days ago
Status changed to Solved Railway • 22 days ago