2 months ago
Hey there 👋
We're migrating from Heroku and are trying to replicate their deployment behavior with Railway so that our system works as expected.
When a build is completed, they stop the old deployment first and only after it was stopped, they start the new deployment.
How can we achieve that with Railway?
That is necessary because we have an architecture where only a single instance must ran at any given time. Multiple instances will result in unexpected behavior, esp. around resources that are not supposed to be shared - like the database.
We're aware that this causes a brief downtime. That's a trade-off we are happy to make to maintain our simple architecture, for now.
Even with a teardown set to 0s overlap and 30s shutdown grace period, the new deployment starts while the old deployment is still in the process of shutting down. That means the old deployment may still write to the database while the new deployment is already reading from the database in parallel.
Thank you,
Marc
5 Replies
2 months ago
try setting the 30s grace period to 0 too maybe, unless your old deployment needs that time to flush
if having both settings set at 0 doesnt help, then you can try attaching a volume to the service, 2 deployments cannot be live at the same time if the service has a volume
-# Although I'm not sure how this works with draining seconds exactly, it's possible it'll wait for your app's grace shutdown period before starting the new deployment, but also maybe not, unsure
if your old deployment does need to finish processes before the new one can go live then I'm doubtful there's anything built into Railway to help there, but you can likely build custom functionality for this
my immediate thought is using Railway's GQL API to check for the service's deployments status to wait until its offline before starting your app's main process
Thanks!
We do need a grace period. The service needs to complete the processing of ongoing HTTP requests and complete the processing of items in the event queue that it already started processing.
I'll try the idea with an attached volume. Let's see what this does 🙏
2 months ago
Sounds good, hopefully it does respect draining seconds, fingers crossed
Stopping container > … Stopped container > Starting container > …
It seems to be working 😯
Great hack that saves a lot of time, thanks 🙏
(Fun fact: Silly me tried to mount the unused volume to /dev/null, which just froze the deployment when it tried to start the container 😬)
2 months ago
lmao, well i'm glad it worked out
Status changed to Solved dev • about 2 months ago