Hi Alexandre,

You may want to look at Healthchecks in Railway to ensure the handoff between deployments. See the Docs:

• Healthchecks

• Configure Healthchecks and Restart Policy

Regards,
Christian

Hi christian,

I am already using healthchecks, but I think it is not enough for my use case. While it does ensure zero-downtime handoff, it does not prevent overlap between two service versions.

To prevent data consistency issues at the database level, my workflow involves downtime (which is OK for me) and looks like this:

• Service v1 is up with db schema v1

• Service v1 is brought down

• Database migrations are run, now db schema is at v2

• Service v2 comes up

If two service versions are running at the same time, there is a chance that service v1 issues queries to a db with schema v2, or that service v2 issues queries to a db with schema v1. I want to ensure strict separation so that a query with mismatched versions never occurs.

I've seen discussion about an OVERLAP_SECONDS setting that could be set to 0, but I don't think it gives strict enough guarantees due to the time the migrations take to run.

How can I achieve this goal?

Additional note: not sure if that changes something but my database is not hosted on Railway.

Thank you for the additional context.

It seems like OVERLAP_SECONDS = 0 might work for your use case. The default is 20 seconds. The variable is documented here.

How are you running your migrations? Are they run as part of the deployment steps for your service? Or via a separate service you deploy? If part of a separate service, you can use the Startup Order configuration to ensure your "Service v2" doesn't come up until your migration service has run.

By setting OVERLAP_SECONDS=0 is there a strong guarantee that there will be strictly no overlap between the two service deployments? If it's the case then it should indeed be the solution for me.

Thanks for pointing me to the Startup Order config, I didn't know about it and it may come to be useful. Currently I am running migrations as part of the service startup.

Yes, setting OVERLAP_SECODNS=0 will guarantee there will not be overlap between the services.

To ensure the migration steps run first, you can set your start command to something like:

[migration command] && [start command]

Thanks a lot for your help, I was able to implement what I needed.

I am now moving to a design with multiple services and am facing new issues regarding the same use case. I made the following design choices:

• The db migrations are moved to a separate, ephemeral service that just runs the migrations and exits.

• The services depending on the db check the db revision at startup and wait for it to be up to date before starting. This ensures that services v2 don't start emitting queries before the db schema is at v2.

The issue I have is about ensuring that services v1 are stopped before running the db migrations.

• Currently all services are linked to the same repo, and are built with the same Dockerfile. Is the build common to all services, or does it run separately for each one?

• This ensures that all services are redeployed, however, I can't control precisely when. By setting OVERLAP_SECONDS=0 they are stopped as soon as the build finishes and the new container starts. But the migration service could well start before the other services v1 stop emitting queries. How could I ensure all services are stopped before starting the new ones?

sorry maybe not fit for your usecase but I use maintenance mode middleware in my app

so I can disable certain features on demand when apply migration and block user request and query

Hi tonyhart7, thanks for your suggestion, thats seems like a very good idea that could work for my use case. How do you communicate with your services to turn on maintenance mode in your case?

My concern is that it would require substantial work. I was wondering if there were any Railway features that I could leverage to get it done easily?

My initial intent was to build Docker images externally and push them to a registry, use the Railway CLI to bring services down using railway down, then back up with railway redeploy. But it seems railway down doesn't work in a non-interactive environment with a RAILWAY_TOKEN. Is this intentional? How can I achieve this? Maybe through the Railway API?

Hi Alexandre,

For now, your best option might be to run your migration and app in the same service, and run them via a single command like [migration command] && [start command].

We are looking at implementing features in the future that will allow you to run pre-run and post-run commands, which would give you an extra set of controls like what you're after.

Regards,

Christian

well, I think it's depends in your tech stack (?)

i use rust webserver that read into redis when user hit certain critical endpoints and can disable that on demand via admin web
having that built in into railway is nice thought. soon maybe but for now, this is what I do