Skip to content
Are predeploys broken?

a year ago

I have deployments hanging exclusively on services with pre-deploy scripts, no error logs.

This particular service usually deploys in ~2minutes, but its been hanging for 20 minutes (after successfully running the pre-deploy)

Solved

48 Replies

a year ago

I temporarily removed the predeploy script and the service is now deploying correctly 👀


a year ago

What type of script are you running on pre-deploy? migrations?


a year ago

Mind sharing that service link?



a year ago

Yeah, migrations


a year ago

The migrations complete though, and then it just hangs

1444347371134648381


a year ago

(the "stopping container" only happened after I aborted the deploy)


a year ago

🤔


a year ago

Any recent changes to the pre-deploy command? Was it running fine before?


a year ago

No changes, I don't even have new migrations 😅


a year ago

Yeah been running fine for months


a year ago

!t


a year ago

This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response medim • 12 months ago


a year ago

I'm not seeing anything that is hanging, could I get a link to that?



a year ago

It sat for 20 minutes before I removed the predeploy command and redeployed



a year ago

For context, this is the script, so if the Done Migrations line is hit, there is nothing more

1444352951232499752


a year ago

It isn't a big deal for me, I'm just messing around on my staging for now, but I just wanted to raise it incase it is a broader issue 🙂


a year ago

Can you explicitly call os.Exit()?


Status changed to Awaiting User Response Railway • 12 months ago


a year ago

Oh


a year ago

That worked


a year ago

You made the change that fast?


a year ago

I did


a year ago


a year ago

28 second build too, nice


a year ago

Undoing the os.Exit to double check that was it


a year ago

Yeah it hangs again. That is new behaviour then, no? I haven't updated this script in a while


a year ago

I see two other reports, so something is up on our end.


a year ago

Ah alright, good luck! <:salute:1137099685417451530>

Thanks for the help as always <:bufo_pray:1331539894769684513>


a year ago

No problem!


efstajas

a year ago

sorry for the "same", but same

1444369025457983639


a year ago

As suggested previously, try adding an exit command at the end of your script!


efstajas

a year ago

i assume the previous behavior will be restored? would rather wait then, my predeploy cmd is currently just running drizzle-kit migrate. would like to avoid wrapping it in a custom script on 3 different services


efstajas

a year ago

drizzle-kit migrate definitely exits with 0 on success btw 🤔

1444371294312333456


a year ago

I haven't attempted to reproduce yet, and even so, it's unfortunately not something we will tackle over the weekend.


efstajas

a year ago

Could you maybe clarify what this new requirement is for predeploy commands? I understand in the other case adding an explicit exit made it work, but my command already exits with 0 (and always has), so I don't really see what else I can do at the moment to make it work.


efstajas

a year ago

Also, ofc understand it's a weekend, but if this new behavior is unintended and unclear, feels like this would warrant an incident? 🤔


maciejrrr

a year ago

Also having problems with deployments now. Everything is stuck.


Status changed to Awaiting Railway Response Railway • 12 months ago


maciejrrr

Also having problems with deployments now. Everything is stuck.

a year ago

Please open your own thread.


Status changed to Awaiting User Response Railway • 12 months ago


a year ago

It's not a new requirement, but I have yet to confirm if it is a regression. An incident won't be called, as incidents are for platform-wide issues that affect the majority of the platform.


efstajas

a year ago

Alrighty. Re it being a regression - I can say in our case no changes have been made to anything pre-deploy related since our last successful deploy for this same service on Friday at Nov 28, 2025, 3:58:37 PM GMT+1. Today since (at the latest) Nov 29, 2025, 5:26:24 PM GMT+1 our deploys are stuck.


a year ago

@json - Could I ask you to open your own thread? it makes it easier for us to track.


Railway
BOT

a year ago

Hello!

We're acknowledging your issue and attaching a ticket to this thread.

We don't have an ETA for it, but, our engineering team will take a look and you will be updated as we update the ticket.

Please reply to this thread if you have any questions!


Railway
BOT

a year ago

🛠️ The ticket Pre-deployment delays occurring has been marked as triage.


Railway
BOT

a year ago

🛠️ The ticket Deployment delays occurring has been marked as triage.


Railway
BOT

a year ago

🛠️ The ticket Deployment delays occurring has been marked as in progress.


a year ago

Update here: While we have not tackled the root cause, we have fixed it for the time being and will address the root cause this week. Your pre-deploy commands should now finish just as they always have.


Status changed to Solved medim • 12 months ago


Railway
BOT

a year ago

✅ The ticket Deployment delays occurring has been marked as completed.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...