Are predeploys broken?

10 months ago

I have deployments hanging exclusively on services with pre-deploy scripts, no error logs.

This particular service usually deploys in ~2minutes, but its been hanging for 20 minutes (after successfully running the pre-deploy)

Solved

48 Replies

10 months ago

I temporarily removed the predeploy script and the service is now deploying correctly 👀


10 months ago

What type of script are you running on pre-deploy? migrations?


10 months ago

Mind sharing that service link?



10 months ago

Yeah, migrations


10 months ago

The migrations complete though, and then it just hangs

1444347371134648381


10 months ago

(the "stopping container" only happened after I aborted the deploy)


10 months ago

🤔


10 months ago

Any recent changes to the pre-deploy command? Was it running fine before?


10 months ago

No changes, I don't even have new migrations 😅


10 months ago

Yeah been running fine for months


10 months ago

!t


10 months ago

This thread has been escalated to the Railway team.

Status changed to Awaiting Railway Response medim • 10 months ago


10 months ago

I'm not seeing anything that is hanging, could I get a link to that?



10 months ago

It sat for 20 minutes before I removed the predeploy command and redeployed



10 months ago

For context, this is the script, so if the Done Migrations line is hit, there is nothing more

1444352951232499752


10 months ago

It isn't a big deal for me, I'm just messing around on my staging for now, but I just wanted to raise it incase it is a broader issue 🙂


10 months ago

Can you explicitly call os.Exit()?


Status changed to Awaiting User Response Railway • 10 months ago


10 months ago

Oh


10 months ago

That worked


10 months ago

You made the change that fast?


10 months ago

I did


10 months ago


10 months ago

28 second build too, nice


10 months ago

Undoing the os.Exit to double check that was it


10 months ago

Yeah it hangs again. That is new behaviour then, no? I haven't updated this script in a while


10 months ago

I see two other reports, so something is up on our end.


10 months ago

Ah alright, good luck! <:salute:1137099685417451530>

Thanks for the help as always <:bufo_pray:1331539894769684513>


10 months ago

No problem!


efstajas
PRO

10 months ago

sorry for the "same", but same

1444369025457983639


10 months ago

As suggested previously, try adding an exit command at the end of your script!


efstajas
PRO

10 months ago

i assume the previous behavior will be restored? would rather wait then, my predeploy cmd is currently just running drizzle-kit migrate. would like to avoid wrapping it in a custom script on 3 different services


efstajas
PRO

10 months ago

drizzle-kit migrate definitely exits with 0 on success btw 🤔

1444371294312333456


10 months ago

I haven't attempted to reproduce yet, and even so, it's unfortunately not something we will tackle over the weekend.


efstajas
PRO

10 months ago

Could you maybe clarify what this new requirement is for predeploy commands? I understand in the other case adding an explicit exit made it work, but my command already exits with 0 (and always has), so I don't really see what else I can do at the moment to make it work.


efstajas
PRO

10 months ago

Also, ofc understand it's a weekend, but if this new behavior is unintended and unclear, feels like this would warrant an incident? 🤔


maciejrrr
HOBBY

10 months ago

Also having problems with deployments now. Everything is stuck.


Status changed to Awaiting Railway Response Railway • 10 months ago


maciejrrr

Also having problems with deployments now. Everything is stuck.

10 months ago

Please open your own thread.


Status changed to Awaiting User Response Railway • 10 months ago


10 months ago

It's not a new requirement, but I have yet to confirm if it is a regression. An incident won't be called, as incidents are for platform-wide issues that affect the majority of the platform.


efstajas
PRO

10 months ago

Alrighty. Re it being a regression - I can say in our case no changes have been made to anything pre-deploy related since our last successful deploy for this same service on Friday at Nov 28, 2025, 3:58:37 PM GMT+1. Today since (at the latest) Nov 29, 2025, 5:26:24 PM GMT+1 our deploys are stuck.


10 months ago

@json - Could I ask you to open your own thread? it makes it easier for us to track.


Railway
BOT

10 months ago

Hello!

We're acknowledging your issue and attaching a ticket to this thread.

We don't have an ETA for it, but, our engineering team will take a look and you will be updated as we update the ticket.

Please reply to this thread if you have any questions!


Railway
BOT

10 months ago

🛠️ The ticket Pre-deployment delays occurring has been marked as triage.


Railway
BOT

10 months ago

🛠️ The ticket Deployment delays occurring has been marked as triage.


Railway
BOT

10 months ago

🛠️ The ticket Deployment delays occurring has been marked as in progress.


10 months ago

Update here: While we have not tackled the root cause, we have fixed it for the time being and will address the root cause this week. Your pre-deploy commands should now finish just as they always have.


Status changed to Solved medim • 10 months ago


Railway
BOT

10 months ago

✅ The ticket Deployment delays occurring has been marked as completed.


Welcome!

Sign in to your Railway account to join the conversation.

Loading...