a year ago
Has anyone noticed deploys are going out pretty slow? I'm deploying vanilla Go servers, no docker or special NIX dependencies required, but I'm seeing deploy times up to several minutes. Sometimes they go by quickly, <1 minute, and sometimes it takes several minutes per deploy. If an admin sees this, here is a relevant build:
It's the container step that's taking much longer that I'm used to.
8 Replies
Logs if it helps
a year ago
yeah deploy times are not the best right now, the team is working on speeding them up.
cc @angelo
zaydek@Zaydeks-MacBook-Pro orderediting-go % railway up --service=headers-experimental --detach
Indexed
Compressed [====================] 100%
Failed Failed to upload code with status code 500 Internal Server Error
zaydek@Zaydeks-MacBook-Pro orderediting-go % railway up --service=headers-experimental --detach
Indexed
Compressed [====================] 100%
Failed Failed to upload code with status code 500 Internal Server Error
@Brody If I use a Dockerfile will it bypass the deploy container step or not necessarily?
a year ago
container still has to be deployed regardless dockerfile, nixpacks, source image, etc
a year ago
Yea, the publish step is the same regardless
a year ago
Hope to see this improve soon. Love railway, but these recent deployment times are really killin it for me.
stuck on step Deploy > Create Container for 15 minutes, and then it timed out with activity StartToClose timeout
a year ago
the team is actively working on improving this
can we get an announcement or something in the incidents channel when the fix is deployed?
a year ago
its possible that the team will post some kind of post mortem
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
a year ago
please open your own thread as this seems unrelated to the topic of this thread.
a year ago
thats a private thread, this topic does not need to be a private thread
a year ago
Going to investigate- in this case, I don't mind.
a year ago
Point being is that the team, and esp. myself know how frustrating this whole experience is right now.
as we are pro, and because of what it says it was supposed to be faster, I opened it this way, as we have been stopped several teams because of this problem
a year ago
i understand, but please know that it says that message no matter the option selected
a year ago
Okay @here, raised a post-facto incident, https://discord.com/channels/713503345364697088/846875565357006878/1261026504574369894 and we expect to have a fix to this in 4 hours. Early reports are looking like a host config issue, we plan to share more on what happened.
a year ago
Angelo, if it's not too much trouble, could you re-share that incident on this forum or in a spot with more visibility?
a year ago
@Brody / @angelo As of 9:18 PDT, the Railway status incidents page shows All Systems Operational. However, I'm still observing degraded deploy speeds. I've seen this across a few different apps including MySQL. Here are a couple of screenshots of a stuck MySQL deployment. Are you observing these issues on your end as well?
Attachments
a year ago
Both those screenshots show a single active deployment, in this case the deployment event stage listing as not started is likely just a bug.
a year ago
@Brody I agree it's a bug. I want to add that another observation: when I push a new deployment, I need to manually remove the previous one. That suggests there's still an issue. Is the team planning to investigate this issue?
a year ago
We are working on that one as we speak, does that deployment have a volume per chance?
a year ago
Thanks.
Yes, it does. I had a hunch it was related to volumes and wrote up what I'm seeing here: https://help.railway.app/questions/issues-with-volumes-68850731
a year ago
Is this still an issue? I tried a deploy just now and the deploy phase took 4m.
a year ago
yes this is still an issue
a year ago
you would want to optimize your image down to below 1gb ideally
a year ago
thats not the same as the built image, please work on writing a dockerfile that results in a smaller image