2 years ago
Has anyone noticed deploys are going out pretty slow? I'm deploying vanilla Go servers, no docker or special NIX dependencies required, but I'm seeing deploy times up to several minutes. Sometimes they go by quickly, <1 minute, and sometimes it takes several minutes per deploy. If an admin sees this, here is a relevant build:
It's the container step that's taking much longer that I'm used to.
44 Replies
Logs if it helps
2 years ago
yeah deploy times are not the best right now, the team is working on speeding them up.
cc @angelo
zaydek@Zaydeks-MacBook-Pro orderediting-go % railway up --service=headers-experimental --detach
Indexed
Compressed [====================] 100%
Failed Failed to upload code with status code 500 Internal Server Error
zaydek@Zaydeks-MacBook-Pro orderediting-go % railway up --service=headers-experimental --detach
Indexed
Compressed [====================] 100%
Failed Failed to upload code with status code 500 Internal Server Error
@Brody If I use a Dockerfile will it bypass the deploy container step or not necessarily?
2 years ago
container still has to be deployed regardless dockerfile, nixpacks, source image, etc
2 years ago
Yea, the publish step is the same regardless
2 years ago
Hope to see this improve soon. Love railway, but these recent deployment times are really killin it for me.
stuck on step Deploy > Create Container for 15 minutes, and then it timed out with activity StartToClose timeout
2 years ago
the team is actively working on improving this
can we get an announcement or something in the incidents channel when the fix is deployed?
2 years ago
its possible that the team will post some kind of post mortem
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
container event container died
2 years ago
please open your own thread as this seems unrelated to the topic of this thread.
2 years ago
thats a private thread, this topic does not need to be a private thread
2 years ago
Going to investigate- in this case, I don't mind.
2 years ago
Point being is that the team, and esp. myself know how frustrating this whole experience is right now.
as we are pro, and because of what it says it was supposed to be faster, I opened it this way, as we have been stopped several teams because of this problem
2 years ago
i understand, but please know that it says that message no matter the option selected
2 years ago
Okay @here, raised a post-facto incident, https://discord.com/channels/713503345364697088/846875565357006878/1261026504574369894 and we expect to have a fix to this in 4 hours. Early reports are looking like a host config issue, we plan to share more on what happened.
2 years ago
Angelo, if it's not too much trouble, could you re-share that incident on this forum or in a spot with more visibility?
2 years ago
@Brody / @angelo As of 9:18 PDT, the Railway status incidents page shows All Systems Operational. However, I'm still observing degraded deploy speeds. I've seen this across a few different apps including MySQL. Here are a couple of screenshots of a stuck MySQL deployment. Are you observing these issues on your end as well?
Attachments
2 years ago
Both those screenshots show a single active deployment, in this case the deployment event stage listing as not started is likely just a bug.
2 years ago
@Brody I agree it's a bug. I want to add that another observation: when I push a new deployment, I need to manually remove the previous one. That suggests there's still an issue. Is the team planning to investigate this issue?
2 years ago
We are working on that one as we speak, does that deployment have a volume per chance?
2 years ago
Thanks.
Yes, it does. I had a hunch it was related to volumes and wrote up what I'm seeing here: https://help.railway.app/questions/issues-with-volumes-68850731
2 years ago
Is this still an issue? I tried a deploy just now and the deploy phase took 4m.
2 years ago
yes this is still an issue
2 years ago
you would want to optimize your image down to below 1gb ideally
locally, my folder builded, has 851 mb, idk what's using more than 2GB.
2 years ago
thats not the same as the built image, please work on writing a dockerfile that results in a smaller image