3 months ago
Project ID: 968fbf0e-c851-4e88-a772-1ece2d77f4be
Service ID: 874ab1b1-f4d1-460a-8cc8-ea426df9ee70
This service has been running for months and deployed to nearly every day. It is connected to a Github repo that Railway builds via Railpack on every push to main. We have a custom railpack.json so that we can build FFMPEG along with our Elixir app and esbuild our JS & CSS. Again, this entire setup has been working for months, and we have not changed it.
Somehow our latest changes are successfully building, but immediately crash on deploy with the error
/bin/bash: line 1: /app/_build/prod/rel/ekg/bin/server: Permission deniedIt worked fine 19 hours ago, but broken 1 hour ago. I could not see any differences in Railpack to explain this, and we did not change our build process at all. As far as I can tell, something must have changed about the Railway platform and how they do deploys?
This is preventing us from deploying any changes, which is pretty scary. Thanks for your help.
9 Replies
3 months ago
Hello!
We've escalated your issue to our engineering team.
We aim to provide an update within 1 business day.
Please reply to this thread if you have any questions!
Status changed to Awaiting User Response Railway • 3 months ago
3 months ago
We are looking into this!
3 months ago
For now a workaround is to change the building process to use metal
Status changed to Awaiting Railway Response Railway • 3 months ago
Status changed to Awaiting User Response brody • 3 months ago
Same problem here
I added this line to our railway dockerfile and it fixed the problem:
# Make entire ./scripts folder executable
RUN chmod +x ./scripts/*Of course the scripts are already all executable, and this suddenly started happening today without touching those scripts, so not sure what's going on.
3 months ago
Switching to V3 should fix this.
3 months ago
This should be fixed. Let us know if you still have issues.
Status changed to Solved sam-a • 3 months ago
3 months ago
I'm not sure what "switching to v3" means, but we were able to mitigate it by moving our builds off of Railway entirely and just deploying pre-built docker images.
While I appreciate that the problem is solved, it would have been nice to have a bit of an explanation of what caused it and who was affected. It seems like the problem affected several users, but likely not a majority? Is there something us affected users are doing that is considered legacy or likely to break again in the future?
3 months ago
The v3 builder.
This was an issue that effected 5% of legacy builds, we changed the way we pulled the code archive from GitHub, this new way did not preserve the execution bit of the files from GitHub.
3 months ago
Gotcha, I appreciate the insight!
3 months ago
No problem!