Slow builds: node_modules re-installed every time
w1nt3r-eth
PROOP

a year ago

Every time I deploy a new change, the build process goes through dependency installation step, even though my package.json/yarn.lock haven't changed.

Looking at the build logs, I see

#9 [stage-0  5/10] COPY . /app/.
…
#10 [stage-0  6/10] RUN --mount=type=cache,… yarn install --frozen-lockfile#11 [stage-0  7/10] COPY . /app/.

Railway is mounting cache at cache/yarn/v6, but it still takes a while to re-install/re-link the packages from cache into the Docker container layer.

I'd expect Railway to cache the whole Docker layer after installation and reuse it if package.json/yarn.lock didn't change. To do that, step #9 should be something like COPY package.json yarn.lock /app/

Any reason not to do that? Would you recommend switching to a Dockerfile deploys instead?

Solved

7 Replies

melissa
PRO

a year ago

Hey @w1nt3r-eth,

Without a Dockerfile present, Railway uses Nixpacks to build your code from the source. You can control the behavior of Nixpacks using env vars and/or a config file: https://docs.railway.app/guides/build-configuration#nixpacks-options || https://nixpacks.com/docs/configuration/file

If you have the appetite and know-how, I would suggest switching to a Dockerfile though, as it gives you more control over the build phase and leads to faster builds as well.


Status changed to Awaiting User Response Railway over 1 year ago


w1nt3r-eth
PROOP

a year ago

Thanks @melissa! Dockerfile sounds good to me, and it looks like it does cache layers between the deploys


Status changed to Awaiting Railway Response Railway over 1 year ago


melissa
PRO

a year ago

Ah yes, it does! But yes, Dockerfile is the way to go, and the more you can reduce your image size, the faster your deploys will be


Status changed to Awaiting User Response Railway over 1 year ago


w1nt3r-eth
PROOP

a year ago

Converted to Dockerfile, but caching is not happening.

I can see CACHED after it's processing FROM node:20, but then it didn't cache the layers where I install node_modules, even though my package.json didn't change. Link to the build


Status changed to Awaiting Railway Response Railway over 1 year ago


melissa
PRO

a year ago

Hrm, yeah I'm seeing the same in a little test I just set up. I'm going to flag to our infra team and see if something is awry. Will keep you posted!


Status changed to Awaiting User Response Railway over 1 year ago


w1nt3r-eth
PROOP

a year ago

Thank you!


Status changed to Awaiting Railway Response Railway over 1 year ago


Status changed to In Progress melissa over 1 year ago


melissa
PRO

a year ago

alrighty, just downloaded some wisdom from our lead engineer on the build system.

our fleet of builders scales up/down depending on build volume, so it's not abnormal to see cache misses, especially during times of high volume. basically, to get a cache hit, your build must land on the same builder as the previous build.

I think this is a great bit of info for our docs, so I've created a ticket to add it.

Beyond that, if you need to ensure fast deploys, you may consider setting up a build pipeline to build the image yourself, so you can just deploy directly from the image.

I hope this helps, let me know if you have any questions


Status changed to Solved melissa over 1 year ago


Loading...