Force full rebuild from GitHub private library
maurobiscotti-ship-it
HOBBYOP

10 days ago

I have a series of services that I configured as sepated projects in my Railway workspace.

All these services are written in python and require a (common) library to work. My library is placed on GitHub as a private library, and i configured in Railway the necessary GITHUB_TOKEN to access it, and then referenced it in the resources.txt file to be installed at each build

clb @ git+https://x-access-token:${GITHUB_TOKEN}@github.com/maurobiscotti-ship-it/CLB.git@main

I alsto put in the library a yaml file and a TARGET_RAILWAY variable with all the enviroment/services to trigger the automatic rebuild of each service when I update the lubrary.

The problem is that (apparently) Railway is always rebuilding using some layer cache, so that the correct new library code is never used (unless I delete or duplicate the service). I put NO_CACHE = 1 as a variable and toggled on the wait for CI, but nothing seems to work....

Can you kindly help me?

Attachments

Solved$10 Bounty

Pinned Solution

I think @main is a moving ref, but the dependency line in requirements.txt never changes, ...so railway/pip can reuse cached layers or treat it as the same install and skip fetching the latest commit.Can you try pinning the gitHub package to a commit SHA/tag (and updating that SHA on each library release) changes the build input, which kind of forces a fresh install of the new library version.

3 Replies

I think @main is a moving ref, but the dependency line in requirements.txt never changes, ...so railway/pip can reuse cached layers or treat it as the same install and skip fetching the latest commit.Can you try pinning the gitHub package to a commit SHA/tag (and updating that SHA on each library release) changes the build input, which kind of forces a fresh install of the new library version.


maurobiscotti-ship-it
HOBBYOP

10 days ago

Thank you for your answer; I considered that option but it seems a bit unreasonable to update the SHA on multiple requirements.txt files (one for each service) every time I update the library (and I’m doing it pretty often). I just want to have Railway rebuild from the library the services with the latest main branch without using the caches. It seems me a reasonable expectation. I cannot understand why I cannot simply disable the caching. A simple solution for a simple problem.


dharmateja

I think @main is a moving ref, but the dependency line in requirements.txt never changes, ...so railway/pip can reuse cached layers or treat it as the same install and skip fetching the latest commit.Can you try pinning the gitHub package to a commit SHA/tag (and updating that SHA on each library release) changes the build input, which kind of forces a fresh install of the new library version.

maurobiscotti-ship-it
HOBBYOP

4 days ago

Hi Dharmateja, at the end I did as you suggested, pinning it to the Tag with a Tag deployment script and then propagating it on all the local requirements.txt in the repositories and then pushing them in remote. I would have preferred something faster and more simpler, but it works pretty fine. Thank you again for your advice.


Status changed to Solved ray-chen 3 days ago


Loading...