Backend needs QMD - Mini LLM model for re-ranking - OOM exception
studyhourlabs
HOBBYOP

a month ago

Hi

My project uses QMD library which downloads a 1.28GB model. There are 2 issues

1. Project has gone Out Of Memory - I need to expand Volume Storage to at least 3 GB

2. Model is being downloaded to - ~/.cache/qmd/models - This is only one time and it should not be ephemeral. Across new build sessions, it should survive. Ideally should be on FileSystem

I need to point like so XDG_CACHE_HOME=/app/data/.cache - see here https://github.com/tobi/qmd/blob/cfd640ed3499769b3ee41a7118119ff884dbe8c5/CHANGELOG.md?plain=1#L90

  1. I need to run a one time node set up script for sqlite QMD db reindexing, how do I do that on railway. I do not want to do it during build as it is one time set up. Only when a new document is uploaded, I should re run again.

Can someone help me with 3 points please ? I am ok to pay for Volume Expansion as most of my work is with sqlite and files on disk.

Thanks

Solved$10 Bounty

11 Replies

Status changed to Open Railway about 1 month ago


domehane
FREE

a month ago

Hello,

for point 1, go to your volume settings on the railway dashboard and hit live resize, bump it to 3gb+. it works on hobby and pro with zero downtime (you should upgrade your free plan), no redeploy needed


domehane
FREE

a month ago

for point 2, mount your volume to /app/data then add XDG_CACHE_HOME=/app/data/.cache as an environment variable in your service settings , since you already confirmed qmd respects that variable, the model will download to your persistent volume and survive redeploys , one important thing , volumes are not mounted during build time on railway, only at runtime, so make sure the model download is triggered at runtime not during build


domehane
FREE

a month ago

for point 3 , use railway ssh. install the railway cli, right click your service on the dashboard, select "copy ssh command", then run your reindex script from your terminal. that's the railway native way to run a one-off command without touching your build or start command

Hope this help you :)


studyhourlabs
HOBBYOP

a month ago

AMazing - Thanks for the quick reply folks, trying now. Will come back.


studyhourlabs
HOBBYOP

a month ago

Hi Folks

I did the steps. For step 3 , I wrote an endpoint /POST to run the npm command and this /POST is protected. When I run that , I see it is not taking the Volume path correctly. I used FileBrowser service to upload and volume is mapped to /app/data but, looks like Volume service and FileBrowser are not linked for some reason or I am not able to figure out.

On Railway, REPO_ROOT is effectively /app, so the setup script is looking for:

  • /app/data/myFolder1
  • /app/data/myFolder2

My log shows:

  • ENOENT: no such file or directory, scandir '/app/data/myFolder1'

So this is not an auth issue and not a QMD issue.

It is a data-path / volume-mount issue I think - How to ensure this please ?


studyhourlabs

Hi Folks I did the steps. For step 3 , I wrote an endpoint /POST to run the npm command and this /POST is protected. When I run that , I see it is not taking the Volume path correctly. I used FileBrowser service to upload and volume is mapped to /app/data but, looks like Volume service and FileBrowser are not linked for some reason or I am not able to figure out. On Railway, REPO\_ROOT is effectively /app, so the setup script is looking for: * /app/data/myFolder1 * /app/data/myFolder2 My log shows: * ENOENT: no such file or directory, scandir '/app/data/myFolder1' So this is not an auth issue and not a QMD issue. It is a data-path / volume-mount issue I think - How to ensure this please ?

domehane
FREE

a month ago

i think on railway , volumes cannot be shared between two service. so your filebrowser service and your backend service each have their own separate volume, they cannot see each other's files, which is why your backend gets ENOENT on /app/data/myFolder1

to fix this you can drop filebrowser as a separate service and handle file uploads directly through your backend service, which already has the volume mounted at /app/data. that way everything is on the same volume


studyhourlabs
HOBBYOP

a month ago

I need FileBrowser service UI to upload which will then sync to my original studyhour-volume. I am just stuck , FileBrowser is just not working , randomly crashing. Is there a step by Step tutorial on how to attach and moutn a file volume. It is the most crticail part of my project!

Attachments


studyhourlabs

I need FileBrowser service UI to upload which will then sync to my original studyhour-volume. I am just stuck , FileBrowser is just not working , randomly crashing. Is there a step by Step tutorial on how to attach and moutn a file volume. It is the most crticail part of my project! ![](https://station-server.railway.com/attachments/att_01kpt4hkt0faha8mn0rr68q6ek)

domehane
FREE

a month ago

filebrowser is crashing because the volume is still attached to your backend service. the volume-filebrowser tool is not designed to run at the same time as your backend. this is how it actually works :

step 1 , right click your volume, go to settings, and note the current mount path

step 2 , disconnect the volume from your backend service

step 3, on the filebrowser service, make sure the environment variable USE_VOLUME_ROOT is set to 1, then mount the volume to the filebrowser service

step 4 , now filebrowser will deploy correctly and you can upload your files

step 5, once done disconnect the volume from filebrowser and remount it back to your backend service using the original mount path you noted in step 1

so filebrowser and your backend cannot both use the volume at the same time on railway you swap the volume between them when you need to manage files


studyhourlabs
HOBBYOP

a month ago

Many thanks for the response.

I deleted everything , added FileBrowser, set up user and password. USE_VOLUME_ROOT was not needed, I am not sure what is that used for, it is 0.

----

Step 5 was the KEY & the trick. Thanks.

I needed to

Right-click volume → Disconnect

Right-click volume → Mount → select service → set path

Deploy

----

The only issue or rather small issue is that the data is at /app/data/storage. I need to change code to read from this DIR in Prod. I will do that. So, looks liek I need to point cache folder to this path then. The XDG_CACHE_HOME=/app/data/storage/.cache

Now that infra is setup, I will change code and come back here.

Thank you!


studyhourlabs

Many thanks for the response. I deleted everything , added FileBrowser, set up user and password. USE\_VOLUME\_ROOT was not needed, I am not sure what is that used for, it is 0. \---- Step 5 was the KEY & the trick. Thanks. I needed to Right-click volume → Disconnect Right-click volume → Mount → select service → set path Deploy \---- The only issue or rather small issue is that the data is at /app/data/storage. I need to change code to read from this DIR in Prod. I will do that. So, looks liek I need to point cache folder to this path then. The XDG\_CACHE\_HOME=/app/data/storage/.cache Now that infra is setup, I will change code and come back here. Thank you!

domehane
FREE

a month ago

glad it worked, yes just update XDG_CACHE_HOME=/app/data/storage/.cache in your railway service variables and the qmd model will download to the right place on your persistent volume


domehane

glad it worked, yes just update XDG\_CACHE\_HOME=/app/data/storage/.cache in your railway service variables and the qmd model will download to the right place on your persistent volume

studyhourlabs
HOBBYOP

a month ago

It works end to end now. Thank you for all the help.


Status changed to Solved studyhourlabs about 1 month ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...