a month ago
Hi
My project uses QMD library which downloads a 1.28GB model. There are 2 issues
1. Project has gone Out Of Memory - I need to expand Volume Storage to at least 3 GB
2. Model is being downloaded to - ~/.cache/qmd/models - This is only one time and it should not be ephemeral. Across new build sessions, it should survive. Ideally should be on FileSystem
I need to point like so XDG_CACHE_HOME=/app/data/.cache - see here https://github.com/tobi/qmd/blob/cfd640ed3499769b3ee41a7118119ff884dbe8c5/CHANGELOG.md?plain=1#L90
- I need to run a one time node set up script for sqlite QMD db reindexing, how do I do that on railway. I do not want to do it during build as it is one time set up. Only when a new document is uploaded, I should re run again.
Can someone help me with 3 points please ? I am ok to pay for Volume Expansion as most of my work is with sqlite and files on disk.
Thanks
11 Replies
Status changed to Open Railway • about 1 month ago
a month ago
Hello,
for point 1, go to your volume settings on the railway dashboard and hit live resize, bump it to 3gb+. it works on hobby and pro with zero downtime (you should upgrade your free plan), no redeploy needed
a month ago
for point 2, mount your volume to /app/data then add XDG_CACHE_HOME=/app/data/.cache as an environment variable in your service settings , since you already confirmed qmd respects that variable, the model will download to your persistent volume and survive redeploys , one important thing , volumes are not mounted during build time on railway, only at runtime, so make sure the model download is triggered at runtime not during build
a month ago
for point 3 , use railway ssh. install the railway cli, right click your service on the dashboard, select "copy ssh command", then run your reindex script from your terminal. that's the railway native way to run a one-off command without touching your build or start command
Hope this help you :)
a month ago
AMazing - Thanks for the quick reply folks, trying now. Will come back.
a month ago
Hi Folks
I did the steps. For step 3 , I wrote an endpoint /POST to run the npm command and this /POST is protected. When I run that , I see it is not taking the Volume path correctly. I used FileBrowser service to upload and volume is mapped to /app/data but, looks like Volume service and FileBrowser are not linked for some reason or I am not able to figure out.
On Railway, REPO_ROOT is effectively /app, so the setup script is looking for:
- /app/data/myFolder1
- /app/data/myFolder2
My log shows:
- ENOENT: no such file or directory, scandir '/app/data/myFolder1'
So this is not an auth issue and not a QMD issue.
It is a data-path / volume-mount issue I think - How to ensure this please ?
studyhourlabs
Hi Folks I did the steps. For step 3 , I wrote an endpoint /POST to run the npm command and this /POST is protected. When I run that , I see it is not taking the Volume path correctly. I used FileBrowser service to upload and volume is mapped to /app/data but, looks like Volume service and FileBrowser are not linked for some reason or I am not able to figure out. On Railway, REPO\_ROOT is effectively /app, so the setup script is looking for: * /app/data/myFolder1 * /app/data/myFolder2 My log shows: * ENOENT: no such file or directory, scandir '/app/data/myFolder1' So this is not an auth issue and not a QMD issue. It is a data-path / volume-mount issue I think - How to ensure this please ?
a month ago
i think on railway , volumes cannot be shared between two service. so your filebrowser service and your backend service each have their own separate volume, they cannot see each other's files, which is why your backend gets ENOENT on /app/data/myFolder1
to fix this you can drop filebrowser as a separate service and handle file uploads directly through your backend service, which already has the volume mounted at /app/data. that way everything is on the same volume
a month ago
I need FileBrowser service UI to upload which will then sync to my original studyhour-volume. I am just stuck , FileBrowser is just not working , randomly crashing. Is there a step by Step tutorial on how to attach and moutn a file volume. It is the most crticail part of my project!
Attachments
studyhourlabs
I need FileBrowser service UI to upload which will then sync to my original studyhour-volume. I am just stuck , FileBrowser is just not working , randomly crashing. Is there a step by Step tutorial on how to attach and moutn a file volume. It is the most crticail part of my project! 
a month ago
filebrowser is crashing because the volume is still attached to your backend service. the volume-filebrowser tool is not designed to run at the same time as your backend. this is how it actually works :
step 1 , right click your volume, go to settings, and note the current mount path
step 2 , disconnect the volume from your backend service
step 3, on the filebrowser service, make sure the environment variable USE_VOLUME_ROOT is set to 1, then mount the volume to the filebrowser service
step 4 , now filebrowser will deploy correctly and you can upload your files
step 5, once done disconnect the volume from filebrowser and remount it back to your backend service using the original mount path you noted in step 1
so filebrowser and your backend cannot both use the volume at the same time on railway you swap the volume between them when you need to manage files
a month ago
Many thanks for the response.
I deleted everything , added FileBrowser, set up user and password. USE_VOLUME_ROOT was not needed, I am not sure what is that used for, it is 0.
----
Step 5 was the KEY & the trick. Thanks.
I needed to
Right-click volume → Disconnect
Right-click volume → Mount → select service → set path
Deploy
----
The only issue or rather small issue is that the data is at /app/data/storage. I need to change code to read from this DIR in Prod. I will do that. So, looks liek I need to point cache folder to this path then. The XDG_CACHE_HOME=/app/data/storage/.cache
Now that infra is setup, I will change code and come back here.
Thank you!
studyhourlabs
Many thanks for the response. I deleted everything , added FileBrowser, set up user and password. USE\_VOLUME\_ROOT was not needed, I am not sure what is that used for, it is 0. \---- Step 5 was the KEY & the trick. Thanks. I needed to Right-click volume → Disconnect Right-click volume → Mount → select service → set path Deploy \---- The only issue or rather small issue is that the data is at /app/data/storage. I need to change code to read from this DIR in Prod. I will do that. So, looks liek I need to point cache folder to this path then. The XDG\_CACHE\_HOME=/app/data/storage/.cache Now that infra is setup, I will change code and come back here. Thank you!
a month ago
glad it worked, yes just update XDG_CACHE_HOME=/app/data/storage/.cache in your railway service variables and the qmd model will download to the right place on your persistent volume
domehane
glad it worked, yes just update XDG\_CACHE\_HOME=/app/data/storage/.cache in your railway service variables and the qmd model will download to the right place on your persistent volume
a month ago
It works end to end now. Thank you for all the help.
Status changed to Solved studyhourlabs • about 1 month ago

