3 months ago
Hello, I would like to upload pdf documents which are 138MB in total to railway volume. I have created the volume but on the documentation I cannot see how to upload files to railway volume. Even the CLI docs do not have this. I have a RAG app which uses pinecone as a vector db. I already deployed but now I want my app to be able to retrieve a full pdf file when a user asks for the full document and not summary coming from chunks saved on pinecone. So the easiest way I thought this would be is to create liks of the pdf docs and map them to my pdf docs which would be saved on railway and from here I ingest my chunks from scratch with those links as metadata and this links will be used to generate full pdf docs to the user. Please assist
Another question is how can I turn my endpoint into an API and configure it with API keys?
6 Replies
3 months ago
Uploading directly to a volume is not a supported feature. You can use an S3 bucket to achieve this behavior, or something similar.
If your application already has a REST API built into it you can go to settings and generate a domain. Then point it to the port.
For adding an API key, you'd likely have to proxy requests unless your service directly supports it. You can use this repository for that. Just point your service to that directory.
Thank you for the response. Do i have to use an s3 bucket from aws account if I had one, or does it mean I can spin up the s3 within railway?
Can I not do the below in documentation instead? I saw these from the docs and I deployed a service that allows me to upload files via web browser. My deployment was successful But when I tried creating the domain endpoint, I got an error that says malformeddomain which I couldnt understand how to debug.
I got the following from documentaion :
https://docs.railway.com/guides/cli
Connect volume to file explorer service: Deploy a simple file browser service that mounts the same volume as your main application. This provides web-based access to your files for download and upload operations.
3 months ago
You can try deploying the template for Minio (https://railway.com/deploy/SMKOEA) and connect it to your volume, then you can just use the CLI to do the upload / download https://github.com/minio/mc
3 months ago
You can use curl, wget or git to download files from online (such as cloud storage, file sharing service, or git repositories) to your volume. These tools are already accessible from within Railway SSH. I would recommend git because it makes file management easier. Just upload your files to some git service online. In my case, since I deployed my Git instance to Railway, I can do everything with my deployment.
3 months ago
Railway volumes don’t let you directly upload files. The easiest way is to first put your PDFs on cloud storage (like S3, Google Drive, or Dropbox) and then save the links as metadata in Pinecone. This way, your app can fetch the full PDF when needed. If you really want to use Railway volumes, you’ll need to write a script in your app that downloads or uploads the files into the volume at runtime.
To turn your app into an API, just add routes (e.g., /get-document) and protect them with an API key by checking the request header against a secret key stored in Railway variables.
3 months ago
You can attach the volume to an app like Filebrowser
Status changed to Solved noahd • 3 months ago