Ollama | Self-Host Open-Source LLMs
7 months ago
Description: Self-host your own AI model server with a single click.
Category: AI/ML
URL: https://railway.com/deploy/ollama-or-self-host-open-source-llms
2 Replies
pdtran3k6
HOBBY
13 days ago
How do I pull Gemma (or any models) to this template?
11 days ago
hey, there are couple of options to pull models in ollama -
- Once the ollama is up, you can curl the pull endpoint and download any model programmatically (one by one for each model you would want). It would look something like this - curl https://.up.railway.app/api/pull -d '{"name":"llama3.2"}'
- Via start command, so add an env variable like
MODELS=qwen2.5:0.5b,<any-other-model(s)>and then update your ollama's start command tosh -c 'ollama serve & pid=$!; until ollama list >/dev/null 2>&1; do sleep 1; done; for m in $(echo "$MODELS" | tr "," " "); do ollama pull "$m"; done; wait $pid'. It will download all your comma separated specified models one by one
Make sure you have enough space available in your railway project that the models you are trying to pull can fit in memory.
Status changed to Solved Railway • 11 days ago