Ollama | Self-Host Open-Source LLMs

7 months ago

Description: Self-host your own AI model server with a single click.

Category: AI/ML

URL: https://railway.com/deploy/ollama-or-self-host-open-source-llms

Solved

2 Replies

pdtran3k6
HOBBY

13 days ago

How do I pull Gemma (or any models) to this template?


11 days ago

hey, there are couple of options to pull models in ollama -

  1. Once the ollama is up, you can curl the pull endpoint and download any model programmatically (one by one for each model you would want). It would look something like this - curl https://.up.railway.app/api/pull -d '{"name":"llama3.2"}'
  2. Via start command, so add an env variable like MODELS = qwen2.5:0.5b,<any-other-model(s)> and then update your ollama's start command to sh -c 'ollama serve & pid=$!; until ollama list >/dev/null 2>&1; do sleep 1; done; for m in $(echo "$MODELS" | tr "," " "); do ollama pull "$m"; done; wait $pid' . It will download all your comma separated specified models one by one

Make sure you have enough space available in your railway project that the models you are trying to pull can fit in memory.


Status changed to Solved Railway 11 days ago


Welcome!

Sign in to your Railway account to join the conversation.

Loading...