a year ago
I'm trying to deploy Ollama using Docker and need guidance on the following points:
- Deployment Process:
- Are there any official Docker files or community-contributed ones for deploying Ollama?
- If I need to create my own Dockerfile, what steps should I follow?
- Endpoint URL:
- Once deployed, how do I determine the correct endpoint URL to use for API requests?
- Should it default to
http://localhost:11434or something else when deployed in Docker?
- My Current Setup:
- I have a Docker environment set up on [local machine/remote server].
- I am using the Ollama model (e.g.,
llama2,llama3). - Here is my current Dockerfile configuration (if applicable):
dockerfileFROM ubuntu:22.04RUN apt-get update && apt-get install -y \curl \&& rm -rf /var/lib/apt/lists/*RUN curl -L https://ollama.ai/install.sh | shEXPOSE 11434# Start Ollama and preload the modelCMD ["sh", "-c", "ollama serve & sleep 10 && ollama pull llama3.2 && tail -f /dev/null"] - Error or Confusion:
- If I run the above Docker container, I'm unsure of the endpoint URL to use for testing the deployed Ollama API.
- Are there additional configurations or flags required during deployment to ensure the API runs on the expected port?
- Goal:
- I want to deploy Ollama, access the API using a custom URL, and integrate it into my applications.
6 Replies
a year ago
brody
Hello, Have you checked out our Ollama template? <https://railway.com/template/T9CQ5w>
a year ago
Yes, but this doesn't download model.
a year ago
I'm not familiar with Ollama, but can you initiate a model download via the WebUI?
brody
I'm not familiar with Ollama, but can you initiate a model download via the WebUI?
a year ago
Not using WebUI, have my own custom app, is that integration possible?
a year ago
Right, but can you initiate a model download via the provided WebUI? then you can use Ollama in your own app.
a year ago
No, it will not work that way for the use case I am trying to build. Will try to explore though, thanks.