a year ago
I'm trying to deploy Ollama using Docker and need guidance on the following points:
Deployment Process:
Are there any official Docker files or community-contributed ones for deploying Ollama?
If I need to create my own Dockerfile, what steps should I follow?
Endpoint URL:
Once deployed, how do I determine the correct endpoint URL to use for API requests?
Should it default to
http://localhost:11434or something else when deployed in Docker?
My Current Setup:
I have a Docker environment set up on [local machine/remote server].
I am using the Ollama model (e.g.,
llama2,llama3).Here is my current Dockerfile configuration (if applicable):
dockerfileFROM ubuntu:22.04RUN apt-get update && apt-get install -y \curl \&& rm -rf /var/lib/apt/lists/*RUN curl -L https://ollama.ai/install.sh | shEXPOSE 11434# Start Ollama and preload the modelCMD ["sh", "-c", "ollama serve & sleep 10 && ollama pull llama3.2 && tail -f /dev/null"]
Error or Confusion:
If I run the above Docker container, I'm unsure of the endpoint URL to use for testing the deployed Ollama API.
Are there additional configurations or flags required during deployment to ensure the API runs on the expected port?
Goal:
I want to deploy Ollama, access the API using a custom URL, and integrate it into my applications.
6 Replies
a year ago
Hello,
Have you checked out our Ollama template?
https://railway.com/template/T9CQ5w
brody
Hello,Have you checked out our Ollama template?https://railway.com/template/T9CQ5w
a year ago
Yes, but this doesn't download model.
a year ago
I'm not familiar with Ollama, but can you initiate a model download via the WebUI?
brody
I'm not familiar with Ollama, but can you initiate a model download via the WebUI?
a year ago
Not using WebUI, have my own custom app, is that integration possible?
a year ago
Right, but can you initiate a model download via the provided WebUI? then you can use Ollama in your own app.
a year ago
No, it will not work that way for the use case I am trying to build. Will try to explore though, thanks.