docker-compose-cookbooks/ollama

Ollama with Webui

There is a cpu version and a gpu version. For ollama I'm just running latest, no version lock. Same with standard ports, no option to change.

Dockerhub: https://hub.docker.com/r/ollama/ollama Github (instructions): https://github.com/open-webui/open-webui Reverse Proxy Port webui: 8080 (The docker-compose sets the port to 3000) Ollama Models: https://ollama.com/search

Overview

Local large language model runner with the Open WebUI front end.

Project Details

• Container Image: ollama/ollama:latest
• Reverse Proxy Port: 11434

Getting Started

1. Start the container: docker compose up -d
2. Open http://localhost:11434 in your browser
3. Follow the initial setup wizard to configure the application

Environment Variable Notes

OLLAMA_KEEP_ALIVE – default: 5m
OLLAMA_PORT – default: 11434
WEBUI_URL – default: http://localhost:8080
WEBUI_PORT – default: 8080

Volume Notes

/root/.ollama – host path /data/ollama
/root/.cache – host path /data/webui/cache
/app/backend/data – host path /data/webui/data

Network Notes

Requires proxy network

Docker Run

docker run -d \
  --name ollama \
  -p 11434:11434 \
  -v /data/ollama:/root/.ollama \
  -v /data/webui/cache:/root/.cache \
  -v /data/webui/data:/app/backend/data \
  ollama/ollama:latest

See compose.yaml for the full set of environment variables.

Additional Notes / Gotchas

Nothing specific to this stack so far.

Dockhand Stack, Deploy from Git

Cookbooks Repository stackname: ollama Compose file path: ollama/compose.yaml Additional env file (optional): ollama/sample.env

Then "Load" ollama/sample.env into the Environmental variables in dockhand

Create the Stack

Preperation

You need to have nvidia-drivers, cuda and nvidia-container-toolkit installed before proceeding.

docker run for refrence:

docker run -d --runtime=nvidia --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

docker exec -it ollama ollama pull llama3
docker exec -it ollama ollama run llama3

Environment

WEBUI_SECRET_KEY is just a random secret string used to secure sessions, cookies, and auth in your WebUI. openssl rand -hex 32

You could also set OLLAMA_KEEP_ALIVE=5m if you are running multiple models. 5m is default, 0 removes as soon as done (then would have to reload model, so not quick), more than 5m is longer (minutes). See "Docker Container Cheatsheet" below for how to remove models from memory.

Use

• Ollama Models

When using the webui for the first time after an upgrade, you will need to wait a long time. This is due to the model pulling process.

Docker Container Cheatsheet

You can alias:

alias ollamac="docker compose exec -it ollama ollama"

#example then would be
ollamac list

Standard commands

# Enter a shell
docker exec -it ollama-ollama-1 sh

# Add/Pull a  model
ollama pull llama3.1:8b

# list models
ollama list

# Remove a model
ollama rm <model-name>

# Run CLI
ollama run <model-name> "What is Star Trek?"

# See what's running
ollama ps

# Clear memory
ollama stop <model-name>

API

curl http://ollama.lan:11434/api/tags

Notes

Straight docker run:

docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

Recommended Pulls

For a NVIDIA GeForce RTX 5060 Ti, your GPU may vary.

Generating prompts for InvokeAI
ollama pull llama3.1:8b
ollama pull mistral:7b-instruct
ollama pull mythomax-l2:13b

Generating PHP code
ollama pull codellama:13b-code
ollama pull deepseek-coder:6.7b-instruct
ollama pull wizardcoder:15b

Help with writing a book
ollama pull llama3.1:70b-instruct
ollama pull mythomax-l2:13b
ollama pull openhermes2.5-mistral:7b

Help with psychology questions/learning
ollama pull samantha-mistral:7b
ollama pull openchat:7b
ollama pull nous-hermes2-mixtral:8x7b-dpo

n8n agents
ollama pull qwen2.5-coder:7b-instruct
ollama pull llama3.1:8b-instruct
ollama pull gorilla-openfunctions-v2:7b

Summarizing documents
ollama pull granite3.2:8b-instruct
ollama pull cognitivetech/obook_summary:7b
ollama pull bge-m3:8b