mirror of
https://github.com/nickyeoman/docker-compose-cookbooks.git
synced 2026-09-04 10:56:23 +00:00
codeandClaude Fable 5
340866f9f4
Existing content preserved and reordered to the canonical section order; missing sections generated from each stack's compose data (ports, volumes, env vars, image links). Fixes copy-paste errors in Dockhand sections (qbittorrent pointed at archivebox; invokeai/thunderbird said SERVICENAME) and persistant->persistent typos in gatus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
184 lines
4.3 KiB
Markdown
184 lines
4.3 KiB
Markdown
# Ollama with Webui
|
||
|
||
|
||
|
||
There is a cpu version and a gpu version.
|
||
For ollama I'm just running latest, no version lock.
|
||
Same with standard ports, no option to change.
|
||
|
||
Dockerhub: https://hub.docker.com/r/ollama/ollama
|
||
Github (instructions): https://github.com/open-webui/open-webui
|
||
Reverse Proxy Port webui: 8080 (The docker-compose sets the port to 3000)
|
||
Ollama Models: https://ollama.com/search
|
||
|
||
## Overview
|
||
|
||
Local large language model runner with the Open WebUI front end.
|
||
|
||
## Project Details
|
||
|
||
- **Container Image:** [ollama/ollama:latest](https://hub.docker.com/r/ollama/ollama)
|
||
- **Reverse Proxy Port:** `11434`
|
||
|
||
## Getting Started
|
||
|
||
1. Start the container: `docker compose up -d`
|
||
2. Open http://localhost:11434 in your browser
|
||
3. Follow the initial setup wizard to configure the application
|
||
|
||
## Environment Variable Notes
|
||
|
||
OLLAMA_KEEP_ALIVE – default: 5m
|
||
OLLAMA_PORT – default: 11434
|
||
WEBUI_URL – default: http://localhost:8080
|
||
WEBUI_PORT – default: 8080
|
||
|
||
## Volume Notes
|
||
|
||
/root/.ollama – host path /data/ollama
|
||
/root/.cache – host path /data/webui/cache
|
||
/app/backend/data – host path /data/webui/data
|
||
|
||
## Network Notes
|
||
|
||
Requires proxy network
|
||
|
||
## Docker Run
|
||
|
||
```bash
|
||
docker run -d \
|
||
--name ollama \
|
||
-p 11434:11434 \
|
||
-v /data/ollama:/root/.ollama \
|
||
-v /data/webui/cache:/root/.cache \
|
||
-v /data/webui/data:/app/backend/data \
|
||
ollama/ollama:latest
|
||
```
|
||
|
||
See compose.yaml for the full set of environment variables.
|
||
|
||
## Additional Notes / Gotchas
|
||
|
||
Nothing specific to this stack so far.
|
||
|
||
## Dockhand Stack, Deploy from Git
|
||
|
||
Cookbooks Repository
|
||
stackname: ollama
|
||
Compose file path: ollama/compose.yaml
|
||
Additional env file (optional): ollama/sample.env
|
||
|
||
Then "Load" ollama/sample.env into the Environmental variables in dockhand
|
||
|
||
Create the Stack
|
||
|
||
## Preperation
|
||
|
||
You need to have nvidia-drivers, cuda and nvidia-container-toolkit installed before proceeding.
|
||
|
||
docker run for refrence:
|
||
```
|
||
docker run -d --runtime=nvidia --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
|
||
|
||
docker exec -it ollama ollama pull llama3
|
||
docker exec -it ollama ollama run llama3
|
||
```
|
||
|
||
## Environment
|
||
|
||
WEBUI_SECRET_KEY is just a random secret string used to secure sessions, cookies, and auth in your WebUI. ```openssl rand -hex 32```
|
||
|
||
You could also set OLLAMA_KEEP_ALIVE=5m if you are running multiple models. 5m is default, 0 removes as soon as done (then would have to reload model, so not quick), more than 5m is longer (minutes).
|
||
See "Docker Container Cheatsheet" below for how to remove models from memory.
|
||
|
||
## Use
|
||
|
||
- [Ollama Models](https://ollama.com/search)
|
||
|
||
When using the webui for the first time after an upgrade, you will need to wait a long time. This is due to the model pulling process.
|
||
|
||
### Docker Container Cheatsheet
|
||
|
||
You can alias:
|
||
```bash
|
||
alias ollamac="docker compose exec -it ollama ollama"
|
||
|
||
#example then would be
|
||
ollamac list
|
||
```
|
||
|
||
Standard commands
|
||
|
||
```bash
|
||
# Enter a shell
|
||
docker exec -it ollama-ollama-1 sh
|
||
|
||
# Add/Pull a model
|
||
ollama pull llama3.1:8b
|
||
|
||
# list models
|
||
ollama list
|
||
|
||
# Remove a model
|
||
ollama rm <model-name>
|
||
|
||
# Run CLI
|
||
ollama run <model-name> "What is Star Trek?"
|
||
|
||
# See what's running
|
||
ollama ps
|
||
|
||
# Clear memory
|
||
ollama stop <model-name>
|
||
|
||
```
|
||
|
||
### API
|
||
|
||
```bash
|
||
curl http://ollama.lan:11434/api/tags
|
||
```
|
||
|
||
# Notes
|
||
|
||
Straight docker run:
|
||
```bash
|
||
docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda
|
||
```
|
||
|
||
## Recommended Pulls
|
||
|
||
For a NVIDIA GeForce RTX 5060 Ti, your GPU may vary.
|
||
|
||
```
|
||
Generating prompts for InvokeAI
|
||
ollama pull llama3.1:8b
|
||
ollama pull mistral:7b-instruct
|
||
ollama pull mythomax-l2:13b
|
||
|
||
Generating PHP code
|
||
ollama pull codellama:13b-code
|
||
ollama pull deepseek-coder:6.7b-instruct
|
||
ollama pull wizardcoder:15b
|
||
|
||
Help with writing a book
|
||
ollama pull llama3.1:70b-instruct
|
||
ollama pull mythomax-l2:13b
|
||
ollama pull openhermes2.5-mistral:7b
|
||
|
||
Help with psychology questions/learning
|
||
ollama pull samantha-mistral:7b
|
||
ollama pull openchat:7b
|
||
ollama pull nous-hermes2-mixtral:8x7b-dpo
|
||
|
||
n8n agents
|
||
ollama pull qwen2.5-coder:7b-instruct
|
||
ollama pull llama3.1:8b-instruct
|
||
ollama pull gorilla-openfunctions-v2:7b
|
||
|
||
Summarizing documents
|
||
ollama pull granite3.2:8b-instruct
|
||
ollama pull cognitivetech/obook_summary:7b
|
||
ollama pull bge-m3:8b
|
||
```
|