docs: add Windows setup README and default Ollama URL to localhost
Add README with Windows clone/run instructions and Ollama model setup. Default OLLAMA_API_BASE_URL to host.docker.internal so it works out of the box on the same PC, with comments on pointing it at a remote PC. Co-authored-by: Claude <claude-code@anthropic.com>
This commit is contained in:
commit
9518fc1052
8
.gitignore
vendored
Normal file
8
.gitignore
vendored
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
# Local environment / secrets
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# OS / editor cruft
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
152
README.md
Normal file
152
README.md
Normal file
@ -0,0 +1,152 @@
|
|||||||
|
# Open WebUI (CUDA) + Ollama
|
||||||
|
|
||||||
|
A Dockerized [Open WebUI](https://github.com/open-webui/open-webui) setup using the **CUDA/GPU** image, wired up to talk to an [Ollama](https://ollama.com) backend for local LLMs.
|
||||||
|
|
||||||
|
This runs Open WebUI in a container with NVIDIA GPU acceleration and serves the web interface on **port `3012`**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prerequisites (Windows)
|
||||||
|
|
||||||
|
You'll need the following installed on your Windows machine:
|
||||||
|
|
||||||
|
1. **Git** — https://git-scm.com/download/win
|
||||||
|
2. **Docker Desktop** — https://www.docker.com/products/docker-desktop/
|
||||||
|
- During/after install, make sure the **WSL 2 backend** is enabled (Docker Desktop → Settings → General → *Use the WSL 2 based engine*).
|
||||||
|
3. **NVIDIA GPU + drivers** — this uses the CUDA image, so you need an NVIDIA GPU.
|
||||||
|
- Install the latest **NVIDIA Game Ready / Studio driver**: https://www.nvidia.com/download/index.aspx
|
||||||
|
- GPU passthrough to Docker works automatically through WSL 2 with a recent driver — you do **not** need to install the CUDA toolkit separately.
|
||||||
|
- In Docker Desktop → Settings → Resources → make sure your WSL distro is enabled.
|
||||||
|
4. **Ollama** — https://ollama.com/download/windows
|
||||||
|
- This is what actually runs the models. Open WebUI is just the front end.
|
||||||
|
|
||||||
|
> 💡 No NVIDIA GPU? See [Running without a GPU](#running-without-a-gpu) at the bottom.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Clone the repository
|
||||||
|
|
||||||
|
Open **PowerShell** (or Git Bash / Windows Terminal) and run:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
git clone https://your-gitea-server/your-username/OpenWebUI-Cuda.git
|
||||||
|
cd OpenWebUI-Cuda
|
||||||
|
```
|
||||||
|
|
||||||
|
> Replace the URL above with the actual clone URL shown on the Gitea repo page (the green **Clone** button). If the repo is private, Gitea will prompt for your username and password / access token.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Set up Ollama and pull your models
|
||||||
|
|
||||||
|
Open WebUI does **not** download models itself — Ollama does. So first make sure Ollama is running and has at least one model.
|
||||||
|
|
||||||
|
1. After installing Ollama, it runs automatically in the background (check the system tray). You can confirm with:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
ollama --version
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Pull a model (this downloads it). For example:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
ollama pull llama3.2
|
||||||
|
ollama pull qwen2.5
|
||||||
|
```
|
||||||
|
|
||||||
|
3. List what you have available:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
ollama list
|
||||||
|
```
|
||||||
|
|
||||||
|
Anything listed here will show up in Open WebUI's model dropdown once connected.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. (Optional) Point Open WebUI at Ollama on another PC
|
||||||
|
|
||||||
|
**By default this works out of the box** if Ollama is running on the *same* Windows PC. The compose file is already set to:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- OLLAMA_API_BASE_URL=http://host.docker.internal:11434/api
|
||||||
|
```
|
||||||
|
|
||||||
|
`host.docker.internal` lets the container reach Ollama running on your host machine — no changes needed for the common case.
|
||||||
|
|
||||||
|
**Only** if you want to use Ollama running on a **different machine on your network**, edit `docker-compose.yml` and swap in that machine's IP, e.g.:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- OLLAMA_API_BASE_URL=http://192.168.1.50:11434/api
|
||||||
|
```
|
||||||
|
|
||||||
|
> By default Ollama only listens on `localhost`. If Ollama is on a **different** machine, you must also set the `OLLAMA_HOST=0.0.0.0` environment variable on that machine so it accepts connections from the network. (Not needed if it's on the same PC.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Start it up
|
||||||
|
|
||||||
|
From inside the `OpenWebUI-Cuda` folder:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
The first run will download the Open WebUI CUDA image (a few GB), so give it a few minutes.
|
||||||
|
|
||||||
|
Then open your browser to:
|
||||||
|
|
||||||
|
### 👉 http://localhost:3012
|
||||||
|
|
||||||
|
On first visit, create an admin account (the first account registered becomes the admin). Your models from `ollama list` should appear in the model selector at the top.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Everyday commands
|
||||||
|
|
||||||
|
| Action | Command |
|
||||||
|
|---|---|
|
||||||
|
| Start (in background) | `docker compose up -d` |
|
||||||
|
| Stop | `docker compose down` |
|
||||||
|
| Restart | `docker compose restart` |
|
||||||
|
| View logs | `docker compose logs -f` |
|
||||||
|
| Update to latest Open WebUI | `docker compose pull` then `docker compose up -d` |
|
||||||
|
|
||||||
|
Your chats, settings, and accounts are stored in a Docker volume (`open-webui`) and survive restarts and updates.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
**No models show up in the dropdown**
|
||||||
|
- Run `ollama list` — do you actually have models pulled?
|
||||||
|
- Double-check the `OLLAMA_API_BASE_URL` in `docker-compose.yml` ([step 3](#3-optional-point-open-webui-at-ollama-on-another-pc)).
|
||||||
|
- Make sure Ollama is running (system tray icon, or `ollama list` works).
|
||||||
|
- After editing the compose file, re-run `docker compose up -d` to apply changes.
|
||||||
|
|
||||||
|
**`docker compose up` fails with a GPU / nvidia error**
|
||||||
|
- Update your NVIDIA driver (see prerequisites).
|
||||||
|
- Make sure Docker Desktop is using the WSL 2 backend.
|
||||||
|
- Test GPU access with: `docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi`
|
||||||
|
- If you don't have an NVIDIA GPU, see below.
|
||||||
|
|
||||||
|
**Port 3012 already in use**
|
||||||
|
- Change the left side of the ports line in `docker-compose.yml`, e.g. `"3013:8080"`, then use that port in your browser.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Running without a GPU
|
||||||
|
|
||||||
|
The CUDA image requires an NVIDIA GPU. If your friend doesn't have one, switch to the standard image: in `docker-compose.yml`, change
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
image: ghcr.io/open-webui/open-webui:cuda
|
||||||
|
```
|
||||||
|
|
||||||
|
to
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
image: ghcr.io/open-webui/open-webui:main
|
||||||
|
```
|
||||||
|
|
||||||
|
and delete the entire `deploy:` block (the `resources` / `nvidia` GPU section). Ollama will still use the GPU itself if one is present — this only affects Open WebUI's own acceleration.
|
||||||
30
docker-compose.yml
Executable file
30
docker-compose.yml
Executable file
@ -0,0 +1,30 @@
|
|||||||
|
services:
|
||||||
|
open-webui2:
|
||||||
|
image: ghcr.io/open-webui/open-webui:cuda
|
||||||
|
container_name: open-webui2
|
||||||
|
ports:
|
||||||
|
- "3012:8080"
|
||||||
|
volumes:
|
||||||
|
- open-webui:/app/backend/data
|
||||||
|
environment:
|
||||||
|
# Defaults to Ollama running on this same PC (the Docker host).
|
||||||
|
# To use Ollama on ANOTHER machine on your network, replace
|
||||||
|
# host.docker.internal with that machine's IP, e.g.
|
||||||
|
# http://192.168.1.50:11434/api
|
||||||
|
# (On that remote machine, set OLLAMA_HOST=0.0.0.0 so it accepts LAN connections.)
|
||||||
|
- OLLAMA_API_BASE_URL=http://host.docker.internal:11434/api
|
||||||
|
- WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY:-default-random-key}
|
||||||
|
- OPENAI_API_KEY=${OPENAI_API_KEY:-}
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
reservations:
|
||||||
|
devices:
|
||||||
|
- driver: nvidia
|
||||||
|
count: all
|
||||||
|
capabilities: [gpu]
|
||||||
|
extra_hosts:
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
|
restart: always
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
open-webui:
|
||||||
Loading…
x
Reference in New Issue
Block a user