Ollama is the fastest and simplest way to run powerful open-source AI language models locally on your Windows 11 PC. Unlike complex Python environment setups that can take hours, Ollama installs in under 2 minutes and runs models with a single terminal command. This complete Windows 11-focused tutorial covers everything from installation to advanced GPU configuration and API integration.

Step 1: Install Ollama on Windows 11

  1. Open your browser and use to ollama.com/download.
  2. Click the Download for Windows button — this downloads OllamaSetup.exe (approximately 85MB).
  3. Run the installer. Ollama installs silently and automatically adds itself to your system PATH. No manual PATH configuration needed.
  4. Verify installation: Open PowerShell and type ollama --version. You should see the current version number.

Step 2: Enable GPU Acceleration (Critical for Speed)

Ollama automatically detects and uses your NVIDIA or AMD GPU for dramatically faster inference. Ensure your GPU drivers are up-to-date:

✅ NVIDIA GPUs (Best Support)

  • Requires CUDA 11.8+ drivers
  • Update via: NVIDIA App or nvidia.com/drivers
  • Verify CUDA: nvidia-smi in PowerShell
  • GTX 1660 (6GB VRAM) → Runs 7B models
  • RTX 3080 (10GB) → Runs 13B–20B models
  • RTX 4090 (24GB) → Runs up to 70B models

🔴 AMD GPUs (ROCm Support)

  • ROCm support on Windows is limited
  • RX 6000/7000 series: partial support
  • Install AMD Adrenalin 24.x+ drivers
  • Fallback: CPU inference (slower but works)
  • Best AMD support via WSL2 (Linux backend)

Step 3: Pull and Run Your First Model

Open PowerShell and run these commands:

# Pull Llama 3.2 (3B params, ~2GB, great for quick chat)
ollama pull llama3.2

# Pull Mistral 7B (better quality, ~4GB)
ollama pull mistral

# Pull DeepSeek R1 for reasoning tasks
ollama pull deepseek-r1:7b

# Start an interactive chat session
ollama run llama3.2

# List all downloaded models
ollama list

# Remove a model to free disk space
ollama rm mistral

Step 4: Use the Ollama REST API

Ollama runs a local HTTP server on localhost:11434 that accepts OpenAI-compatible API calls. This lets you integrate it into any application, script, or tool that supports custom API endpoints:

# Chat completion (streaming)
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {"role": "user", "content": "Write a Python function to sort a list"}
  ]
}'

# Non-streaming generation
curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "Explain blockchain in simple terms",
  "stream": false
}'

Step 5: Install Open WebUI for a ChatGPT-Like Interface

Open WebUI gives you a beautiful browser-based interface for Ollama — identical in feel to ChatGPT but running 100% locally. Install via Docker Desktop:

# Install Docker Desktop first from docker.com
# Then run:
docker run -d -p 3000:8080   --add-host=host.docker.internal:host-gateway   -v open-webui:/app/backend/data   --name open-webui   ghcr.io/open-webui/open-webui:main

Open your browser and go to http://localhost:3000. Create a local account and start chatting with any Ollama model through a polished ChatGPT-style interface.

Ollama Environment Variables (Windows Configuration)

VariablePurposeExample Value
OLLAMA_MODELSChange model storage locationD:OllamaModels
OLLAMA_HOSTBind to network (LAN access)0.0.0.0:11434
OLLAMA_NUM_PARALLELConcurrent requests4
OLLAMA_FLASH_ATTENTIONFaster attention (NVIDIA)1
CUDA_VISIBLE_DEVICESSelect specific GPU0

Set these via Windows System Properties → Environment Variables, or prepend them to the Ollama command in PowerShell: $env:OLLAMA_HOST="0.0.0.0:11434"; ollama serve.

Configuring GPU Layer Offloading & Environment Variables

To extract peak token generation speeds in Windows 11, configure Ollama's environment variables in System Properties:

# Force Ollama to keep models resident in VRAM for 30 minutes
setx OLLAMA_KEEP_ALIVE "30m"

# Listen on all local network adapters for home lab sharing
setx OLLAMA_HOST "0.0.0.0:11434"

Setting OLLAMA_KEEP_ALIVE avoids repeated cold-start latency when making intermittent coding queries throughout the day.

Multi-GPU Load Balancing in Local Inference

If your PC includes multiple graphics cards (such as an RTX 4070 paired with an older RTX 3060), Ollama automatically splits transformer layers across both GPUs via CUDA peer-to-peer memory access, expanding total usable VRAM buffer capacity without manual tensor parallel configuration.