Ollama is the fastest and simplest way to run powerful open-source AI language models locally on your Windows 11 PC. Unlike complex Python environment setups that can take hours, Ollama installs in under 2 minutes and runs models with a single terminal command. This complete Windows 11-focused tutorial covers everything from installation to advanced GPU configuration and API integration.
Step 1: Install Ollama on Windows 11
- Open your browser and use to ollama.com/download.
- Click the Download for Windows button — this downloads
OllamaSetup.exe(approximately 85MB). - Run the installer. Ollama installs silently and automatically adds itself to your system PATH. No manual PATH configuration needed.
- Verify installation: Open PowerShell and type
ollama --version. You should see the current version number.
Step 2: Enable GPU Acceleration (Critical for Speed)
Ollama automatically detects and uses your NVIDIA or AMD GPU for dramatically faster inference. Ensure your GPU drivers are up-to-date:
✅ NVIDIA GPUs (Best Support)
- Requires CUDA 11.8+ drivers
- Update via: NVIDIA App or nvidia.com/drivers
- Verify CUDA:
nvidia-smiin PowerShell - GTX 1660 (6GB VRAM) → Runs 7B models
- RTX 3080 (10GB) → Runs 13B–20B models
- RTX 4090 (24GB) → Runs up to 70B models
🔴 AMD GPUs (ROCm Support)
- ROCm support on Windows is limited
- RX 6000/7000 series: partial support
- Install AMD Adrenalin 24.x+ drivers
- Fallback: CPU inference (slower but works)
- Best AMD support via WSL2 (Linux backend)
Step 3: Pull and Run Your First Model
Open PowerShell and run these commands:
# Pull Llama 3.2 (3B params, ~2GB, great for quick chat) ollama pull llama3.2 # Pull Mistral 7B (better quality, ~4GB) ollama pull mistral # Pull DeepSeek R1 for reasoning tasks ollama pull deepseek-r1:7b # Start an interactive chat session ollama run llama3.2 # List all downloaded models ollama list # Remove a model to free disk space ollama rm mistral
Step 4: Use the Ollama REST API
Ollama runs a local HTTP server on localhost:11434 that accepts OpenAI-compatible API calls. This lets you integrate it into any application, script, or tool that supports custom API endpoints:
# Chat completion (streaming)
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{"role": "user", "content": "Write a Python function to sort a list"}
]
}'
# Non-streaming generation
curl http://localhost:11434/api/generate -d '{
"model": "mistral",
"prompt": "Explain blockchain in simple terms",
"stream": false
}'
Step 5: Install Open WebUI for a ChatGPT-Like Interface
Open WebUI gives you a beautiful browser-based interface for Ollama — identical in feel to ChatGPT but running 100% locally. Install via Docker Desktop:
# Install Docker Desktop first from docker.com # Then run: docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
Open your browser and go to http://localhost:3000. Create a local account and start chatting with any Ollama model through a polished ChatGPT-style interface.
Ollama Environment Variables (Windows Configuration)
| Variable | Purpose | Example Value |
|---|---|---|
| OLLAMA_MODELS | Change model storage location | D:OllamaModels |
| OLLAMA_HOST | Bind to network (LAN access) | 0.0.0.0:11434 |
| OLLAMA_NUM_PARALLEL | Concurrent requests | 4 |
| OLLAMA_FLASH_ATTENTION | Faster attention (NVIDIA) | 1 |
| CUDA_VISIBLE_DEVICES | Select specific GPU | 0 |
Set these via Windows System Properties → Environment Variables, or prepend them to the Ollama command in PowerShell: $env:OLLAMA_HOST="0.0.0.0:11434"; ollama serve.
Configuring GPU Layer Offloading & Environment Variables
To extract peak token generation speeds in Windows 11, configure Ollama's environment variables in System Properties:
# Force Ollama to keep models resident in VRAM for 30 minutes
setx OLLAMA_KEEP_ALIVE "30m"
# Listen on all local network adapters for home lab sharing
setx OLLAMA_HOST "0.0.0.0:11434"
Setting OLLAMA_KEEP_ALIVE avoids repeated cold-start latency when making intermittent coding queries throughout the day.
Multi-GPU Load Balancing in Local Inference
If your PC includes multiple graphics cards (such as an RTX 4070 paired with an older RTX 3060), Ollama automatically splits transformer layers across both GPUs via CUDA peer-to-peer memory access, expanding total usable VRAM buffer capacity without manual tensor parallel configuration.
- ⚡ Best Free AI Coding Assistants for VS Code in 2026: Replace GitHub Copilot Without Paying a Dollar
- ⚡ How to Run AI Completely Offline Without Internet in 2026: Private Local AI Setup Guide
- ⚡ Best Local AI Models 2026: Top 10 Open-Source LLMs to Run on Your PC (Ranked by Performance)
- ⚡ How to Speed Up Windows 11 in 2026: 15 Proven Tweaks for Faster Boot, Gaming & Daily Use
- ➔ How to Fix Windows 11 24H2 8.63 GB Windows Update Cleanup Bug: Checkpoint Cumulative Updates Explained
- ➔ Google DeepMind Unveils Genie 4: Real-Time Photorealistic Interactive World Model with Native Audio & Gemini Omni Architecture (2026 Deep Dive)
- ➔ How to Fix Windows 11 "Preparing Automatic Repair" Loop: BCD Rebuild & SrtTrail.txt Guide (2026)
- ➔ How Much VRAM Do You Really Need for Local LLMs in 2026? (7B, 14B, 32B & 70B GGUF Sizing Guide)
💬 Reader Discussion & Feedback 0