Overview
Only do this one if more than one person needs the AI. For one person, the laptop guide is simpler.
Ollama is the model engine; Open WebUI is a browser interface that connects to it. Together they can serve a household's chat and document questions from a local computer. This is more maintenance than a desktop application: you are responsible for two services and any document-processing dependencies.
There are two different claims to check: that the model runs on your hardware, and that the complete workflow works without internet. A local chat model can coexist with remote embeddings, web search or cloud extraction. One successful chat doesn't establish that document retrieval is offline.
Use the current official installation commands rather than an old copied Docker command. The Open WebUI offline tutorial is hosted in its documentation but marked as community-contributed, not team-supported. The acceptance checks below are editorial recommendations, not a vendor certification.
What you'll need
- A computer capable of running the selected local model
- Official Ollama installation and a completed model download
- Docker and the current Open WebUI quick-start
- Persistent storage for application data and model files
- A trusted LAN and a plan for accounts, updates and backups
Tool considerations
Ollama
- Good fit
- The model engine behind a custom offline setup.
- What stands out
- A reusable local inference service for tools such as Open WebUI. Its default listener is localhost, not the whole home network.
- The trade-off
- The engine alone doesn't curate manuals or provide a complete document library. CPU/GPU speed and memory depend on the selected model.
- Before going offline
- Current versions also offer cloud models and web search. Download a local model and explicitly disable cloud features for a local-only setup.
Open WebUI
- Good fit
- A household wanting browser access to a local model.
- What stands out
- Separates the interface from the model engine and supports document questions. Docker volumes preserve the interface's application data.
- The trade-off
- You maintain the interface, inference engine and document-processing dependencies. The official-hosted offline tutorial is community-contributed.
- Before going offline
- Local chat isn't enough: embeddings, extraction and optional reranking must also be local. OFFLINE_MODE isn't an outbound firewall.
Instructions
Make Ollama work locally first
Install Ollama for your operating system. While online, download and run a local model; the official FAQ uses ollama run llama3.2 as an example. Verify a simple prompt works before introducing a second interface.
Disable Ollama cloud features
Follow the FAQ's operating-system-specific instructions to set OLLAMA_NO_CLOUD=1, or set disable_ollama_cloud to true in Ollama's server.json. Restart Ollama and verify its log reports cloud disabled. Current Ollama includes cloud features, so local-only operation must be deliberate.
Install Open WebUI
Follow the current Docker quick-start and choose the deployment that can reach your Ollama service. Preserve the open-webui data volume and generate a persistent WEBUI_SECRET_KEY as instructed. Keep that key private; don't paste it into a public tutorial or repository.
Set up accounts and the local connection
Open http://localhost:3000 for the standard quick-start, complete account setup and select your local model. Test a chat. A Docker container's localhost isn't necessarily the host's localhost, so use the networking instructions for your chosen deployment.
Prepare the entire document pipeline
In Admin Settings → Documents, inspect embedding and extraction providers and any optional reranker. Choose local services and download required model files while online. Upload and query a non-sensitive document to initialize the needed components.
Configure offline mode
Set OFFLINE_MODE=true in the deployment configuration and restart according to the offline guide. This changes offline-related behavior, including specified model-update behavior, but doesn't block every outbound connection. Remove remote providers and disable optional web-connected tools.
Choose the sharing boundary
Keep Ollama's default 127.0.0.1:11434 binding unless another service genuinely requires access. If configuring OLLAMA_HOST or exposing Open WebUI on the LAN, deliberately set access controls and host firewall rules. Don't port-forward either service to the public internet.
Test new work without internet
Restart the stack with the WAN/uplink disconnected. Start a new chat and index a new short document, then ask a question with a known answer. A cached document can hide a missing embedding-model download, which is why fresh indexing matters.
Back up and rehearse recovery
Follow the installed versions' backup procedures for volumes, configuration and models. Protect account data and keys. Verify restoration safely on a spare setup rather than assuming copying a running database produces a consistent backup.
Check it works offline
- Both services start after a reboot with the internet disconnected.
- A fresh chat and fresh document indexing succeed offline.
- Only intended users on the trusted network can access the interface.
- A tested backup includes application data, model files and required configuration.
Things to watch for
OFFLINE_MODE isn't a firewall or proof of zero outbound traffic. Network isolation is the stronger offline check.
Don't publish server addresses, credentials or signing keys. Use the current security guidance for the installed version.
An account-protected interface doesn't automatically secure every underlying service.
Sources and further reading
Related how-tos
Run an offline AI assistant on your laptop
Start with LM Studio, download a model that fits your computer and prove both chat and document questions work disconnected.
Read the how-toAsk questions of your offline manuals
Use AnythingLLM workspaces to retrieve passages from a small, trusted library while keeping the original documents as the authority.
Read the how-toRun an offline knowledge and maps drill
Restart devices, cut internet access and test fresh work — not just a cached page or an old chat — before depending on the kit.
Read the how-to