For high-security environments or doomsday preppers, here is how to go 100% offline.
1. Download Everything First
You need the model files (.gguf) and the installer (Ollama or LM Studio) on a USB drive.
2. Sideloading Models in Ollama
If you have a .gguf file (e.g., llama3.gguf) and no internet:
- Create a file named
Modelfile:
FROM ./llama3.gguf- Run:
ollama create my-offline-model -f Modelfile - Run:
ollama run my-offline-model
3. Verifying Integrity
Always check the SHA256 hash of the downloaded files against the Hugging Face repository to ensure no corruption or tampering occurred during download.