Complete Local RAG Pipeline

Written by Jakub Rusinowski · Last updated July 10, 2026

Ready to build a 'Chat with PDF' app? Here are the easiest ways to set up a full RAG pipeline locally.

In This Guide

• 1. AnythingLLM (Desktop App)
• 2. PrivateGPT
• 3. Open WebUI (RAG Built-in)

Ready to build a 'Chat with PDF' app? Here are the easiest ways to set up a full RAG pipeline locally.

1. AnythingLLM (Desktop App)

The easiest all-in-one solution.

• Download: useanything.com
• Setup: Install, point it to Ollama, and drag-and-drop your PDFs.
• Features: It handles the vector database and embedding automatically.

2. PrivateGPT

A popular Python project for querying documents.

• Repo: imartinez/privateGPT
• Setup: Requires Python environment setup. Good for developers who want to customize the code.

3. Open WebUI (RAG Built-in)

If you already installed Open WebUI (Tutorial 3), it has RAG built-in.

• Click the + button in the chat bar.
• Upload a file.
• Use # to reference the file in your chat.

← All Guides | Check GPU Compatibility