Claude OS

╔═══════════════════════════════════════════════════════════════╗
║                                                               ║
║  ██████╗██╗      █████╗ ██╗   ██╗██████╗ ███████╗    ██████╗ ███████╗
║ ██╔════╝██║     ██╔══██╗██║   ██║██╔══██╗██╔════╝   ██╔═══██╗██╔════╝
║ ██║     ██║     ███████║██║   ██║██║  ██║█████╗     ██║   ██║███████╗
║ ██║     ██║     ██╔══██║██║   ██║██║  ██║██╔══╝     ██║   ██║╚════██║
║ ╚██████╗███████╗██║  ██║╚██████╔╝██████╔╝███████╗   ╚██████╔╝███████║
║  ╚═════╝╚══════╝╚═╝  ╚═╝ ╚═════╝ ╚═════╝ ╚══════╝    ╚═════╝ ╚══════╝
║                                                               ║
║           Localized Multi-Knowledge-Base RAG System           ║
║                    with MCP Integration                       ║
║                                                               ║
╚═══════════════════════════════════════════════════════════════╝

Production-grade RAG system with SQLite vector search, Ollama LLMs, and Model Context Protocol. Runs 100% locally on your machine.

---

🚀 What is Claude OS?

Claude OS is a localized RAG (Retrieval-Augmented Generation) system designed to make AI assistants deeply knowledgeable about your codebase and documentation. It combines:

• 🗄️ SQLite + sqlite-vec - Lightweight vector database for semantic search
• 🤖 Ollama - Local LLMs (llama3.1) with no API keys needed
• 🔌 MCP Integration - Claude Desktop integration via Model Context Protocol
• ⚡ React UI - Modern dashboard for managing knowledge bases
• 📚 Multi-KB Support - Separate knowledge bases for different projects
• 🧠 Advanced RAG - Hybrid search, reranking, and agentic modes

Perfect For

✅ Making Claude deeply familiar with your codebase ✅ Private, secure knowledge base (never leaves your machine) ✅ Team collaboration (share with coworkers easily) ✅ Integration with Claude Desktop via MCP ✅ Building AI-assisted development workflows

---

⚡ Quick Start (5-10 minutes)

Prerequisites

You MUST have these installed first:

1. Python 3.11+ (required)

   python3 --version
   

   • macOS: brew install python@3.11
   • Linux: sudo apt-get install python3.11
   • Windows: Download from https://www.python.org/downloads/

2. Git (required)

   git --version
   

   • macOS: brew install git
   • Linux: sudo apt-get install git
   • Windows: Download from https://git-scm.com/

3. Node.js 16+ (optional, for React frontend)

   node --version
   

   • macOS: brew install node
   • Linux: sudo apt-get install nodejs
   • Windows: Download from https://nodejs.org/

---

Quick Install (3 Steps)

Once you have Python 3.11+ and Git, run these commands:

# 1. Clone the repository
git clone https://github.com/brobertsaz/claude-os.git
cd claude-os

# 2. Run the setup script
./setup.sh

# 3. Start all services
./start_all_services.sh

The setup script will automatically:

• ✅ Install Ollama (if needed) + download LLM models
• ✅ Install Redis (if needed) for caching/queues
• ✅ Set up Python virtual environment
• ✅ Install all Python dependencies
• ✅ Create SQLite database
• ✅ Install frontend dependencies (if Node.js present)

Done! Visit http://localhost:5173 to start using Claude OS.

Installation (Step by Step)

Step 1: Install Prerequisites (one-time only)

Choose your OS:

macOS:

# Install Python 3.11
brew install python@3.11

# Install Git
brew install git

# Optional: Install Node.js for frontend
brew install node

Linux (Ubuntu/Debian):

# Install Python 3.11
sudo apt-get update
sudo apt-get install python3.11 python3.11-venv

# Install Git
sudo apt-get install git

# Optional: Install Node.js for frontend
sudo apt-get install nodejs npm

Verify installation:

python3 --version  # Should be 3.11+
git --version      # Should exist

---

Step 2: Clone the Repository

git clone https://github.com/brobertsaz/claude-os.git
cd claude-os

---

Step 3: Run the Installer

./setup.sh

This automatically installs:

• ✅ Ollama - LLM engine (auto-installed if missing)
• ✅ Redis - Cache & queues (auto-installed if missing)
• ✅ Python dependencies - All required packages
• ✅ SQLite database - Local vector store
• ✅ Frontend - React UI (if Node.js present)

The script will download ~5-10 GB of LLM models (llama3.1, embeddings).

---

Step 4: Start Services

./start_all_services.sh

This starts:

• 🔌 MCP Server (port 8051) - Backend RAG engine
• 🎨 React UI (port 5173) - Web dashboard
• 🤖 Ollama (port 11434) - LLM service
• 💾 Redis (port 6379) - Cache & queues
• 🧠 RQ Workers - Real-time learning system

---

Step 5: Access the Application

Open your browser and visit:

• Frontend: http://localhost:5173
• API Docs: http://localhost:8051/docs

Start uploading documents to create knowledge bases!

---

🎯 Initialize Your Project (The Magic Sauce ✨)

Once Claude OS is running, follow these 3 simple steps to make Claude an expert on your codebase:

Step 1: Create a Project in Claude OS UI

1. Open the UI - Visit http://localhost:5173 in your browser

2. Click "Create Project" button

3. Fill in the form:

   • Project Name - e.g., "my-awesome-app"
   • Project Path - Select your project directory (e.g., /Users/you/Projects/my-awesome-app)
   • Description - (optional) Brief description of your project

4. Click "Create Project"

Your project is now registered in Claude OS!

Step 2: Get Your Project ID

After creating your project, you'll see it listed in the Projects view. The Project ID is displayed right on the project card (e.g., #1, #2, etc.).

Simply note the number - that's your project ID!

Example:

• If you see #1 on your project card, your project ID is 1
• If you see #3 on your project card, your project ID is 3

Step 3: Initialize Project with Claude Code

This is where the magic happens! The initialization will:

• ✅ Analyze your entire codebase (~5 minutes)
• ✅ Generate coding standards & architecture docs
• ✅ Index 50 key files (~800 code chunks)
• ✅ Set up Git hooks for auto-indexing
• ✅ Create 4 Knowledge Bases automatically

Open Claude Code and copy-paste this command (replace 1 with your project ID):

/initialize-project 1

⚠️ Important: Replace 1 with your actual project ID from Step 2

Examples:

• If your project ID is 1: /initialize-project 1
• If your project ID is 2: /initialize-project 2
• If your project ID is 3: /initialize-project 3

What happens after you run it:

1. Claude analyzes your project structure, patterns, and conventions (takes ~5 minutes)
2. Generates 3 documentation files:
   • CODING_STANDARDS.md - Your code style
   • ARCHITECTURE.md - Your system design
   • DEVELOPMENT_PRACTICES.md - How you build software
3. Creates semantic indexes for 50 key files (~800+ code chunks)
4. Installs Git hooks to auto-update knowledge on commits
5. Creates 4 Knowledge Bases for use with Claude Desktop

When it's done: ✅ Claude is now an expert on your project! ✅ Ask Claude about your codebase and it will know the answers ✅ Every git commit automatically updates Claude's knowledge

---

🔌 MCP Integration with Claude Desktop

To use your knowledge bases with Claude Desktop:

Option 1: Via CLI (Easiest)

# Add PISTN knowledge base to Claude Desktop
claude mcp add pistn http://localhost:8051/mcp/kb/pistn

# Add PISTN Agent OS knowledge base
claude mcp add pistn-agent-os http://localhost:8051/mcp/kb/pistn-agent-os

Then in Claude Desktop, your knowledge bases are available as built-in tools. Ask Claude about your codebase directly!

Option 2: Manual Configuration

Edit ~/.claude_desktop_config.json:

{
  "mcpServers": {
    "pistn": {
      "url": "http://localhost:8051/mcp/kb/pistn"
    },
    "pistn-agent-os": {
      "url": "http://localhost:8051/mcp/kb/pistn-agent-os"
    }
  }
}

Then restart Claude Desktop.

---

📚 Managing Knowledge Bases

Create a Knowledge Base

1. Visit http://localhost:5173
2. Click "Create Knowledge Base"
3. Choose type:
   • GENERIC - General documentation
   • CODE - Source code repositories
   • DOCUMENTATION - Technical docs
   • AGENT_OS - Spec-driven development

Upload Documents

1. Select a knowledge base
2. Click "Upload Documents"
3. Choose files (supports .md, .txt, .pdf, .py, .js, .ts, .json, .yaml)
4. Watch as documents are indexed

Query the Knowledge Base

Via Web UI:

1. Select a KB from the dropdown
2. Type your question
3. View answer with source citations

Via Claude Desktop: Once added as MCP server, Claude can query automatically when working on your project.

---

🏗️ Architecture

┌─────────────────────────────────────────────────┐
│              Claude Desktop                      │
│         (or any MCP client)                      │
└───────────────────┬─────────────────────────────┘
                    │ MCP HTTP
┌───────────────────▼─────────────────────────────┐
│         MCP Server (Port 8051)                   │
│              FastAPI Backend                     │
└───────────────────┬─────────────────────────────┘
                    │
        ┌───────────┴────────────┐
        │                        │
┌───────▼────────────┐  ┌────────▼──────────┐
│   RAG Engine       │  │  React UI         │
│  (llama-index)     │  │  (Port 5173)      │
│  • Vector Search   │  │                   │
│  • Hybrid Search   │  └────────────────────┘
│  • Reranking       │
└───────┬────────────┘
        │
┌───────▼────────────────────────────────────────┐
│   SQLite + sqlite-vec (Local Database)         │
│  • knowledge_bases                             │
│  • documents (with embeddings)                 │
│  • Single-file database                        │
└────────────────────────────────────────────────┘
        │
        └─────────────────────┐
                              │
                    ┌─────────▼──────┐
                    │  Ollama        │
                    │  (Port 11434)  │
                    │ • llama3.1     │
                    │ • Embeddings   │
                    └────────────────┘

---

⚙️ Configuration

Environment Variables

The system uses sensible defaults, but you can customize via environment variables:

# SQLite Database
SQLITE_DB_PATH=data/claude-os.db  # Default: data/claude-os.db

# Ollama
OLLAMA_HOST=http://localhost:11434  # Default: localhost:11434
OLLAMA_MODEL=llama3.1:latest        # Default: llama3.1:latest

# MCP Server
MCP_SERVER_HOST=0.0.0.0         # Default: 0.0.0.0
MCP_SERVER_PORT=8051            # Default: 8051

---

📊 Performance

Native Ollama Setup (Current)

• Response time: ~40 seconds per query
• GPU acceleration: Full Metal GPU on Apple Silicon
• Memory usage: 8-10GB (models + context)
• CPU usage: 12 cores (M4 Pro, leaves 2 for system)

Why it's fast:

• Direct GPU acceleration (no virtualization)
• Efficient vector search in SQLite
• Optimized RAG engine with caching
• Single-file database with minimal overhead

---

🛠️ Scripts Guide

Claude OS includes comprehensive shell scripts for setup, service management, and testing. Here's what each one does:

Installation & Setup

./setup.sh - Complete Setup ⭐ USE THIS ONE

./setup.sh

This is the standard setup script that 99% of users need.

Prerequisites (must be installed first):

• ✅ Python 3.11+ (required - script will fail if missing)
• ✅ Git (required - script will fail if missing)
• ⚠️ Node.js 16+ (optional - script skips frontend if missing)

What it automatically installs:

• ✅ Ollama - Downloads and starts if not installed
• ✅ Redis - Installs and starts if not present
• ✅ Python virtual environment - Isolated environment
• ✅ Python dependencies - All packages from requirements.txt
• ✅ LLM models - llama3.1 and nomic-embed-text (~5-10 GB)
• ✅ SQLite database - Local vector store
• ✅ Frontend dependencies - npm packages (if Node.js present)

Supported platforms: macOS & Linux

When to use:

• ✅ First-time setup (recommended for 99% of users)
• ✅ You have Python 3.11+ and Git already installed
• ✅ You don't have Ollama or Redis yet
• ✅ You want a complete, automated setup
• ✅ You're on any Unix system (macOS/Linux)

---

./setup_native.sh - Fast macOS Setup (Advanced Users Only)

./setup_native.sh

This is an optional lightweight setup for macOS users who already have Ollama installed via Homebrew.

What it does:

• ✅ Verifies Ollama is already installed via Homebrew (fails if not)
• ✅ Pulls required LLM models (llama3.1, nomic-embed-text)
• ✅ Sets up Python virtual environment
• ✅ Installs Python dependencies
• ✅ Creates data/ and logs/ directories
• ✅ Initializes SQLite database
• ⚠️ Does NOT install Redis

When to use (rare):

• ⚠️ Only if you already have Ollama installed via brew install ollama
• ⚠️ You're on macOS only
• ⚠️ You already have Redis running separately

Note: Most users should use ./setup.sh instead. Only use this if you know what you're doing.

---

Service Management

./start_all_services.sh - Start Everything

./start_all_services.sh

Starts:

• 🔌 MCP Server (port 8051) - Backend RAG engine
• 🎨 React Frontend (port 5173) - Web dashboard
• 🤖 RQ Workers - Real-time learning system
• 💾 Redis - Cache & message queue
• 🧠 Ollama - LLM service

Features:

• Automatic health checks for all services
• Creates required directories if missing
• Checks that ports are free
• Logs all services to logs/ directory
• Shows service URLs and PIDs

Output:

Service URLs:
  🎨 Frontend:    http://localhost:5173
  🔌 API Server:  http://localhost:8051
  📚 API Docs:    http://localhost:8051/docs

Log Files:
  MCP Server:   logs/mcp_server.log
  Frontend:     logs/frontend.log
  RQ Workers:   logs/rq_workers.log

---

./stop_all_services.sh - Stop All Services

./stop_all_services.sh

Stops:

• 🔌 MCP Server (port 8051)
• 🎨 React Frontend (port 5173)
• 🤖 RQ Workers (real-time learning)
• 💾 Redis

Note: Ollama is preserved (may be used by other apps)

---

./restart_services.sh - Restart Everything

./restart_services.sh

Does:

1. Stops all services gracefully
2. Waits for ports to be released (3 second delay)
3. Starts all services fresh

Use when: Code changes aren't reflected or services need a clean restart

---

./start_mcp_server.sh - Start Only MCP Server

./start_mcp_server.sh

Starts:

• 🔌 MCP Server (port 8051)

Checks:

• ✅ Ollama is running
• ✅ SQLite database exists
• ✅ Python environment is ready

Use when: You only need the backend API (e.g., for testing)

---

./start_redis_workers.sh - Start RQ Workers

./start_redis_workers.sh

Starts:

• 🤖 RQ Workers for real-time learning system
• Listening on queues: claude-os:learning, claude-os:prompts, claude-os:ingest
• Scheduler for periodic tasks

Checks:

• ✅ Redis is running
• ✅ Python virtual environment exists
• ✅ RQ dependencies are installed

Use when: Running the real-time learning system separately

---

Testing

./run_tests.sh - Run Test Suite

./run_tests.sh [OPTIONS]

Options:

./run_tests.sh                    # Run all tests
./run_tests.sh --unit             # Unit tests only
./run_tests.sh --integration      # Integration tests only
./run_tests.sh --vector           # Vector DB tests
./run_tests.sh --rag              # RAG engine tests
./run_tests.sh --api              # API endpoint tests
./run_tests.sh --coverage         # Generate coverage report
./run_tests.sh --verbose          # Detailed output
./run_tests.sh --unit --coverage  # Combine options

Features:

• Validates PostgreSQL connection
• Creates test database if needed
• Shows test configuration
• Generates HTML coverage report (with --coverage)

---

📁 Project Structure

claude-os/
├── app/                    # Backend application
│   ├── core/              # Core modules
│   │   ├── config.py      # Configuration management
│   │   ├── sqlite_manager.py  # SQLite interface
│   │   ├── rag_engine.py  # RAG logic
│   │   └── ...
│   ├── db/                # Database schemas
│   └── ...
├── frontend/              # React UI (Vite)
│   ├── src/
│   │   ├── pages/
│   │   ├── components/
│   │   └── ...
│   └── package.json
├── mcp_server/           # MCP Server (HTTP)
│   ├── server.py         # FastAPI + MCP endpoints
│   └── ...
├── migrations/           # Database migrations
├── data/                 # SQLite database (auto-created)
│   └── claude-os.db
├── logs/                 # Service logs (auto-created)
├── .sh scripts           # Utility scripts (see Scripts Guide above)
├── requirements.txt      # Python dependencies
└── README.md             # This file

---

🐛 Troubleshooting

Database Issues

The SQLite database is automatically created at data/claude-os.db. If you encounter issues:

# Check if database exists
ls -lh data/claude-os.db

# Remove and recreate (WARNING: deletes all data)
rm data/claude-os.db
# Restart the server to recreate

Ollama Issues

# Check if Ollama is running
ollama list

# Start Ollama manually
ollama serve

# Check for specific model
ollama list | grep llama3.1

Port Already in Use

# Find process on port 8051 (MCP Server)
lsof -i :8051

# Find process on port 5173 (React UI)
lsof -i :5173

# Kill if needed (replace PID with actual process ID)
kill -9 <PID>

MCP Server Not Starting

# Check logs
tail -f /tmp/mcp_server.log

# Verify Python environment
source venv/bin/activate
python mcp_server/server.py

# Check port 8051 is accessible
curl http://localhost:8051/health

---

📖 Additional Documentation

Core Features

• docs/SELF_LEARNING_SYSTEM.md - 🧠 How Claude learns from your conversations automatically
• docs/REAL_TIME_LEARNING_GUIDE.md - Real-time learning system usage guide
• docs/MEMORY_MCP_GUIDE.md - Persistent memory across sessions

Technical Documentation

• README_NATIVE_SETUP.md - Detailed native setup guide
• NATIVE_VS_DOCKER_DECISION.md - Why we chose native Ollama
• PERFORMANCE_TEST_RESULTS.md - Benchmark results
• SLUG_IMPLEMENTATION_SUMMARY.md - URL slug system
• MCP_KB_ENDPOINTS.md - MCP endpoint documentation

---

🤝 Contributing

This is a personal development tool. Feel free to:

• Modify for your specific needs
• Add new KB types
• Optimize RAG strategies
• Contribute improvements back

---

📄 License

MIT License - Use it freely!

---

🎯 Next Steps

1. First-time setup: Run ./setup_native.sh
2. Start services: Run ./start_all_services.sh
3. Create knowledge base: Visit http://localhost:5173
4. Upload documents: Add your codebase/docs
5. Integrate with Claude: Run claude mcp add ...
6. Start coding: Ask Claude about your project!

Happy coding! 🚀