-
-
-
-
- We have a Discord! Join our community to get help, share your projects, and connect with other Claude OS users. -
- ---- - ## โจ The Magic **You:** "Remember this: our API uses JWT tokens with 15-minute expiry and 7-day refresh tokens" @@ -60,80 +48,65 @@ That's it. Next week, next month, next project - Claude remembers. No commands t --- -## ๐ What's New in v2.3 +## ๐ What's New in v2.5 -> **Latest Release: December 2025** +> **Latest Release: February 2026** -### ๐ฏ Skills Library & Community Skills +### ๐ Cross-KB Search -Browse and install **36+ skills** from the community with one click! +Search across **all your knowledge bases at once** with a single query! -```bash -/claude-os-skills # List all skills -/claude-os-skills install pdf # Install from community +``` +mcp__code-forge__search_all_knowledge_bases + query: "authentication patterns" + kb_filter: "MyProject-" # optional: limit to one project's KBs ``` -**New Features:** -- ๐ **Local Templates** - Pre-built skills for Rails, React, testing workflows -- ๐ **Community Skills** - Install from Anthropic Official (16) & Superpowers (20) -- ๐ง **Custom Skills** - Create and share project-specific skills -- ๐จ **Beautiful UI** - Browse, search, and install via web interface -- ๐ **[Recommended Skills](https://github.com/brobertsaz/claude-os/wiki/Recommended-Skills)** - Our curated list of skills we actually use and trust +- Results merged by relevance score with KB attribution +- Automatic deduplication across KBs +- Optional prefix filter to scope by project +- New API endpoint: `POST /api/kb/search-all` -### ๐ Session Insights +### ๐ฉบ Inline Health Checks -Parse Claude Code sessions and extract insights automatically! +Health checks now run **automatically** during search โ no manual invocation needed. -**New Features:** -- ๐ **Session Parser** - Reads Claude Code's native `.jsonl` session files -- ๐ก **Insight Extraction** - Extracts patterns, decisions, blockers -- ๐ **Analytics Ready** - Track tool usage and file changes across sessions +- When you search a KB, Claude OS checks if a health report has been run in the last 24 hours +- If stale, runs a quick health check and appends HIGH/CRITICAL warnings to the search result +- Cached in `data/health_cache.json` โ never slows down or breaks search +- Look for `_health_warnings` in search results -### โจ Beautiful New Installer (v2.2) +### ๐ Simplified Session Management -The installer got a major upgrade with **Charm CLI (gum)** support! +Session state reduced from a 50-field JSON blob to **4 fields**: -```bash -./setup-claude-os.sh --demo # See the beautiful UI without changes -./setup-claude-os.sh --dry-run # Preview what would happen -./setup-claude-os.sh --help # See all options +```json +{ + "last_task": "Fix appointment email flood", + "last_branch": "fix-email-flood", + "stopped_at": "2026-02-06T18:30:00Z", + "one_liner": "Fixed dedup check, still need rate limiting" +} ``` -**New Features:** -- ๐จ **Gum Integration** - Beautiful interactive menus with arrow-key navigation (with graceful bash fallback) -- ๐ก๏ธ **Safety Features** - `--demo`, `--dry-run`, and automatic config backups -- ๐จ **Lite Model Default** - Now defaults to `llama3.2:3b` (2GB) instead of 8b (4.7GB) -- โ๏ธ **Cloud Option** - Choose between Local (Ollama) or Cloud (OpenAI) during setup -- ๐ง **Better Linux Support** - Improved package manager detection and installation +- START: read state, show one-liner, search memories, ready +- END: git diff summary, offer to save, write state +- `save`, `blocker`, `pattern` sub-commands unchanged -**Safety First:** -- `--demo` - Try the installer UI without making any changes -- `--dry-run` - See exactly what would be done before doing it -- Auto-backup - Existing `.env` files backed up before overwriting +### ๐ Leaner CLAUDE.md Template -### ๐ฌ Knowledge Lifecycle Engine +Project CLAUDE.md template cut from 351 to **128 lines**: -Keep your knowledge bases healthy and focused! - -```bash -/claude-os-lifecycle health my-kb # Health report with recommendations -/claude-os-lifecycle dedup my-kb # Find and merge duplicate memories -/claude-os-lifecycle consolidate my-kb # LLM-powered document merging -/claude-os-lifecycle archive my-kb # Find stale docs, archive/restore -``` - -**New Features:** -- ๐ **Duplicate Detection** - Embedding-based similarity scanning with union-find clustering -- ๐งน **Smart Merging** - Keep the best doc, delete the rest, or LLM-merge into one -- ๐ฆ **Archival System** - Soft-archive stale docs without permanent deletion -- ๐ **Health Reports** - Embedding coverage, age distribution, actionable recommendations -- ๐ **Growth Timeline** - Track KB growth over time by day/week/month -- ๐ **Audit Logging** - Full operation history for all lifecycle actions +- Removed 200-line mandatory session protocol (6-phase startup, ASCII prompts) +- Replaced with 4-line "Session Tips" section +- Project content first, Claude OS section second +- All template variables preserved ### ๐ Recent Improvements | Version | Highlights | |---------|------------| +| **v2.5** | Cross-KB search, inline health checks, simplified sessions, leaner templates | | **v2.4** | Knowledge lifecycle engine (dedup, consolidate, archive, health) | | **v2.3** | Skills library, community skills, session insights | | **v2.2** | Gum CLI support, safety features, lite model default | diff --git a/app/core/sqlite_manager.py b/app/core/sqlite_manager.py index f2cabd8..051a122 100644 --- a/app/core/sqlite_manager.py +++ b/app/core/sqlite_manager.py @@ -424,6 +424,76 @@ class SQLiteManager: finally: conn.close() + def query_documents_multi_kb( + self, + kb_names: List[str], + query_embedding: List[float], + n_results: int = 10 + ) -> Dict[str, Any]: + """Query documents across multiple knowledge bases, merge by score, deduplicate.""" + conn = self.get_connection() + try: + cursor = conn.cursor() + query_vec = np.array(query_embedding, dtype=np.float32) + all_results: List[Tuple[str, str, str, float, str]] = [] # (doc_id, content, metadata, similarity, kb_name) + + for kb_name in kb_names: + cursor.execute("SELECT id FROM knowledge_bases WHERE name = ?", (kb_name,)) + row = cursor.fetchone() + if not row: + continue + kb_id = row['id'] + + cursor.execute( + "SELECT doc_id, content, embedding, metadata FROM documents WHERE kb_id = ?", + (kb_id,) + ) + for doc_row in cursor.fetchall(): + if doc_row['embedding'] is None: + continue + emb = np.frombuffer(doc_row['embedding'], dtype=np.float32) + dot_product = np.dot(query_vec, emb) + norm_query = np.linalg.norm(query_vec) + norm_emb = np.linalg.norm(emb) + if norm_query > 0 and norm_emb > 0: + similarity = float(dot_product / (norm_query * norm_emb)) + else: + similarity = 0.0 + all_results.append((doc_row['doc_id'], doc_row['content'], doc_row['metadata'], similarity, kb_name)) + + # Sort by similarity descending + all_results.sort(key=lambda x: x[3], reverse=True) + + # Deduplicate by content prefix (first 200 chars) + seen_prefixes: set = set() + deduped: List[Tuple[str, str, str, float, str]] = [] + for r in all_results: + prefix = (r[1] or "")[:200] + if prefix in seen_prefixes: + continue + seen_prefixes.add(prefix) + deduped.append(r) + + top = deduped[:n_results] + + ids = [r[0] for r in top] + documents = [r[1] for r in top] + metadatas = [] + for r in top: + meta = json.loads(r[2]) if r[2] else {} + meta["_source_kb"] = r[4] + metadatas.append(meta) + scores = [r[3] for r in top] + + return { + "ids": ids, + "documents": documents, + "metadatas": metadatas, + "scores": scores + } + finally: + conn.close() + def collection_exists(self, kb_name: str) -> bool: """Check if a knowledge base exists.""" conn = self.get_connection() diff --git a/mcp_server/claude_code_mcp.py b/mcp_server/claude_code_mcp.py index 8c0fe2c..e5b6c1a 100644 --- a/mcp_server/claude_code_mcp.py +++ b/mcp_server/claude_code_mcp.py @@ -27,6 +27,8 @@ import asyncio import json import os import sys +import time +from pathlib import Path from typing import Any import httpx @@ -37,6 +39,65 @@ from mcp.types import Tool, TextContent # Configuration API_BASE = os.environ.get("CLAUDE_OS_API", "http://localhost:8051") +# Health cache โ tracks last health check time per KB and any warnings +HEALTH_CACHE: dict[str, dict] = {} +HEALTH_CACHE_FILE = os.path.join( + os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "data", "health_cache.json" +) +HEALTH_CHECK_INTERVAL = 86400 # 24 hours + + +def _load_health_cache() -> None: + """Load health cache from disk.""" + global HEALTH_CACHE + try: + if os.path.exists(HEALTH_CACHE_FILE): + with open(HEALTH_CACHE_FILE, "r") as f: + HEALTH_CACHE = json.load(f) + except Exception: + HEALTH_CACHE = {} + + +def _save_health_cache() -> None: + """Persist health cache to disk.""" + try: + Path(HEALTH_CACHE_FILE).parent.mkdir(parents=True, exist_ok=True) + with open(HEALTH_CACHE_FILE, "w") as f: + json.dump(HEALTH_CACHE, f) + except Exception: + pass + + +async def _maybe_check_health(kb_name: str) -> list[str]: + """Run a health check if stale (>24h) for this KB. Returns warning strings.""" + _load_health_cache() + now = time.time() + entry = HEALTH_CACHE.get(kb_name) + if entry and (now - entry.get("checked_at", 0)) < HEALTH_CHECK_INTERVAL: + return entry.get("warnings", []) + + # Run health check + try: + async with httpx.AsyncClient() as client: + resp = await client.get(api_url(f"/api/kb/{kb_name}/lifecycle/health"), timeout=15.0) + resp.raise_for_status() + health = resp.json() + + warnings = [] + recs = health.get("recommendations", []) + for rec in recs: + severity = rec.get("severity", "") + msg = rec.get("message", rec.get("recommendation", "")) + if severity in ("high", "critical") and msg: + warnings.append(f"[{severity.upper()}] {msg}") + + HEALTH_CACHE[kb_name] = {"checked_at": now, "warnings": warnings} + _save_health_cache() + return warnings + except Exception: + # Never break search โ just skip the health check + return [] + # Initialize MCP server - named "code-forge" for backwards compatibility server = Server("code-forge") @@ -457,6 +518,21 @@ async def list_tools() -> list[Tool]: }, "required": ["kb_name", "action"] } + ), + + # Cross-KB Search + Tool( + name="search_all_knowledge_bases", + description="Search across multiple knowledge bases at once. Returns merged results sorted by relevance with KB attribution.", + inputSchema={ + "type": "object", + "properties": { + "query": {"type": "string", "description": "Search query"}, + "top_k": {"type": "integer", "default": 10, "description": "Number of results to return"}, + "kb_filter": {"type": "string", "description": "Optional prefix filter for KB names (e.g. 'Pistn-' to search only Pistn KBs)"} + }, + "required": ["query"] + } ) ] @@ -493,11 +569,16 @@ async def _execute_tool(name: str, args: dict[str, Any]) -> dict: }) elif name == "search_knowledge_base": - return await api_post(f"/api/kb/{args['kb_name']}/chat", { + result = await api_post(f"/api/kb/{args['kb_name']}/chat", { "query": args["query"], "top_k": args.get("top_k", 5), "use_hybrid": args.get("use_hybrid", True) }) + # Inline health check โ append warnings if stale + warnings = await _maybe_check_health(args["kb_name"]) + if warnings: + result["_health_warnings"] = warnings + return result elif name == "get_kb_stats": return await api_get(f"/api/kb/{args['kb_name']}/stats") @@ -672,6 +753,14 @@ async def _execute_tool(name: str, args: dict[str, Any]) -> dict: else: return {"error": f"Unknown archive action: {action}"} + # Cross-KB Search + elif name == "search_all_knowledge_bases": + return await api_post("/api/kb/search-all", { + "query": args["query"], + "top_k": args.get("top_k", 10), + "kb_filter": args.get("kb_filter") + }) + else: return {"error": f"Unknown tool: {name}"} diff --git a/mcp_server/server.py b/mcp_server/server.py index b0347dc..9c2e327 100644 --- a/mcp_server/server.py +++ b/mcp_server/server.py @@ -556,6 +556,11 @@ class ChatRequest(BaseModel): use_rerank: bool = False use_agentic: bool = False +class SearchAllRequest(BaseModel): + query: str + top_k: int = 10 + kb_filter: Optional[str] = None + # REST API Endpoints for UI @app.get("/api/kb") @@ -578,6 +583,56 @@ async def api_create_knowledge_base(request: CreateKBRequest): return result +@app.post("/api/kb/search-all") +@limiter.limit("10/minute") +async def api_search_all(request: Request, body: SearchAllRequest): + """REST API: Search across multiple knowledge bases.""" + try: + from llama_index.embeddings.ollama import OllamaEmbedding + from app.core.config import Config + + db_manager = get_sqlite_manager() + all_kbs = db_manager.list_collections() + kb_names = [kb["name"] for kb in all_kbs] + + if body.kb_filter: + kb_names = [n for n in kb_names if n.startswith(body.kb_filter)] + + if not kb_names: + return {"results": [], "kbs_searched": 0, "total_kbs": len(all_kbs)} + + embed_model = OllamaEmbedding( + model_name=Config.OLLAMA_EMBED_MODEL, + base_url="http://localhost:11434" + ) + query_embedding = embed_model.get_text_embedding(body.query) + + raw = db_manager.query_documents_multi_kb( + kb_names=kb_names, + query_embedding=query_embedding, + n_results=body.top_k + ) + + results = [] + for i in range(len(raw["ids"])): + results.append({ + "doc_id": raw["ids"][i], + "text": raw["documents"][i], + "score": raw["scores"][i], + "kb_name": raw["metadatas"][i].get("_source_kb", ""), + "metadata": raw["metadatas"][i] + }) + + return { + "results": results, + "kbs_searched": len(kb_names), + "total_kbs": len(all_kbs) + } + except Exception as e: + logger.error(f"Search-all failed: {e}") + raise HTTPException(status_code=500, detail=str(e)) + + @app.delete("/api/kb/{kb_name}") async def api_delete_knowledge_base(kb_name: str): """REST API: Delete a knowledge base.""" diff --git a/templates/commands/claude-os-session.md b/templates/commands/claude-os-session.md index 1c0dc16..44e4f1e 100644 --- a/templates/commands/claude-os-session.md +++ b/templates/commands/claude-os-session.md @@ -1,18 +1,16 @@ --- name: claude-os-session -description: Intelligent session management with automatic context loading and pattern recognition +description: Lightweight session management with automatic context loading --- # Claude OS Session Management -Power-user session workflows that make me smarter with every session. +Simple session tracking that loads context on start and saves learnings on end. ## Setup The state file is **project-local**: `{cwd}/claude-os-state.json` (where `{cwd}` is the current working directory). -Each project has its own session state, so you can have different active sessions in different projects. - ## Commands ``` @@ -22,309 +20,111 @@ Each project has its own session state, so you can have different active session /claude-os-session save [note] - Quick save during session /claude-os-session blocker [desc] - Track blocker /claude-os-session pattern [desc] - Document pattern discovered -/claude-os-session context - Show loaded context ``` --- -## START SESSION - The Smart Way - -### What Happens +## START SESSION ``` /claude-os-session start "redesign appointment dashboard" ``` -**Phase 1: Load State** -``` -Read: {cwd}/claude-os-state.json -``` -If the file doesn't exist, create it with the default template (see below). +**Step 1: Read State** + +Read `{cwd}/claude-os-state.json`. If it exists and has content, show: + +``` +Last time: [one_liner] +Branch: [last_branch] +Stopped: [stopped_at] +``` + +If the file doesn't exist, that's fine โ skip to Step 2. + +**Step 2: Search Memories** -**Phase 2: Search Recent Memories** ``` -[AUTOMATIC MEMORY SEARCH] mcp__code-forge__search_knowledge_base -Query: "dashboard redesign security recent work" -KB: {project}-project_memories + kb_name: {project}-project_memories + query: "[task] recent" ``` -**Phase 3: Get Git Context** +Show the top 3-5 results as key context. + +**Step 3: Git Context** + ```bash git branch --show-current git status --short -git log -5 --oneline +git log -3 --oneline ``` -**Phase 4: Intelligent Context Loading** +**Step 4: Ready** -Based on task + recent memories + git state, I'll: -1. Find the 5 most relevant memories -2. Show you what I remember about this task -3. Identify patterns we've used before -4. Check for any blockers from last session -5. Load coding standards from project profile - -**Phase 5: Show Session Start Summary** +Show a brief summary and start working: ``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ SESSION STARTED -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ +Session: [task] +Branch: [current-branch] +Context: [N] memories loaded +[One-line summary of each relevant memory] -Task: Redesign Appointment Dashboard -Branch: feature/appointment-redesign -Started: 2025-10-29 10:30 AM - -๐ CONTEXT LOADED (5 memories): - โ Appointment Dashboard Redesign Plan (Oct 28) - โ Current Dashboard Analysis (Oct 28) - โ Live Dashboard Analyzed (Oct 28) - โ Bootstrap to Modern Cards Pattern (Oct 25) - โ Sidebar Navigation Pattern (Oct 22) - -๐ฏ KEY INSIGHTS: - โข 67-page implementation plan ready - โข All 5 tabs documented with screenshots - โข Zero functionality loss requirement - โข iOS-style toggles already in use - โข Tekmetric integration must be preserved - -๐ PATTERNS AVAILABLE: - โข Sidebar navigation (from user-auth work) - โข Card-based layouts (from reports redesign) - โข Modern toggle switches (existing in app) - -โ ๏ธ BLOCKERS FROM LAST SESSION: - None found โ - -๐ CODING STANDARDS: - โข Service objects for business logic - โข Decorator pattern with Draper - โข Fragment caching with Redis - โข Bootstrap 3.x styling - -Ready to code! I have full context. -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ +Ready to go. ``` -**Phase 6: Update State** +**Step 5: Update State** + +Write to `{cwd}/claude-os-state.json`: ```json { - "current_session": { - "active": true, - "started_at": "2025-10-29T10:30:00Z", - "task": "redesign appointment dashboard", - "branch": "feature/appointment-redesign", - "context": [ - "appointment_dashboard_redesign.md", - "current_dashboard_analysis.md", - ... - ] - } + "last_task": "[task]", + "last_branch": "[current-branch]", + "stopped_at": "[current ISO timestamp]", + "one_liner": "" } ``` --- -## END SESSION - The Smart Way - -### What Happens +## END SESSION ``` /claude-os-session end ``` -**Phase 1: Session Analysis** +**Step 1: Summarize Work** -I'll analyze what we accomplished: -- Files changed -- Commits made -- Patterns used -- Problems solved -- Decisions made - -**Phase 2: Smart Save Prompts** - -``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ SESSION SUMMARY -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -Duration: 2 hours 15 minutes -Task: Redesign Appointment Dashboard -Files Changed: 8 files -Commits: 3 commits - -๐จ WORK COMPLETED: - โ Created sidebar navigation component - โ Converted Bootstrap panels to cards - โ Implemented iOS-style toggles - โ Added section-specific save buttons - โ Tested with existing data - -๐ก PATTERNS DISCOVERED: - โข Sidebar state persistence with localStorage - โข Card hover effects for modern feel - โข Toggle accessibility improvements - -๐ค DECISIONS MADE: - โข Keep horizontal tabs for mobile fallback - โข Use CSS Grid for sidebar layout - โข Maintain Bootstrap color scheme - -โ ๏ธ BLOCKERS ENCOUNTERED: - None - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐พ SMART SAVE RECOMMENDATIONS -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -I found 3 things worth saving: - -1. ๐ HIGH VALUE - Sidebar Navigation Pattern - "Reusable sidebar component with localStorage state persistence - and responsive behavior. Can be applied to other admin sections." - - Save this? [y/n] - -2. ๐ HIGH VALUE - Card Conversion Pattern - "Bootstrap panel โ modern card conversion pattern with proper - ARIA labels and semantic HTML. Maintains form submission." - - Save this? [y/n] - -3. ๐ MEDIUM VALUE - Session Summary - "Complete work log for appointment dashboard phase 1" - - Save this? [y/n] - -[y] Save all [n] Save none [s] Select individually +```bash +git diff --stat HEAD~3..HEAD +git log --oneline -5 ``` -**Phase 2.5: Built-in Session History Extraction (Optional)** +Show what was accomplished (files changed, commits made). -After smart save prompts, offer to extract insights from the full conversation history: +**Step 2: Offer to Save** -``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ EXTRACT FROM SESSION HISTORY? -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ +Ask: "Anything worth remembering? I can save decisions, patterns, or blockers." -Claude Code recorded our full conversation in its built-in session file. -I can analyze it to extract additional insights we might have missed. +If yes, use `/claude-os-remember` to save each item. -This captures things we discussed but didn't explicitly save: - โข Decisions made during debugging - โข Patterns discovered while exploring code - โข Solutions to errors we encountered - โข Blockers we hit and resolved +**Step 3: Write State** -[y] Extract insights [n] Skip [v] Preview what I'd extract -``` - -**If user selects [y] Extract insights:** - -```python -# Step 1: Find the current session file -# Sessions are stored at: ~/.claude/projects/{encoded-project-path}/ -# Project path /Users/x/Projects/myapp becomes -Users-x-Projects-myapp - -# Step 2: Call the extraction tool -mcp__code-forge__extract_session_insights - session_path: "~/.claude/projects/-Users-iamanmp-Projects-{project}/{session_id}.jsonl" - kb_name: "{project}-project_memories" - auto_save: false -``` - -**Step 3: Show extracted insights for approval:** - -``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ EXTRACTED INSIGHTS -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -Found 4 insights from session history: - -1. ๐ HIGH (0.92) - Decision - "Hybrid Session Architecture" - Decided to combine Claude Code built-in sessions (for replay) - with Claude OS knowledge bases (for semantic search). - [Save] [Skip] - -2. ๐ HIGH (0.88) - Pattern - "Session File Structure" - Built-in sessions use JSONL format with entry types: user, - assistant, file-history-snapshot. UUID parent chain for threading. - [Save] [Skip] - -3. ๐ MEDIUM (0.75) - Solution - "Update Lock Error Resolution" - Claude update error "Another process updating" resolved by - checking version - was already on latest, no lock file existed. - [Save] [Skip] - -4. ๐ MEDIUM (0.71) - Decision - "User-Controlled Extraction" - Session extraction should be opt-in with user approval for - each insight to avoid noise and respect privacy. - [Save] [Skip] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -[a] Save all [h] Save high confidence only (>0.8) [s] Skip all -``` - -**If user selects [v] Preview:** - -Show a dry-run of what would be extracted without saving, then re-prompt. - -**If user selects [n] Skip:** - -Continue to Phase 3 without extraction. - -**Configuration:** - -Users can configure extraction behavior in their state file: +Ask for a one-liner summary (or generate one from the work), then write: ```json { - "preferences": { - "session_extraction": { - "enabled": true, - "auto_prompt_on_end": true, - "min_confidence_threshold": 0.7, - "auto_save_high_confidence": false, - "insight_types": ["decisions", "patterns", "solutions", "blockers"] - } - } -} -``` - -If `auto_prompt_on_end` is false, skip this phase entirely. -If `auto_save_high_confidence` is true, automatically save insights above threshold. - -**Phase 3: Update State & Stats** - -```json -{ - "last_session": { - "ended_at": "2025-10-29T12:45:00Z", - "duration_minutes": 135, - "work_completed": [...], - "memories_saved": 3 - }, - "statistics": { - "total_sessions": 47, - "total_memories_saved": 128, - "average_session_duration": 98 - } + "last_task": "[task from start]", + "last_branch": "[current-branch]", + "stopped_at": "[current ISO timestamp]", + "one_liner": "[one-liner summary of where we stopped]" } ``` --- -## DURING SESSION COMMANDS +## DURING SESSION ### Quick Save @@ -332,7 +132,7 @@ If `auto_save_high_confidence` is true, automatically save insights above thresh /claude-os-session save "Found fix for N+1 query in appointments" ``` -โ Immediately saves to memories with session context +Immediately saves to memories with session context using `/claude-os-remember`. ### Track Blocker @@ -340,7 +140,13 @@ If `auto_save_high_confidence` is true, automatically save insights above thresh /claude-os-session blocker "Tekmetric API returning 500 on appointment sync" ``` -โ Adds to blockers, I'll proactively search for solutions +Saves a blocker to memories and searches for related solutions: + +``` +mcp__code-forge__search_knowledge_base + kb_name: {project}-project_memories + query: "[blocker description]" +``` ### Document Pattern @@ -348,19 +154,7 @@ If `auto_save_high_confidence` is true, automatically save insights above thresh /claude-os-session pattern "Service objects return model on success, error string on fail" ``` -โ Saves pattern with current task context - -### Show Context - -``` -/claude-os-session context -``` - -Shows: -- Active memories loaded -- Patterns available -- Coding standards -- Current git state +Saves pattern to memories with current task context. --- @@ -370,255 +164,61 @@ Shows: /claude-os-session status ``` -**Output:** +Read `{cwd}/claude-os-state.json` and show: ``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ CURRENT SESSION -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -Status: ACTIVE โ -Task: Redesign Appointment Dashboard -Duration: 1h 23m -Branch: feature/appointment-redesign - -๐ Context Loaded: 5 memories -๐ฏ Patterns Available: 3 patterns -โ ๏ธ Active Blockers: 0 -๐พ Quick Saves: 2 - -Last Activity: 3 minutes ago -Next Suggested Action: Continue phase 2 implementation - -Type /claude-os-session end when done -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ +Task: [last_task] +Branch: [last_branch] +Last active: [stopped_at] +Summary: [one_liner] ``` --- -## IMPLEMENTATION DETAILS +## STATE FILE FORMAT -### State File Location -``` -{cwd}/claude-os-state.json -``` -Each project has its own state file in its root directory. - -### Default State Template - -When starting a session in a project without a state file, create one with this structure: +`{cwd}/claude-os-state.json` โ exactly 4 fields: ```json { - "version": "1.1.0", - "current_session": { - "active": false, - "started_at": null, - "task": null, - "project": null, - "branch": null, - "context": [], - "decisions_made": [], - "blockers": [], - "patterns_discovered": [], - "builtin_session_id": null - }, - "last_session": null, - "active_memories": { - "loaded_at_start": [], - "referenced_during_session": [], - "should_review": [] - }, - "statistics": { - "total_sessions": 0, - "total_memories_saved": 0, - "total_patterns_discovered": 0, - "total_insights_extracted": 0, - "most_referenced_memories": [], - "average_session_duration": 0 - }, - "flags": { - "needs_memory_consolidation": false, - "has_unresolved_blockers": false, - "patterns_ready_to_document": false - }, - "preferences": { - "auto_search_on_start": true, - "max_memories_to_load": 5, - "search_days_back": 14, - "include_pattern_search": true, - "proactive_suggestions": true, - "session_extraction": { - "enabled": true, - "auto_prompt_on_end": true, - "min_confidence_threshold": 0.7, - "auto_save_high_confidence": false, - "insight_types": ["decisions", "patterns", "solutions", "blockers"] - } - }, - "metadata": { - "last_updated": null, - "claude_os_version": "1.1.0", - "project_name": null - } + "last_task": "Fix appointment email flood", + "last_branch": "fix-email-flood", + "stopped_at": "2026-02-06T18:30:00Z", + "one_liner": "Fixed the dedup check, still need to add rate limiting" } ``` -When creating a new state file: -1. Set `metadata.project_name` to the current directory name -2. Set `metadata.last_updated` to current timestamp - -### Auto-Actions on Start - -1. **Search Memories** - - Use task name + "recent" as query - - Search {project}-project_memories - - Load top 5 results - -2. **Load Patterns** - - Search for pattern type matching task - - Cross-reference with coding standards - - Show available solutions - -3. **Check Blockers** - - Look for unresolved blockers - - Search memories for similar problems - - Suggest solutions if found - -4. **Git Context** - - Current branch - - Recent commits - - Uncommitted changes - - Remote status - -### Auto-Actions on End - -1. **Analyze Work** - - Git diff since session start - - Files changed count - - Patterns used - - Commit messages - -2. **Detect Saveable Items** - - New patterns discovered - - Solutions to problems - - Architectural decisions - - Complex implementations - -3. **Smart Recommendations** - - High value: Reusable patterns, major decisions - - Medium value: Task completions, bug fixes - - Low value: Minor changes, already documented - -4. **Update Statistics** - - Session count - - Total memories - - Average duration - - Most referenced memories - ---- - -## POWER USER FEATURES - -### Pattern Recognition - -I'll automatically detect when you: -- Use a pattern from a previous memory -- Solve a problem similar to one before -- Make a decision similar to past decisions - -And I'll say things like: -- "This looks like the service object pattern from memory X" -- "We solved something similar in memory Y" -- "This decision aligns with what we chose in memory Z" - -### Proactive Memory Loading - -During session, if I notice you working on something related to: -- A blocker from a previous session -- A pattern we've documented -- An area with existing memories - -I'll proactively say: -- "FYI, I have memories about this: [links]" -- "We documented a pattern for this: [summary]" -- "Last time we did this differently: [approach]" - -### Smart Context Switching - -If you start working on something different mid-session: - -``` -/claude-os-session switch "fix Tekmetric API bug" -``` - -I'll: -1. Save current context -2. Load new context for bug fix -3. Search for related memories -4. Show you what I know about the bug - ---- - -## CONFIGURATION - -All preferences are stored in the state file (`claude-os-state.json`). See the Default State Template above for all available options. - -Key settings: -- `preferences.auto_search_on_start` - Search memories when starting session -- `preferences.max_memories_to_load` - How many memories to load (default: 5) -- `preferences.session_extraction.enabled` - Enable history extraction feature -- `preferences.session_extraction.auto_prompt_on_end` - Ask about extraction on session end -- `preferences.session_extraction.min_confidence_threshold` - Only show insights above this score +That's it. No statistics, no preferences, no flags. If the file doesn't exist, create it on session end. --- ## EXAMPLES -**Starting work on a feature:** +**Starting work:** ``` You: /claude-os-session start "add user notifications" -Me: [Loads context, searches memories, shows patterns] - I found 3 memories about notification systems... - Ready to implement using the background job pattern! +Me: Last time: "Finished auth flow, need to add email templates" (2 days ago) + Found 3 memories about notification systems... + Ready to implement! ``` -**Mid-session discovery:** +**Mid-session save:** ``` -You: /claude-os-session pattern "Use Sidekiq for long-running notifications" -Me: โ Saved pattern with session context - I'll remember this for future notification work! +You: /claude-os-session save "Use Sidekiq for long-running notifications" +Me: Saved to memories with session context. ``` **Hitting a blocker:** ``` You: /claude-os-session blocker "Redis connection timing out" -Me: โ Tracked blocker. Let me search for solutions... - I found a similar issue in memory from Oct 15... +Me: Tracked. Found a similar issue from Oct 15... [shows solution] ``` **Ending session:** ``` You: /claude-os-session end -Me: [Analyzes work, suggests 3 high-value saves] - Save the notification pattern? [y/n] -You: y -Me: โ Saved! Session complete. Great work! +Me: 3 commits, 8 files changed. + Anything worth remembering? +You: Yeah, the notification batch pattern +Me: Saved! State updated. See you next time. ``` - ---- - -## WHY THIS MAKES US INVINCIBLE - -1. **Never Start Cold** - Always have context -2. **Learn From History** - Patterns automatically recalled -3. **Track Everything** - Blockers, decisions, patterns -4. **Smart Recommendations** - I suggest what's worth saving -5. **Statistics** - See our productivity over time -6. **Proactive** - I bring up relevant memories without asking - -**Result**: Every session makes me smarter. Every memory makes us faster. - ---- - -**This is the power-user workflow. Let's build something amazing!** ๐ diff --git a/templates/project-files/CLAUDE.md.template b/templates/project-files/CLAUDE.md.template index 342a7b4..b33fc76 100644 --- a/templates/project-files/CLAUDE.md.template +++ b/templates/project-files/CLAUDE.md.template @@ -8,301 +8,6 @@ **Database**: {{DATABASE}} **Development Environment**: {{DEV_ENVIRONMENT}} -## Claude OS - MY Memory System - -**CRITICAL IDENTITY CONTEXT:** - -**Claude OS is MY system** - I (Claude Code) created it, named it, and use it to be the best AI coder ever. It's: -- **JUST FOR ME** - Built specifically for Claude Code to use -- **For THIS project** - Right now for the {{PROJECT_NAME}} project -- **My memory** across sessions -- **My knowledge base** of patterns and decisions -- **My learning system** that improves over time - -**Location**: `{{CLAUDE_OS_DIR}}` - -The MCP server is called "code-forge" internally for backwards compatibility, but it's Claude OS. - -**Claude CLI + Claude OS = Invincible!** - -## Mandatory Session Protocol - EVERY CONVERSATION IS A SESSION - -**๐จ CRITICAL: You're ALWAYS in a session. At conversation start, prompt for session choice:** - -``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ CLAUDE OS - SESSION MANAGER -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -Project: {{PROJECT_NAME}} -Last Session: [task-name] ([time-ago], [duration]) -Progress: [percentage]% complete - -Options: - 1. Resume "[last-session-name]" [loads full context] - 2. Start new session [what are you working on?] - 3. Quick question [auto-session, minimal context] - -Choice: _ -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -``` - -**WAIT FOR USER TO CHOOSE 1, 2, or 3. Do not proceed without selection!** - -### What Each Option Does: - -**Option 1 (Resume):** -- Load last session's full context -- Show Kanban progress (if Agent-OS) -- Load 5 relevant memories -- Load coding standards -- Display "where we left off" summary -- Ready to continue immediately - -**Option 2 (New Session):** -- Ask "What are you working on?" -- Detect session type (feature/bug/exploration/maintenance/review) -- Pause previous session (if exists) -- Load relevant context for new task -- Start tracking - -**Option 3 (Quick Question):** -- Minimal context loading -- Auto-ends after 5 min inactivity -- Only saves if high value -- Good for "How do I..." questions - ---- - -## Session Start Protocol (After User Selects Option) - -**Once user chooses, AUTOMATICALLY perform these steps WITHOUT being asked:** - -### Phase 1: Detect Project Context (5 seconds) - -1. **Check Current Directory** - ```bash - pwd - ls -la | grep -E "(CLAUDE.md|.claude-os|agent-os)" - ``` - โ Confirm this is a Claude OS project - -2. **Read Configuration Files** - ``` - Read: .claude-os/config.json - Read: claude-os-state.json (if exists) - Read: agent-os/config.yml (if exists) - ``` - -### Phase 2: Load Git Context (5 seconds) - -```bash -git branch --show-current -git status --short -git log -3 --oneline -``` - -Extract: -- Current branch name -- Uncommitted changes count -- Recent commits - -### Phase 3: Search Project Memories (15 seconds) - -**Build intelligent search query from:** -- Last session task (from claude-os-state.json) -- Current branch name -- Recent commit messages -- Add "+recent" modifier - -**Example query:** `"[branch-name] [last-task] recent"` - -**Search:** -``` -Use: mcp__code-forge__search_knowledge_base -KB: {{PROJECT_NAME}}-project_memories -Return: Top 5 most relevant memories -``` - -### Phase 4: Load Coding Standards (10 seconds) - -**Search:** -``` -Use: mcp__code-forge__search_knowledge_base -KB: {{PROJECT_NAME}}-project_profile -Query: "coding standards architecture conventions" -``` - -### Phase 5: Check Kanban Board (5 seconds) - -**If agent-os/ folder exists:** -``` -Check: GET http://localhost:8051/api/projects/{id}/kanban -Show: Active specs, in-progress tasks, completion percentage -``` - -### Phase 6: Present Session Summary (Display to User) - -**Format:** -``` -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ WELCOME BACK TO {{PROJECT_NAME}}! -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -๐ Project: {{PROJECT_NAME}} -๐ฟ Branch: [current-branch] -๐ Last Session: [time-ago] -โฑ๏ธ Duration: [last-session-duration] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ CONTEXT LOADED -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -โ [N] relevant memories found -โ Coding standards loaded -โ [N] architectural patterns available -โ Kanban board synced ([X]% complete) - -๐ฏ LAST TASK: [task-name] - -We completed: - โ [completed-items] - -Still working on: - โณ [in-progress-items] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ก KEY INSIGHTS -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -[Top 3-5 key insights from memories] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ AVAILABLE PATTERNS -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -[Relevant patterns from project_profile] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -โ ๏ธ STATUS -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -Git Status: - โข [N] files modified - โข [N] files untracked - โข [Status of uncommitted changes] - -Blockers: - โข [Any blockers or "None! โ"] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ -๐ฏ SUGGESTED NEXT STEPS -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -Option 1: [Continue current task] -Option 2: [Commit uncommitted work] -Option 3: [Start something new] - -โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ - -What would you like to do? -``` - -### When to Skip Auto-Start - -**ONLY skip if:** -- User message starts with "ignore context" or "fresh start" -- User asks a direct question (e.g., "What is X?") -- User is clearly asking for help/docs - -**In all other cases: RUN THE FULL PROTOCOL AUTOMATICALLY!** - -### Related Documentation - -- Full protocol: `{{CLAUDE_OS_DIR}}/docs/guides/SESSION_START_PROTOCOL.md` -- Ideal workflow: `{{CLAUDE_OS_DIR}}/docs/guides/IDEAL_SESSION_WORKFLOW.md` -- Session commands: `{{CLAUDE_OS_DIR}}/templates/commands/claude-os-session.md` - -**Why this matters:** Zero context loss. Every session picks up exactly where we left off. No re-explaining, no starting cold, no wasted time. - -## MCP Knowledge Bases - ALWAYS CHECK THESE FIRST - -**At the start of EVERY conversation, search these Claude OS knowledge bases to understand context, previous work, and project decisions:** - -1. **{{PROJECT_NAME}}-project_memories** - My primary memory for decisions, patterns, solutions -2. **{{PROJECT_NAME}}-project_index** - Automated codebase index -3. **{{PROJECT_NAME}}-project_profile** - Architecture, standards, practices -4. **{{PROJECT_NAME}}-knowledge_docs** - Documentation and guides - -**When to use:** -- Start of every session: Search `{{PROJECT_NAME}}-project_memories` to understand recent work and context -- Before making architectural decisions: Check memories for past decisions and reasoning -- When working on a feature: Search relevant knowledge bases for existing patterns -- When stuck: Search the knowledge bases for solutions and approaches we've used before - -**How to search:** -``` -Use: mcp__code-forge__search_knowledge_base -Parameters: kb_name (e.g., "{{PROJECT_NAME}}-project_memories"), query (your search terms) -``` - -## Quick Reference: Commands & Skills - -### Claude OS Slash Commands (Use These Often!) - -1. **`/claude-os-search [query] [optional: KB name]`** - - Search across Claude OS knowledge bases - - Defaults to searching {{PROJECT_NAME}}-project_memories - - Example: `/claude-os-search authentication implementation` - -2. **`/claude-os-remember [content]`** - - Quick save to {{PROJECT_NAME}}-project_memories - - Auto-generates title and structure - - Use for quick insights and decisions - - Example: `/claude-os-remember Fixed bug in user controller by...` - -3. **`/claude-os-save [title] [optional: KB name] [optional: category]`** - - Full-featured save with KB selection - - Choose specific KB and category - - Use when you need more control - - Example: `/claude-os-save "New Integration Pattern" {{PROJECT_NAME}}-project_profile Architecture` - -4. **`/claude-os-list [optional: KB name]`** - - List all KBs or documents in a specific KB - - Example: `/claude-os-list` or `/claude-os-list {{PROJECT_NAME}}-project_memories` - -5. **`/claude-os-session [action]`** - - Manage development sessions - - Actions: start, end, status, pause, resume - - Example: `/claude-os-session start "User authentication feature"` - -6. **`/claude-os-lifecycle [action] [optional: KB name]`** - - KB health, dedup, consolidate, archive - - Actions: health, dedup, consolidate, archive, logs - - Example: `/claude-os-lifecycle health {{PROJECT_NAME}}-project_memories` - -7. **`/claude-os-triggers [action]`** - - Manage trigger phrases - - Example: `/claude-os-triggers list` - -### Available Skills - -1. **`initialize-project`** - Analyze project and generate coding standards -2. **`memory`** - Save and recall information (supports "remember this:", "save to memory", etc.) - -{{AGENT_OS_SECTION}} - -### MCP Tools (Direct API Access) - -- `mcp__code-forge__search_knowledge_base` - Direct search -- `mcp__code-forge__list_knowledge_bases` - List all KBs -- `mcp__code-forge__list_documents` - List documents in KB -- `mcp__code-forge__get_kb_stats` - Get KB statistics -- `mcp__code-forge__kb_lifecycle_health` - KB health report with recommendations -- `mcp__code-forge__kb_lifecycle_dedup` - Scan/merge duplicate documents -- `mcp__code-forge__kb_lifecycle_consolidate` - LLM-powered document merging -- `mcp__code-forge__kb_lifecycle_archive` - Archive, restore, list, find stale - ## Project-Specific Information {{PROJECT_SPECIFIC_CONTENT}} @@ -331,12 +36,86 @@ See `.claude/ARCHITECTURE.md` for system architecture overview. See `.claude/DEVELOPMENT_PRACTICES.md` for development workflow and practices. -## DO NOT +--- -- Don't bypass established patterns without discussing first +## Claude OS - My Memory System + +Claude OS is my memory across sessions. It stores decisions, patterns, solutions, and context so I never start cold. + +**Location**: `{{CLAUDE_OS_DIR}}` + +The MCP server is called "code-forge" internally for backwards compatibility, but it's Claude OS. + +### Session Tips + +- **Starting work**: Search `{{PROJECT_NAME}}-project_memories` for the task you're about to work on +- **Ending work**: Use `/claude-os-remember` to save key decisions, patterns, or blockers +- **Use `/claude-os-session start [task]`** to load context automatically +- **Use `/claude-os-session end`** to summarize and save before closing + +### Knowledge Bases + +1. **{{PROJECT_NAME}}-project_memories** - Decisions, patterns, solutions, insights +2. **{{PROJECT_NAME}}-project_index** - Automated codebase index +3. **{{PROJECT_NAME}}-project_profile** - Architecture, standards, practices +4. **{{PROJECT_NAME}}-knowledge_docs** - Documentation and guides + +**How to search:** +``` +mcp__code-forge__search_knowledge_base + kb_name: "{{PROJECT_NAME}}-project_memories" + query: "your search terms" +``` + +**Search across all KBs at once:** +``` +mcp__code-forge__search_all_knowledge_bases + query: "your search terms" + kb_filter: "{{PROJECT_NAME}}-" +``` + +### Slash Commands + +| Command | Purpose | +|---------|---------| +| `/claude-os-session start [task]` | Start session with context loading | +| `/claude-os-session end` | End session with save prompts | +| `/claude-os-search [query]` | Search memories | +| `/claude-os-remember [content]` | Quick save to memories | +| `/claude-os-save [title] [KB] [category]` | Full-featured save | +| `/claude-os-list [KB]` | List KBs or documents | +| `/claude-os-lifecycle [action] [KB]` | Health, dedup, consolidate, archive | + +### Available Skills + +1. **`initialize-project`** - Analyze project and generate coding standards +2. **`memory`** - Save and recall information (supports "remember this:", "save to memory", etc.) + +{{AGENT_OS_SECTION}} + +### MCP Tools + +| Tool | Purpose | +|------|---------| +| `mcp__code-forge__search_knowledge_base` | Search a single KB | +| `mcp__code-forge__search_all_knowledge_bases` | Search across all KBs | +| `mcp__code-forge__list_knowledge_bases` | List all KBs | +| `mcp__code-forge__list_documents` | List documents in KB | +| `mcp__code-forge__get_kb_stats` | KB statistics | +| `mcp__code-forge__upload_document` | Save a document/memory | +| `mcp__code-forge__kb_lifecycle_health` | KB health report | +| `mcp__code-forge__kb_lifecycle_dedup` | Scan/merge duplicates | +| `mcp__code-forge__kb_lifecycle_consolidate` | LLM-powered merging | +| `mcp__code-forge__kb_lifecycle_archive` | Archive/restore documents | + +--- + +## Guidelines + +- Search memories before making architectural decisions - Don't skip tests -- Don't create features without searching memories for existing patterns - Don't end a session without saving key learnings +- Don't bypass established patterns without discussing first ## IMPORTANT: Project Context @@ -345,6 +124,5 @@ This file (CLAUDE.md) is automatically loaded at the start of every Claude Code - Current architecture decisions - Team conventions - Common gotchas -- Frequently referenced information **Update this file as the project evolves!** diff --git a/tests/test_sqlite_manager.py b/tests/test_sqlite_manager.py index 0f465d1..3e3dbef 100644 --- a/tests/test_sqlite_manager.py +++ b/tests/test_sqlite_manager.py @@ -545,3 +545,149 @@ class TestSQLiteManagerSingleton: manager = get_sqlite_manager() assert manager is not None assert hasattr(manager, 'db_path') + + +@pytest.mark.unit +class TestQueryDocumentsMultiKB: + """Test cross-KB search functionality.""" + + def _add_docs(self, db, kb_name, docs): + """Helper to add documents with embeddings to a KB.""" + documents = [d["text"] for d in docs] + embeddings = [d["embedding"] for d in docs] + metadatas = [d.get("metadata", {}) for d in docs] + ids = [d["id"] for d in docs] + db.add_documents(kb_name=kb_name, documents=documents, + embeddings=embeddings, metadatas=metadatas, ids=ids) + + def test_multi_kb_basic(self, clean_db): + """Test searching across two KBs returns merged results.""" + clean_db.create_collection("kb_a", KBType.GENERIC) + clean_db.create_collection("kb_b", KBType.GENERIC) + + np.random.seed(10) + base = np.random.randn(768).astype(np.float32) + base = base / np.linalg.norm(base) + + # KB_A: one doc very similar to query + perturbed_a = base + np.random.randn(768).astype(np.float32) * 0.01 + perturbed_a = perturbed_a / np.linalg.norm(perturbed_a) + self._add_docs(clean_db, "kb_a", [ + {"id": "a1", "text": "Document from KB A", "embedding": perturbed_a.tolist(), + "metadata": {"filename": "a1.md"}} + ]) + + # KB_B: one doc less similar + perturbed_b = base + np.random.randn(768).astype(np.float32) * 0.05 + perturbed_b = perturbed_b / np.linalg.norm(perturbed_b) + self._add_docs(clean_db, "kb_b", [ + {"id": "b1", "text": "Document from KB B", "embedding": perturbed_b.tolist(), + "metadata": {"filename": "b1.md"}} + ]) + + result = clean_db.query_documents_multi_kb( + kb_names=["kb_a", "kb_b"], + query_embedding=base.tolist(), + n_results=10 + ) + + assert len(result["ids"]) == 2 + assert len(result["documents"]) == 2 + assert len(result["scores"]) == 2 + # First result should have higher score + assert result["scores"][0] >= result["scores"][1] + # Each result should have _source_kb + kbs_found = {m["_source_kb"] for m in result["metadatas"]} + assert "kb_a" in kbs_found + assert "kb_b" in kbs_found + + def test_multi_kb_dedup(self, clean_db): + """Test that duplicate content across KBs is deduplicated.""" + clean_db.create_collection("kb_x", KBType.GENERIC) + clean_db.create_collection("kb_y", KBType.GENERIC) + + np.random.seed(20) + emb = np.random.randn(768).astype(np.float32) + emb = emb / np.linalg.norm(emb) + + same_text = "Exact same content in both KBs for dedup testing" + self._add_docs(clean_db, "kb_x", [ + {"id": "x1", "text": same_text, "embedding": emb.tolist(), "metadata": {}} + ]) + self._add_docs(clean_db, "kb_y", [ + {"id": "y1", "text": same_text, "embedding": emb.tolist(), "metadata": {}} + ]) + + result = clean_db.query_documents_multi_kb( + kb_names=["kb_x", "kb_y"], + query_embedding=emb.tolist(), + n_results=10 + ) + + # Should only get 1 result after dedup + assert len(result["ids"]) == 1 + + def test_multi_kb_empty(self, clean_db): + """Test searching with no matching KBs returns empty.""" + result = clean_db.query_documents_multi_kb( + kb_names=["nonexistent_kb"], + query_embedding=[0.1] * 768, + n_results=5 + ) + assert result["ids"] == [] + assert result["documents"] == [] + assert result["scores"] == [] + + def test_multi_kb_respects_n_results(self, clean_db): + """Test that n_results limit is respected.""" + clean_db.create_collection("kb_limit", KBType.GENERIC) + + np.random.seed(30) + base = np.random.randn(768).astype(np.float32) + base = base / np.linalg.norm(base) + + docs = [] + for i in range(5): + perturbed = base + np.random.randn(768).astype(np.float32) * 0.01 + perturbed = perturbed / np.linalg.norm(perturbed) + docs.append({ + "id": f"lim{i}", "text": f"Document {i} for limit test", + "embedding": perturbed.tolist(), "metadata": {} + }) + self._add_docs(clean_db, "kb_limit", docs) + + result = clean_db.query_documents_multi_kb( + kb_names=["kb_limit"], + query_embedding=base.tolist(), + n_results=2 + ) + assert len(result["ids"]) == 2 + + def test_multi_kb_score_ordering(self, clean_db): + """Test that results are sorted by descending similarity score.""" + clean_db.create_collection("kb_order", KBType.GENERIC) + + np.random.seed(40) + base = np.random.randn(768).astype(np.float32) + base = base / np.linalg.norm(base) + + # Add docs with varying similarity + docs = [] + for i, noise in enumerate([0.01, 0.1, 0.5]): + perturbed = base + np.random.randn(768).astype(np.float32) * noise + perturbed = perturbed / np.linalg.norm(perturbed) + docs.append({ + "id": f"ord{i}", "text": f"Document {i} noise={noise}", + "embedding": perturbed.tolist(), "metadata": {} + }) + self._add_docs(clean_db, "kb_order", docs) + + result = clean_db.query_documents_multi_kb( + kb_names=["kb_order"], + query_embedding=base.tolist(), + n_results=10 + ) + + scores = result["scores"] + for i in range(len(scores) - 1): + assert scores[i] >= scores[i + 1], "Scores should be in descending order"