f6a1d6b417
Moved 25 files from root to organized subdirectories while keeping essential scripts (install.sh, setup.sh, start*.sh, stop*.sh) in root. ## Changes: ### New Directory Structure: - Created `/scripts/` for utility scripts - Created `/docs/guides/` for user guides - Created `/docs/development/` for dev documentation - Created `/docs/archive/` for historical/legacy docs ### Moved Files: **To `/scripts/` (7 files):** - backup_claude_os.sh - restore_claude_os.sh - run_tests.sh - setup_native.sh - start_mcp_server.sh - start_redis_workers.sh - entrypoint.sh **To `/docs/guides/` (7 files):** - AUTH_SETUP.md - BACKUP_RESTORE_GUIDE.md - HOOK_SETUP_QUICK_START.md - SHARING_GUIDE.md - START_SCRIPTS_GUIDE.md - PORTS_GUIDE.md - WHAT_IS_CLAUDE_OS.md **To `/docs/development/` (4 files):** - COMPREHENSIVE_REVIEW.md - SECURITY_FIXES.md - CLAUDE_OS_DEVELOPER_GUIDE.md - PROJECT_SETUP_AND_COMMANDS.md **To `/docs/archive/` (10 files):** - HYBRID_INDEXING_COMPLETE.md - INITIALIZE_PROJECT_HOOKS_FIX.md - INSTALL_ENHANCEMENTS.md - INSTALL_FIXES.md - PISTN_TEST_RESULTS.md - SESSION_SUMMARY.md - TEST_COVERAGE.md - TEST_RESULTS.md - TESTING_STATUS.md - README_TESTING.md ### Updated References: **Fixed broken documentation links in:** - docs/QUICK_REFERENCE.md - docs/PRODUCTION_DEPLOYMENT.md - docs/SCRIPTS_GUIDE.md - docs/development/SECURITY_FIXES.md - docs/guides/BACKUP_RESTORE_GUIDE.md - docs/development/CLAUDE_OS_DEVELOPER_GUIDE.md - tests/TESTING_GUIDE.md - tests/TEST_COVERAGE.md **Script references updated to:** - `./scripts/backup_claude_os.sh` - `./scripts/restore_claude_os.sh` - `./scripts/run_tests.sh` - `./scripts/setup_native.sh` - `./scripts/start_mcp_server.sh` - `./scripts/start_redis_workers.sh` ### Remaining in Root (9 essential files): - install.sh - setup.sh - start.sh - start_all_services.sh - stop_all_services.sh - restart_services.sh - README.md - CLAUDE.md - requirements.txt (+ GEMINI_DIAGRAM_PROMPT.md) ## Benefits: ✅ Cleaner root directory (9 files vs 34 files) ✅ Organized documentation by purpose ✅ Scripts grouped in dedicated directory ✅ Essential startup commands still in root ✅ All references updated and verified ✅ No broken links Version: 2.0.2
209 lines
4.7 KiB
Markdown
209 lines
4.7 KiB
Markdown
# Claude OS Test Coverage
|
|
|
|
## Overview
|
|
|
|
Comprehensive test suite for Claude OS RAG system covering all critical components.
|
|
|
|
## Test Files
|
|
|
|
### 1. `test_rag_engine.py` - RAG Engine
|
|
**Purpose**: Test RAG query execution and response generation
|
|
|
|
**Tests**:
|
|
- ✅ RAG engine initialization
|
|
- ✅ Query with no documents (empty KB)
|
|
- ✅ Query with documents (full pipeline)
|
|
- ✅ LLM integration
|
|
- ✅ Source retrieval and formatting
|
|
|
|
**Coverage**: End-to-end RAG pipeline, LLM integration, context retrieval
|
|
|
|
---
|
|
|
|
### 3. `test_embeddings.py` - Embedding Generation
|
|
**Purpose**: Test embedding model and vector operations
|
|
|
|
**Tests**:
|
|
- ✅ Ollama embedding generation
|
|
- ✅ Embedding consistency (same text → same embedding)
|
|
- ✅ Embedding dimension validation (768)
|
|
- ✅ Variable text length handling
|
|
- ✅ Cosine similarity calculation
|
|
- ✅ Vector normalization
|
|
|
|
**Coverage**: Embedding generation, vector math, model consistency
|
|
|
|
---
|
|
|
|
### 4. `test_document_processing.py` - Document Ingestion
|
|
**Purpose**: Test document processing and chunking
|
|
|
|
**Tests**:
|
|
- ✅ Text file ingestion
|
|
- ✅ PDF file ingestion
|
|
- ✅ Markdown file ingestion
|
|
- ✅ Fixed-size chunking
|
|
- ✅ Sentence-based chunking
|
|
- ✅ File type detection
|
|
|
|
**Coverage**: Document parsing, chunking strategies, file handling
|
|
|
|
---
|
|
|
|
### 5. `test_api.py` - FastAPI Endpoints
|
|
**Purpose**: Test all API endpoints
|
|
|
|
**Tests**:
|
|
- ✅ List knowledge bases
|
|
- ✅ Create knowledge base
|
|
- ✅ Delete knowledge base
|
|
- ✅ Get KB statistics
|
|
- ✅ List documents
|
|
- ✅ Chat endpoint (query)
|
|
- ✅ Chat with empty KB
|
|
- ✅ Chat with invalid KB
|
|
- ✅ Health check endpoint
|
|
- ✅ Document upload (text files)
|
|
- ✅ Invalid file type handling
|
|
|
|
**Coverage**: API endpoints, request/response validation, error handling
|
|
|
|
---
|
|
|
|
## Test Categories (Markers)
|
|
|
|
### Unit Tests (`@pytest.mark.unit`)
|
|
- Fast, no external dependencies
|
|
- Test individual functions and utilities
|
|
- Mock external services
|
|
|
|
### Integration Tests (`@pytest.mark.integration`)
|
|
- Require database and/or Ollama
|
|
- Test component interactions
|
|
- Use real services
|
|
|
|
### Specific Markers
|
|
- `@pytest.mark.vector` - Vector operations
|
|
- `@pytest.mark.rag` - RAG engine tests
|
|
- `@pytest.mark.api` - API endpoint tests
|
|
- `@pytest.mark.embeddings` - Embedding generation
|
|
- `@pytest.mark.slow` - Tests that take >5 seconds
|
|
|
|
---
|
|
|
|
## Running Tests
|
|
|
|
### All tests:
|
|
```bash
|
|
./scripts/run_tests.sh
|
|
```
|
|
|
|
### Unit tests only (fast):
|
|
```bash
|
|
./scripts/run_tests.sh --unit
|
|
```
|
|
|
|
### Integration tests:
|
|
```bash
|
|
./scripts/run_tests.sh --integration
|
|
```
|
|
|
|
### Specific category:
|
|
```bash
|
|
./scripts/run_tests.sh --vector
|
|
./scripts/run_tests.sh --rag
|
|
./scripts/run_tests.sh --api
|
|
```
|
|
|
|
### With coverage:
|
|
```bash
|
|
./scripts/run_tests.sh --coverage
|
|
```
|
|
|
|
---
|
|
|
|
## Test Fixtures
|
|
|
|
### Database Fixtures
|
|
- `test_db_config` - Test database configuration
|
|
- `db_connection` - Session-scoped database connection
|
|
- `clean_db` - Clean database before each test
|
|
- `sample_kb` - Sample knowledge base
|
|
- `sample_documents` - Sample documents with embeddings
|
|
|
|
### Data Fixtures
|
|
- `sample_embedding` - 768-dimensional embedding
|
|
- `sample_text_file` - Sample text file
|
|
- `sample_pdf_file` - Sample PDF file
|
|
- `sample_markdown_file` - Sample Markdown file
|
|
|
|
### Mock Fixtures
|
|
- `mock_ollama_embedding` - Mock embedding model
|
|
- `mock_ollama_llm` - Mock LLM
|
|
- `api_client` - FastAPI test client
|
|
|
|
---
|
|
|
|
## Coverage Goals
|
|
|
|
| Component | Target | Current |
|
|
|-----------|--------|---------|
|
|
| PostgreSQL Manager | 90% | TBD |
|
|
| RAG Engine | 85% | TBD |
|
|
| Document Processing | 80% | TBD |
|
|
| API Endpoints | 90% | TBD |
|
|
| Embeddings | 85% | TBD |
|
|
| **Overall** | **85%** | **TBD** |
|
|
|
|
---
|
|
|
|
## CI/CD Integration
|
|
|
|
Tests are designed to run in CI/CD pipelines:
|
|
|
|
```yaml
|
|
# GitHub Actions example
|
|
- name: Run tests
|
|
run: |
|
|
pytest --cov=app --cov-report=xml
|
|
|
|
- name: Upload coverage
|
|
uses: codecov/codecov-action@v3
|
|
```
|
|
|
|
---
|
|
|
|
## Known Issues / TODO
|
|
|
|
- [ ] Add performance benchmarks
|
|
- [ ] Add stress tests for concurrent queries
|
|
- [ ] Add tests for different chunking strategies
|
|
- [ ] Add tests for hybrid search (BM25 + vector)
|
|
- [ ] Add tests for reranking
|
|
- [ ] Add tests for different LLM models
|
|
- [ ] Add tests for error recovery
|
|
- [ ] Add tests for rate limiting
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
When adding new features:
|
|
|
|
1. **Write tests first** (TDD approach)
|
|
2. **Use appropriate markers** (`@pytest.mark.unit`, etc.)
|
|
3. **Add fixtures** to `conftest.py` if reusable
|
|
4. **Update this document** with new test coverage
|
|
5. **Ensure >80% coverage** for new code
|
|
|
|
---
|
|
|
|
## Test Data
|
|
|
|
Test data is generated using:
|
|
- **Faker** - For realistic test data
|
|
- **NumPy** - For random embeddings (seeded for reproducibility)
|
|
- **Fixtures** - For consistent test scenarios
|
|
|
|
All test data is isolated and cleaned up after tests.
|
|
|