CLAUDE.md

1# CLAUDE.md
2 
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4 
5## Project Overview
6 
7This is an educational workshop repository for building Retrieval-Augmented Generation (RAG) applications, used for the O'Reilly training course "Building Reliable RAG Applications: From PoC to Production". The workshop follows a progressive structure from basic to advanced RAG techniques.
8 
9## Workshop Structure
10 
111. **Naive RAG** (`naive-rag/`) - Basic RAG implementation concepts
122. **Naive RAG Challenges** - Identifying limitations and areas for improvement  
133. **Advanced RAG** (`advanced-rag/`) - Reranking and hybrid search techniques
144. **SciFact Dataset** (`advanced-rag/scifact/`) - Advanced demonstrations using scientific data
15 
16## Environment Setup
17 
18This project uses [uv](https://docs.astral.sh/uv/) for dependency management.
19 
20```bash
21# Install uv (if not installed)
22curl -LsSf https://astral.sh/uv/install.sh | sh
23 
24# Install Python 3.11 and all dependencies
25uv sync
26 
27# Set environment variables in .env file
28OPENAI_API_KEY=your_key_here
29QDRANT_URL=your_qdrant_url_here
30QDRANT_API_KEY=your_qdrant_api_key_here  # For cloud setup only
31COHERE_API_KEY=your_cohere_key_here  # Optional, for advanced features
32```
33 
34## Running Notebooks
35 
36Start Jupyter and run notebooks in order:
37 
38```bash
39uv run jupyter lab
40```
41 
421. `naive-rag/01-naive-rag.ipynb` - Basic RAG implementation
432. `naive-rag/02-naive-rag-challenges.ipynb` - Exploring limitations
443. `advanced-rag/01-advanced-rag-rerank.ipynb` - Advanced techniques with reranking
45 
46All notebooks automatically detect Qdrant setup (cloud vs local) based on environment variables.
47 
48## Data Setup Commands
49 
50Two setup options available:
51 
52**Option A: Qdrant Cloud (Recommended)**
53```bash
54uv run python scripts/ingest_to_qdrant_cloud.py
55```
56 
57**Option B: Local Docker Setup**
58```bash
59docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant:v1.13.2
60uv run python scripts/ingest_to_qdrant_cloud.py
61```
62 
63The ingestion script loads 61 Wikipedia articles, creates 1,210 chunks (300 chars with 50 char overlap), and uploads to Qdrant.
64 
65## Key Architecture Components
66 
67### Data Processing Pipeline
68- **Raw Data**: Wikipedia articles fetched via `scripts/create_extended_dataset.py`
69- **Text Processing**: Clean markup, remove citations using mwparserfromhell and BeautifulSoup
70- **Chunking**: 300 character chunks with 50 character overlap (configurable)
71- **Embeddings**: OpenAI text-embedding-3-small (1536 dimensions)
72- **Vector Storage**: Qdrant with COSINE distance metric
73 
74### RAG Components
75- **Retrieval**: Vector similarity search via Qdrant
76- **Reranking**: Cross-encoders using FlagEmbedding and Cohere
77- **Generation**: OpenAI GPT models for answer synthesis
78- **Evaluation**: RAGAS metrics for comprehensive assessment
79 
80### Evaluation Framework
81- **Location**: `naive-rag/rag_evaluator_v2.py`
82- **Metrics**: Context recall, precision, relevance using RAGAS
83- **Test Data**: `data/wiki_eval_dataset.json` with question-answer pairs
84 
85## Technical Configuration
86 
87### Vector Database Settings
88- Collection Name: `workshop_wikipedia_extended`
89- Vector Dimension: 1536
90- Distance Metric: COSINE
91- Batch Size: 50 (for API rate limiting)
92 
93### Model Parameters
94- Embedding Model: `text-embedding-3-small`
95- Chunk Size: 300 characters
96- Chunk Overlap: 50 characters
97- Temperature: 0.1 (for consistent outputs)
98 
99## Development Guidelines
100 
101### Notebook Structure
102- Start with clear objectives and prerequisites
103- Include comprehensive imports section
104- Add configuration cells for API keys and parameters
105- Structure: Setup → Data Loading → Processing → Evaluation
106- End with results summary and next steps
107 
108### Error Handling
109- Validate environment variables before API calls
110- Handle API rate limits gracefully with retry logic
111- Provide fallback options for different LLM providers
112- Include clear error messages for common setup issues
113 
114### Performance Considerations
115- Monitor token usage and costs
116- Implement caching for embeddings when possible
117- Use batch processing for large datasets
118- Optimize chunk sizes based on content type
119 
120## Common Issues & Troubleshooting
121 
122**Collection Not Found**: Run data ingestion script from project root
123**Docker Issues**: Check container status with `docker ps`
124**API Rate Limits**: Ingestion script includes delay and batch processing
125**Environment Variables**: Ensure `.env` file is in project root and restart Jupyter kernel
126 
127## File Structure
128 
129```
130├── naive-rag/                    # Basic RAG implementation
131├── advanced-rag/                 # Advanced techniques (reranking, hybrid)
132├── scripts/                      # Data preparation and ingestion
133├── data/                         # Processed datasets and evaluation data
134├── pyproject.toml                # Python dependencies (uv)
135├── uv.lock                       # Locked dependencies
136├── SETUP.md                      # Detailed setup instructions
137└── .env                          # Environment variables (not in git)
138```
139 
140When working with this codebase, prioritize educational clarity and maintain the progressive learning structure from naive to advanced RAG techniques.
141