CLAUDE.md

1# CLAUDE.md
2 
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4 
5# Instructor Development Guide
6 
7## Commands
8- Install deps: `uv pip install -e ".[dev,anthropic]"` or `poetry install --with dev,anthropic`
9- Run tests: `uv run pytest tests/ -n auto`
10- Run specific test: `uv run pytest tests/path_to_test.py::test_name`
11- Skip LLM tests: `uv run pytest tests/ -k 'not llm and not openai'`
12- Type check: `uv run ty check`
13- Lint: `uv run ruff check instructor examples tests`
14- Format: `uv run ruff format instructor examples tests`
15- Generate coverage: `uv run coverage run -m pytest tests/ -k "not docs"` then `uv run coverage report`
16- Build documentation: `uv run mkdocs serve` (for local preview) or `./build_mkdocs.sh` (for production)
17- Waiting: use `sleep <seconds>` for explicit pauses (e.g., CI waits) or to let external processes finish
18 
19## Installation & Setup
20- Fork the repository and clone your fork
21- Install UV: `pip install uv`
22- Create virtual environment: `uv venv`
23- Install dependencies: `uv pip install -e ".[dev]"`
24- Install pre-commit: `uv run pre-commit install`
25- Run tests to verify: `uv run pytest tests/ -k "not openai"`
26 
27## Code Style Guidelines
28- **Typing**: Use strict typing with annotations for all functions and variables
29- **Imports**: Standard lib → third-party → local imports
30- **Formatting**: Follow Black's formatting conventions (enforced by Ruff)
31- **Models**: Define structured outputs as Pydantic BaseModel subclasses
32- **Naming**: snake_case for functions/variables, PascalCase for classes
33- **Error Handling**: Use custom exceptions from exceptions.py, validate with Pydantic
34- **Comments**: Docstrings for public functions, inline comments for complex logic
35 
36## Conventional Commits
37- **Format**: `type(scope): description`
38- **Types**: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert
39- **Examples**:
40  - `feat(anthropic): add support for Claude 3.5`
41  - `fix(openai): correct response parsing for streaming`
42  - `docs(README): update installation instructions`
43  - `test(gemini): add validation tests for JSON mode`
44 
45## Core Architecture
46- **Base Classes**: `Instructor` and `AsyncInstructor` in client.py are the foundation
47- **Factory Pattern**: Provider-specific factory functions (`from_openai`, `from_anthropic`, etc.)
48- **Unified Access**: `from_provider()` function in auto_client.py for automatic provider detection
49- **Mode System**: `Mode` enum categorizes different provider capabilities (tools vs JSON output)
50- **Patching Mechanism**: Uses Python's dynamic nature to patch provider clients for structured outputs
51- **Response Processing**: Transforms raw API responses into validated Pydantic models
52- **DSL Components**: Special types like Partial, Iterable, Maybe extend the core functionality
53 
54## Provider Architecture
55- **Supported Providers**: OpenAI, Anthropic, Gemini, Cohere, Mistral, Groq, VertexAI, Fireworks, Cerebras, Writer, Databricks, Anyscale, Together, LiteLLM, Bedrock, Perplexity
56- **Provider Implementation**: Each provider has a dedicated client file (e.g., `client_anthropic.py`) with factory functions
57- **Modes**: Different providers support specific modes (`Mode` enum): `ANTHROPIC_TOOLS`, `GEMINI_JSON`, etc.
58- **Common Pattern**: Factory functions (e.g., `from_anthropic`) take a native client and return patched `Instructor` instances
59- **Provider Testing**: Tests in `tests/llm/` directory, define Pydantic models, make API calls, verify structured outputs
60- **Provider Detection**: `get_provider` function analyzes base URL to detect which provider is being used
61 
62## Key Components
63- **process_response.py**: Handles parsing and converting LLM outputs to Pydantic models
64- **patch.py**: Contains the core patching logic for modifying provider clients
65- **function_calls.py**: Handles generating function/tool schemas from Pydantic models
66- **hooks.py**: Provides event hooks for intercepting various stages of the LLM request/response cycle
67- **dsl/**: Domain-specific language extensions for specialized model types
68- **retry.py**: Implements retry logic for handling validation failures
69- **validators.py**: Custom validation mechanisms for structured outputs
70 
71## Testing Guidelines
72- Tests are organized by provider under `tests/llm/`
73- Each provider has its own conftest.py with fixtures
74- Standard tests cover: basic extraction, streaming, validation, retries
75- Evaluation tests in `tests/llm/test_provider/evals/` assess model capabilities
76- Use parametrized tests when testing similar functionality across variants
77- **IMPORTANT**: No mocking in tests - tests make real API calls
78 
79## Documentation Guidelines
80- Every provider needs documentation in `docs/integrations/` following standard format
81- Provider docs should include: installation, basic example, modes supported, special features
82- When adding a new provider, update `mkdocs.yml` navigation and redirects
83- Example code should include complete imports and environment setup
84- Tutorials should progress from simple to complex concepts
85- New features should include conceptual explanation in `docs/concepts/`
86- **Writing Style**: Grade 10 reading level, all examples must be working code
87 
88## Branch and Development Workflow
891. Fork and clone the repository
902. Create feature branch: `git checkout -b feat/your-feature`
913. Make changes and add tests
924. Run tests and linting
935. Commit with conventional commit message
946. Push to your fork and create PR
957. Use stacked PRs for complex features
96 
97## Adding New Providers
98 
99### Step-by-Step Guide
1001. **Update Provider Enum** in `instructor/utils.py`:
101```python
102   class Provider(Enum):
103       YOUR_PROVIDER = "your_provider"
104```
105 
1062. **Add Provider Modes** in `instructor/mode.py`:
107```python
108   class Mode(enum.Enum):
109       YOUR_PROVIDER_TOOLS = "your_provider_tools"
110       YOUR_PROVIDER_JSON = "your_provider_json"
111```
112 
1133. **Create Client Implementation** `instructor/client_your_provider.py`:
114   - Use overloads for sync/async variants
115   - Validate mode compatibility
116   - Return appropriate Instructor/AsyncInstructor instance
117   - Handle provider-specific edge cases
118 
1194. **Add Conditional Import** in `instructor/__init__.py`:
120```python
121   if importlib.util.find_spec("your_provider_sdk") is not None:
122       from .client_your_provider import from_your_provider
123       __all__ += ["from_your_provider"]
124```
125 
1265. **Update Auto Client** in `instructor/auto_client.py`:
127   - Add to `supported_providers` list
128   - Implement provider handling in `from_provider()`
129   - Update `get_provider()` function if URL-detectable
130 
1316. **Create Tests** in `tests/llm/test_your_provider/`:
132   - `conftest.py` with client fixtures
133   - Basic extraction tests
134   - Streaming tests
135   - Validation/retry tests
136   - No mocking - use real API calls
137 
1387. **Add Documentation** in `docs/integrations/your_provider.md`:
139   - Installation instructions
140   - Basic usage examples
141   - Supported modes
142   - Provider-specific features
143 
1448. **Update Navigation** in `mkdocs.yml`:
145   - Add to integrations section
146   - Include redirects if needed
147 
148## Contributing to Evals
149- Standard evals for each provider test model capabilities
150- Create new evals following existing patterns
151- Run evals as part of integration test suite
152- Performance tracking and comparison
153 
154## Pull Request Guidelines
155- Keep PRs small and focused
156- Include tests for all changes
157- Update documentation as needed
158- Follow PR template
159- Link to relevant issues
160- **Update CHANGELOG.md**: Every PR that changes behavior (fix, feat, security, deprecation) must add an entry under the current `[Unreleased]` section in `CHANGELOG.md`. Format: `- **Area**: Description ([#PR](url))`
161 
162## Type System and Best Practices
163 
164### Type Checking with ty
165- **Type Checker**: Using `ty` for fast, incremental type checking
166- **Python Version**: 3.9+ for compatibility
167- **Configuration**: Uses `pyproject.toml` settings for type checking
168- Run `uv run ty check` before committing - aim for zero errors
169 
170### Code Quality Checks Before Committing
171Always run these checks before committing code:
1721. **Ruff linting**: `uv run ruff check .` - Fix all errors
1732. **Ruff formatting**: `uv run ruff format .` - Apply consistent formatting
1743. **Type checking**: `uv run ty check` - Aim for zero type errors
1754. **Tests**: Run relevant tests to ensure changes don't break functionality
176 
177### Type Patterns
178- **Bounded TypeVars**: Use `T = TypeVar("T", bound=Union[BaseModel, ...])` for constraints
179- **Version Compatibility**: Handle Python 3.9 vs 3.10+ typing differences explicitly
180- **Union Type Syntax**: Use `from __future__ import annotations` to enable Python 3.10+ union syntax (`|`) in Python 3.9
181- **Simple Type Detection**: Special handling for `list[Union[int, str]]` patterns
182- **Runtime Type Handling**: Graceful fallbacks for compatibility
183 
184### Pydantic Integration
185- Heavy use of `BaseModel` for structured outputs
186- `TypeAdapter` used internally for JSON schema generation
187- Field validators and custom types
188- Models serve dual purpose: validation and documentation
189 
190## Building Documentation
191 
192### Setup
193```bash
194# Install documentation dependencies
195pip install -r requirements-doc.txt
196```
197 
198### Local Development
199```bash
200# Serve documentation locally with hot reload
201uv run mkdocs serve
202 
203# Build documentation for production
204./build_mkdocs.sh
205```
206 
207### Documentation Features
208- **Material Theme**: Modern UI with extensive customization
209- **Plugins**:
210  - `mkdocstrings` - API documentation from docstrings
211  - `mkdocs-jupyter` - Notebook integration
212  - `mkdocs-redirects` - URL management
213  - Custom hooks for code processing
214- **Custom Processing**: `hide_lines.py` removes code marked with `# <%hide%>`
215- **Redirect Management**: Comprehensive redirect maps for moved content
216 
217### Writing Documentation
218- Follow templates in `docs/templates/` for consistency
219- Grade 10 reading level for accessibility
220- All code examples must be runnable
221- Include complete imports and environment setup
222- Progressive complexity: simple → advanced
223 
224## Project Structure
225- `instructor/` - Core library code
226  - Base classes (`client.py`): `Instructor` and `AsyncInstructor`
227  - Provider clients (`client_*.py`): Factory functions for each provider
228  - DSL components (`dsl/`): Partial, Iterable, Maybe, Citation extensions
229  - Core logic: `patch.py`, `process_response.py`, `function_calls.py`
230  - CLI tools (`cli/`): Batch processing, file management, usage tracking
231- `tests/` - Test suite organized by provider
232  - Provider-specific tests in `tests/llm/test_<provider>/`
233  - Evaluation tests for model capabilities
234  - No mocking - all tests use real API calls
235- `docs/` - MkDocs documentation
236  - `concepts/` - Core concepts and features
237  - `integrations/` - Provider-specific guides
238  - `examples/` - Practical examples and cookbooks
239  - `learning/` - Progressive tutorial path
240  - `blog/posts/` - Technical articles and announcements
241  - `templates/` - Templates for new docs (provider, concept, cookbook)
242- `examples/` - Runnable code examples
243  - Feature demos: caching, streaming, validation, parallel processing
244  - Use cases: classification, extraction, knowledge graphs
245  - Provider examples: anthropic, openai, groq, mistral
246  - Each example has `run.py` as the main entry point
247- `typings/` - Type stubs for untyped dependencies
248 
249## Documentation Structure
250- **Getting Started Path**: Installation → First Extraction → Response Models → Structured Outputs
251- **Learning Patterns**: Simple Objects → Lists → Nested Structures → Validation → Streaming
252- **Example Organization**: Self-contained directories with runnable code demonstrating specific features
253- **Blog Posts**: Technical deep-dives with code examples in `docs/blog/posts/`
254 
255## Example Patterns
256When creating examples:
257- Use `run.py` as the main file name
258- Include clear imports: stdlib → third-party → instructor
259- Define Pydantic models with descriptive fields
260- Show expected output in comments
261- Handle errors appropriately
262- Make examples self-contained and runnable
263 
264## Dependency Management
265 
266### Core Dependencies
267- **Minimal core**: `openai`, `pydantic`, `docstring-parser`, `typer`, `rich`
268- **Python requirement**: `<4.0,>=3.9`
269- **Pydantic version**: `<3.0.0,>=2.8.0` (constrained for stability)
270 
271### Optional Dependencies
272Provider-specific packages as extras:
273```bash
274# Install with specific provider
275pip install &quot;instructor[anthropic]&quot;
276pip install &quot;instructor[google-generativeai]&quot;
277pip install &quot;instructor[groq]&quot;
278```
279 
280### Development Dependencies
281```bash
282# Install all development dependencies
283uv pip install -e &quot;.[dev]&quot;
284```
285Includes:
286- ty 
287- `pytest` and `pytest-asyncio` - Testing
288- `ruff` - Linting and formatting
289- `coverage` - Test coverage
290- `mkdocs` and plugins - Documentation
291 
292### Version Constraints
293- **Upper bounds on all dependencies** for stability
294- **Provider SDK versions** pinned to tested versions
295- **Test dependencies** include evaluation frameworks
296 
297### Managing Dependencies
298- Update `pyproject.toml` for new dependencies
299- Test with multiple Python versions (3.9-3.12)
300- Run full test suite after dependency updates
301- Document any provider-specific version requirements
302 
303The library enables structured LLM outputs using Pydantic models across multiple providers with type safety.
304