AGENTS.md

1# AGENTS.md — dspy
2 
3## Project Overview
4 
5DSPy is a framework for programming foundation models (LMs), not just prompting them. Its core philosophy is to separate a program's logic from its parameters (like prompts and model weights), enabling a compiler to automatically optimize these parameters for a given metric. The primary language is Python, and the framework is built around three core concepts: declarative `Signatures`, composable `Modules`, and prompt-optimizing `Teleprompters`.
6 
7## Tech Stack
8 
9- **Language:** Python (`>=3.10`, `<3.15`)
10- **Build System:** `setuptools`
11- **Testing:** `pytest`, `pytest-mock`, `pytest-asyncio`
12- **Linting & Formatting:** `ruff` (managed via `pre-commit`)
13- **Key Libraries:** `litellm`, `datasets`, `pandas`, `optuna`, `langchain_core`
14 
15## Architecture
16 
17The project follows a standard Python library structure with a clear separation of concerns. The main logic resides in the `dspy/` directory, with tests mirroring this structure in `tests/`.
18 
19- `pyproject.toml`: The central configuration file for project metadata, dependencies, and `ruff` settings.
20- `dspy/`: The main source code directory.
21  - `dspy/primitives/`: Core data structures and base classes.
22  - `dspy/signatures/`: Logic for defining declarative `Signature` objects, which specify the input/output of LM tasks.
23  - `dspy/predict/`: Contains foundational `Module` building blocks like `dspy.Predict` and `dspy.ChainOfThought`.
24  - `dspy/teleprompt/`: Home of the `Teleprompter` optimizers (e.g., `BootstrapFewShot`) that compile programs.
25  - `dspy/evaluate/`: Tools and metrics for evaluating program performance.
26  - `dspy/retrievers/`: Modules for integrating with various retrieval models.
27  - `dspy/clients/`: Contains clients for interacting with different LM providers (e.g., OpenAI, Anthropic).
28- `tests/`: Contains all tests, mirroring the `dspy/` source directory structure.
29- `docs/`: Project documentation in Markdown format.
30 
31## Code Style
32 
33Code style is strictly enforced by `ruff` and is non-negotiable, as `pre-commit` hooks will prevent commits that fail checks.
34 
35- **Formatter:** `ruff format` is used for all code formatting.
36- **Linter:** `ruff check` is used for linting.
37- **Configuration:** All rules for `ruff` are defined in `pyproject.toml`.
38- **Imports:** Imports should be ordered and formatted according to `ruff`'s rules.
39- **Naming Conventions:**
40  - Classes should be `PascalCase` (e.g., `BootstrapFewShot`).
41  - Functions and variables should be `snake_case` (e.g., `email_body`).
42- **Signatures:** The core building block of a DSPy program is the `dspy.Signature` class. It uses a declarative, class-based syntax.
43 
44```python
45# GOOD: Declarative signature definition
46class EmailClassifier(dspy.Signature):
47    """Classify an email and explain why."""
48    email_body = dspy.InputField()
49    classification = dspy.OutputField(desc="Spam or Not Spam")
50    reason = dspy.OutputField(desc="A short explanation.")
51```
52 
53## Anti-Patterns & Restrictions
54 
55- **NEVER use hardcoded f-string prompts:** The entire philosophy of DSPy is to separate program logic from the prompt's implementation. Prompts are parameters to be learned by a `Teleprompter`. Defining complex, hardcoded f-strings within your modules undermines the framework's purpose.
56- **NEVER create monolithic modules:** Complex tasks should be decomposed into smaller, interconnected `dspy.Module` instances. This promotes reusability, testability, and makes the program's logic easier to understand and optimize.
57 
58## Database & State Management
59 
60DSPy manages two primary types of state:
61 
621.  **Program State (Learnable Parameters):** The state of a DSPy program, which includes optimized prompts and few-shot examples, is stored directly within instances of `dspy.Module`. When a `Teleprompter` compiles a module, it modifies its internal state to store the optimized parameters. This is analogous to how a framework like PyTorch stores learned weights in its `nn.Module` layers.
63 
642.  **Global Configuration:** Global settings, such as the active Language Model (LM) and Retrieval Model (RM), are managed via the singleton `dspy.settings` object. Before executing a program, you must configure the necessary services.
65 
66```python
67    import dspy
68 
69    # Example: Configuring a global LM
70    lm = dspy.OpenAI(model='gpt-3.5-turbo', api_key='...')
71    dspy.settings.configure(lm=lm)
72```
73 
74There is no central database; state is managed in-memory within these two constructs.
75 
76## Error Handling & Logging
77 
78- **Error Handling:** Standard Python exceptions are the primary mechanism for error handling. Use built-in exception types where appropriate. For framework-specific errors, custom exception types may be defined. Use `assert` statements and runtime checks liberally to validate inputs and intermediate states, especially within `Module` forward passes.
79 
80- **Logging:** The standard Python `logging` module is used to provide visibility into the framework's operations, which is crucial for debugging the behavior of LMs and the compilation process of `Teleprompters`. Use the logger to trace the flow of data and the decisions made by different components.
81 
82## Testing Commands
83 
84- **Run all tests:**
85```bash
86  pytest
87```
88- **Check for linting errors:**
89```bash
90  ruff check .
91```
92- **Apply code formatting:**
93```bash
94  ruff format .
95```
96- **Run all pre-commit checks (including linting and formatting):**
97```bash
98  pre-commit run --all-files
99```
100 
101## Testing Guidelines
102 
103- **Framework:** All tests are written using `pytest`. Mocks are handled with `pytest-mock`.
104- **Location:** Tests are located in the top-level `tests/` directory. The file and directory structure within `tests/` must mirror the `dspy/` source directory. For example, tests for `dspy/teleprompt/bootstrap.py` should be in `tests/teleprompt/test_bootstrap.py`.
105- **Requirement:** All new features and bug fixes must be accompanied by new or updated tests.
106- **Test Types:** A combination of unit tests (for individual functions and classes) and integration tests (for interactions between `Modules`, `Signatures`, and `Teleprompters`) is required.
107- **Mocking:** When testing components that interact with external APIs (like LMs), use mocks to avoid making actual network calls. The `dspy.testing.vllm` module or `mocker` fixture from `pytest-mock` can be used for this.
108 
109```python
110# Example test structure in tests/predict/test_predict.py
111import dspy
112from dspy.predict.predict import Predict
113 
114def test_predict_initialization():
115    signature = "input -> output"
116    predictor = Predict(signature)
117    assert predictor.signature == dspy.Signature(signature)
118```
119 
120## Security & Compliance
121 
122- **NEVER hardcode API keys or other secrets.** API keys for external services (OpenAI, Cohere, Anthropic, etc.) must be managed through environment variables. The respective client modules (e.g., `dspy.OpenAI`) are designed to read these from the environment.
123- **Prompt Injection:** Be aware that DSPy does not inherently sanitize inputs against prompt injection attacks. This responsibility lies with the application developer building on top of the framework. Treat all user-provided data that is passed to an LM as potentially untrusted.
124 
125## Dependencies & Environment
126 
127- **Python Version:** Python `>=3.10` and `<3.15` is required.
128- **Installation:** To set up a development environment, clone the repository and run:
129```bash
130  git clone https://github.com/stanford-futuredata/dspy.git
131  cd dspy
132  pip install -e '.[dev]'
133  pre-commit install
134```
135- **Dependency Management:** Dependencies are defined in `pyproject.toml`.
136  - Core dependencies are in `[project.dependencies]`.
137  - Optional dependencies for development or specific integrations (e.g., `anthropic`, `weaviate`) are in `[project.optional-dependencies]`. Use `pip install -e '.[anthropic,weaviate]'` to install them.
138- **Environment Variables:** For running tests or applications that call external services, you must set the appropriate environment variables. For example, for OpenAI:
139```bash
140  export OPENAI_API_KEY=&quot;your-api-key-here&quot;
141```
142 
143## PR & Git Rules
144 
145- **Workflow:** The project uses the fork-and-pull-request model. All changes must be submitted via a PR from your personal fork.
146- **Branching:** Create new branches from the `main` branch for your feature or bugfix.
147- **Pull Requests:**
148  - Keep PRs small and focused on a single issue or feature.
149  - Provide a clear title and a detailed description of the changes.
150  - Ensure all automated checks (linting via `ruff`, tests via `pytest`) are passing before requesting a review. Commits that fail the `pre-commit` hooks will be blocked.
151 
152## Documentation Standards
153 
154- **Docstrings:** All public-facing modules, classes, and functions must have clear, descriptive docstrings. These are used to generate API documentation.
155- **Project Documentation:** The main documentation is written in Markdown and located in the `docs/` directory. When adding a new feature or making a significant change, update the relevant documentation files. The documentation is published at [dspy.ai](https://dspy.ai).
156 
157## Common Patterns
158 
159The most common and critical pattern in DSPy is the **Standard Workflow**, which strictly separates concerns:
160 
1611.  **Decompose and Declare with `dspy.Signature`**: ALWAYS start by defining the input/output behavior of a task declaratively. This is the "what".
162```python
163    class Summarize(dspy.Signature):
164        """Summarize the given text."""
165        text = dspy.InputField()
166        summary = dspy.OutputField()
167```
168 
1692.  **Compose with `dspy.Module`**: Assemble signatures into a program that defines the control flow. This is the "how".
170```python
171    class MyProgram(dspy.Module):
172        def __init__(self):
173            super().__init__()
174            self.summarizer = dspy.Predict(Summarize)
175 
176        def forward(self, document):
177            return self.summarizer(text=document)
178```
179 
1803.  **Optimize with `dspy.Teleprompter`**: Use a teleprompter to compile the module, optimizing its underlying prompts against a metric and training data. This automates prompt engineering.
181 
182```python
183    from dspy.teleprompt import BootstrapFewShot
184 
185    # ... setup lm, metric, and trainset ...
186    optimizer = BootstrapFewShot(metric=metric)
187    compiled_program = optimizer.compile(MyProgram(), trainset=trainset)
188```
189 
190## Agent Workflow / SOP
191 
192When assigned a task, follow this Standard Operating Procedure (SOP):
193 
1941.  **Understand the Goal:** Clearly identify the overall objective. What problem is the new or modified DSPy program trying to solve?
1952.  **Decompose the Task:** Break the problem down into logical, sequential, or conditional steps. For example, a question-answering task might be decomposed into "search for context" -> "synthesize answer from context".
1963.  **Define Signatures:** For each step identified, create a declarative `dspy.Signature` class. Define the necessary `InputField`s and `OutputField`s. Write a clear docstring for the signature.
1974.  **Implement the Module:** Create a `dspy.Module` that composes the signatures into a coherent program. In the `__init__`, instantiate the necessary DSPy prediction modules (e.g., `dspy.Predict`, `dspy.ChainOfThought`). In the `forward` method, define the control flow that calls these modules.
1985.  **Write Tests First:** Create a new test file in the `tests/` directory that mirrors the location of your new module. Write a test case that instantiates your module and runs a `forward` pass with mock data. Use `dspy.Example` to structure your test data.
1996.  **Verify and Refine:** Run the tests to ensure your module works as expected. Refine the logic as needed.
2007.  **Run Linters and Formatters:** Before finalizing, run `ruff format .` and `ruff check .` to ensure the code adheres to the project's style guidelines.
2018.  **Finalize:** Once all tests and checks pass, the task is complete. Document your changes clearly in preparation for a pull request.
202 
203## Few-Shot Examples
204 
205### Good: Using `Signature` and `Module`
206This example correctly separates the task definition (`EmailClassifier` signature) from the program logic (`SpamClassifier` module), allowing a `Teleprompter` to optimize the underlying prompt.
207 
208```python
209import dspy
210 
211# 1. GOOD: Define the I/O contract declaratively.
212class EmailClassifier(dspy.Signature):
213    """Classify an email and explain why."""
214    email_body = dspy.InputField()
215    classification = dspy.OutputField(desc="Spam or Not Spam")
216    reason = dspy.OutputField(desc="A short explanation.")
217 
218# 2. GOOD: Compose the signature into a reusable module.
219class SpamClassifier(dspy.Module):
220    def __init__(self):
221        super().__init__()
222        # The logic uses a pre-built module that will be optimized.
223        self.classify = dspy.ChainOfThought(EmailClassifier)
224 
225    def forward(self, email):
226        return self.classify(email_body=email)
227 
228# This structure allows a Teleprompter to optimize the `self.classify` module.
229# optimizer.compile(SpamClassifier(), trainset=...)
230```
231 
232### Bad: Hardcoding Prompts in an F-String
233This example violates the core DSPy philosophy by embedding a complex, hardcoded prompt directly into the program logic. This makes it impossible for a `Teleprompter` to automatically optimize the prompt, turning the code into a rigid, non-learnable script.
234 
235```python
236# BAD: Bypassing the Signature/Module system with a hardcoded prompt.
237def classify_email_badly(email_body: str):
238    # This prompt is now static and cannot be optimized by DSPy.
239    prompt = f"""
240    You are an email classification expert.
241    Analyze the following email and determine if it is "Spam" or "Not Spam".
242    Provide a short reason for your classification.
243 
244    Email: "{email_body}"
245    ---
246    Classification:
247    Reason:
248    """
249    
250    # This is an ad-hoc call to the LM, not part of a learnable DSPy module.
251    # response = dspy.settings.lm(prompt) 
252    # This code is brittle, hard to maintain, and cannot be optimized.
253    return "This approach is incorrect"
254 
255```