.cursorrules (deprecated)

1# LangChain / LLM Application Development — Cursor Rules
2 
3You are an expert AI application developer building LLM-powered applications with LangChain, LangGraph, and related frameworks in Python.
4 
5## Code Style
6 
7- Use Python 3.11+ with type hints throughout. Type all function signatures.
8- Use `snake_case` for functions and variables, `PascalCase` for classes, `UPPER_SNAKE_CASE` for constants.
9- Use `async/await` for all LLM calls and I/O operations. LLM calls are I/O-bound.
10- Use `pydantic` v2 for all data models and configuration. LangChain uses Pydantic internally.
11- Use `python-dotenv` for environment variables. Never hardcode API keys.
12- Import LangChain modules specifically: `from langchain_openai import ChatOpenAI`, not `from langchain import *`.
13- Use the LangChain ecosystem packages: `langchain-core`, `langchain-openai`, `langchain-anthropic`, `langchain-community`.
14- Prefer `langchain-core` abstractions (Runnables, LCEL) over legacy Chain classes.
15- Follow Black formatting (88 char line length) and Ruff linting.
16 
17## LangChain Expression Language (LCEL)
18 
19- Use LCEL (pipe `|` syntax) for composing chains. It is the preferred way to build chains:
20```python
21  chain = prompt | llm | output_parser
22  result = await chain.ainvoke({"input": "Hello"})
23```
24- Use `RunnablePassthrough` for passing data through unchanged.
25- Use `RunnableLambda` for custom transformation functions in a chain.
26- Use `RunnableParallel` for executing multiple branches concurrently.
27- Use `RunnableBranch` for conditional routing between different chain paths.
28- Use `.with_retry()` for automatic retry on transient errors.
29- Use `.with_fallbacks()` for graceful degradation to alternative models.
30- Use `.batch()` for processing multiple inputs concurrently.
31- All chains should support both `.invoke()` (sync) and `.ainvoke()` (async).
32 
33## Prompt Engineering
34 
35- Use `ChatPromptTemplate` with explicit system, human, and AI message roles.
36- Store prompts in separate files or a prompt registry, not inline in code.
37- Use input variables with clear, descriptive names: `{user_query}`, `{document_context}`, `{chat_history}`.
38- Use few-shot examples in prompts for consistent output format.
39- Include output format instructions in the system prompt. Use structured output when possible.
40- Use `ChatPromptTemplate.from_messages()` for multi-turn conversation prompts.
41- Version your prompts. Track which prompt version was used for each response.
42- Test prompts with diverse inputs, including edge cases and adversarial inputs.
43- Keep system prompts concise and focused. One clear role, one clear task.
44 
45## Structured Output
46 
47- Use Pydantic models with `.with_structured_output()` for type-safe LLM responses:
48```python
49  class Analysis(BaseModel):
50      sentiment: Literal["positive", "negative", "neutral"]
51      confidence: float = Field(ge=0, le=1)
52      reasoning: str
53 
54  chain = prompt | llm.with_structured_output(Analysis)
55```
56- Define clear field descriptions in Pydantic models — they help the LLM understand the expected output.
57- Use `Literal` types for fields with fixed allowed values.
58- Use `Field(description="...")` for every field to guide the LLM.
59- Handle parsing failures gracefully. Use fallback logic when structured output parsing fails.
60 
61## RAG (Retrieval-Augmented Generation)
62 
63- Use the standard RAG pipeline: query -> retrieve -> format context -> generate.
64- Split documents appropriately: use `RecursiveCharacterTextSplitter` with overlap for general text.
65- Choose chunk size based on the retrieval task: 500-1000 tokens for Q&A, larger for summarization.
66- Use embedding models matched to your vector store: `OpenAIEmbeddings`, `HuggingFaceEmbeddings`.
67- Store embeddings in a vector database: Chroma (local), Pinecone, Weaviate, Qdrant (production).
68- Use metadata filtering on retrieval for scoping results (by document type, date, source).
69- Implement hybrid search: combine vector similarity with keyword search (BM25) for better recall.
70- Use `MultiQueryRetriever` or query expansion for improved retrieval on complex questions.
71- Use `ContextualCompressionRetriever` to filter and compress retrieved documents before generation.
72- Always include source attribution in RAG responses. Return the source documents with the answer.
73- Handle the "no relevant documents found" case — tell the user when the knowledge base doesn't have the answer.
74 
75## Agents and Tools
76 
77- Use LangGraph for complex agent workflows. It provides better control flow than legacy AgentExecutor.
78- Define tools with clear descriptions and typed parameters:
79```python
80  @tool
81  def search_database(query: str, limit: int = 10) -> list[dict]:
82      """Search the product database for items matching the query.
83      Args:
84          query: Natural language search query
85          limit: Maximum number of results to return
86      """
87```
88- Keep tool descriptions concise but precise. The LLM uses descriptions to decide when to use each tool.
89- Validate tool inputs before execution. Return clear error messages for invalid inputs.
90- Implement tool timeouts. LLM-called tools should not block indefinitely.
91- Use `ToolMessage` for returning tool results to the LLM.
92- Limit the number of tools available to an agent (5-10 max). Too many tools degrade tool selection accuracy.
93- Log all tool calls for debugging and auditing.
94 
95## LangGraph
96 
97- Use LangGraph for stateful, multi-step agent workflows with conditional logic.
98- Define graph state with a TypedDict:
99```python
100  class AgentState(TypedDict):
101      messages: Annotated[list[BaseMessage], add_messages]
102      context: str
103      iteration_count: int
104```
105- Use nodes for processing steps and edges for flow control.
106- Use conditional edges for routing based on LLM decisions or state values.
107- Implement human-in-the-loop with `interrupt_before` or `interrupt_after` on nodes.
108- Use checkpointing (`MemorySaver`, `SqliteSaver`) for conversation persistence and state recovery.
109- Set maximum iteration limits to prevent infinite loops in agent cycles.
110- Use subgraphs for encapsulating complex sub-workflows.
111 
112## Memory and Conversation
113 
114- Use `ChatMessageHistory` implementations for conversation memory.
115- Use `RunnableWithMessageHistory` to attach memory to chains.
116- Choose memory strategy based on use case:
117  - `ConversationBufferMemory` for short conversations
118  - `ConversationSummaryMemory` for long conversations
119  - `ConversationBufferWindowMemory` for recent context only
120- Store conversation history in a persistent backend (Redis, PostgreSQL) for production apps.
121- Implement token-aware truncation to stay within model context limits.
122- Clear or summarize memory periodically for long-running conversations.
123 
124## Error Handling
125 
126- Catch LLM API errors explicitly: `openai.RateLimitError`, `openai.APIConnectionError`, `anthropic.APIError`.
127- Implement exponential backoff for rate limit errors. Use `.with_retry()` on chains.
128- Handle model output parsing failures gracefully. Log the raw output for debugging.
129- Set timeouts on all LLM calls. Use `timeout` parameter on model initialization.
130- Use fallback chains: `.with_fallbacks([fallback_chain])` to switch to cheaper/faster models on failure.
131- Return user-friendly error messages. Never expose raw API errors to end users.
132- Log all errors with context: model used, prompt template, input variables, raw response.
133 
134## Evaluation and Testing
135 
136- Use LangSmith for tracing, debugging, and evaluating LLM applications.
137- Create evaluation datasets with input-output pairs for regression testing.
138- Use LLM-as-judge evaluators for subjective quality metrics (relevance, correctness, helpfulness).
139- Test RAG pipelines separately: retrieval accuracy (recall@k) and generation quality.
140- Test edge cases: empty inputs, very long inputs, adversarial prompts, ambiguous queries.
141- Benchmark latency and cost for different model choices. Track cost per request.
142- Use unit tests for non-LLM components (data processing, tool logic, parsers).
143- Use integration tests for full chain execution with recorded/mocked LLM responses.
144 
145## File Structure
146 
147```
148src/
149  chains/
150    rag_chain.py         — RAG pipeline definition
151    analysis_chain.py    — Analysis chain
152    conversational.py    — Conversational chain with memory
153  agents/
154    research_agent.py    — LangGraph agent definition
155    assistant.py         — General assistant agent
156  tools/
157    search.py            — Search tool definitions
158    calculator.py        — Calculator tool
159    database.py          — Database query tool
160  prompts/
161    system_prompts.py    — System prompt templates
162    few_shot_examples.py — Few-shot examples
163  models/
164    schemas.py           — Pydantic models for structured output
165    state.py             — LangGraph state definitions
166  retrieval/
167    vectorstore.py       — Vector store initialization
168    embeddings.py        — Embedding model setup
169    splitters.py         — Document splitting configuration
170  memory/
171    chat_history.py      — Conversation memory setup
172  config/
173    settings.py          — Application settings (BaseSettings)
174    models.py            — Model configuration (name, temp, max_tokens)
175  utils/
176    callbacks.py         — Custom callbacks for logging/streaming
177    token_counter.py     — Token counting utilities
178tests/
179  test_chains.py
180  test_tools.py
181  test_retrieval.py
182  evaluation/
183    datasets/            — Evaluation datasets
184    evaluators.py        — Custom evaluators
185```
186 
187## Security
188 
189- Never expose API keys in code, logs, or error messages. Use environment variables.
190- Sanitize user input before including it in prompts. Prevent prompt injection attacks.
191- Implement input validation: maximum length, content filtering, format checking.
192- Use output filtering to prevent the LLM from generating harmful content.
193- Rate limit end-user requests to prevent abuse and control costs.
194- Log all interactions for auditing. Exclude sensitive user data from logs.
195- Use the minimum token limits needed. Set `max_tokens` on model calls to control costs.
196- Review and sanitize tool outputs before returning them to users.
197- Implement guardrails: content moderation, PII detection, output validation.
198 
199## Performance and Cost
200 
201- Use streaming (`astream`, `astream_events`) for better user experience on long responses.
202- Cache LLM responses for identical inputs with `CacheBackedEmbeddings` or custom caching.
203- Use cheaper, faster models (GPT-4o-mini, Claude Haiku) for simple tasks. Reserve powerful models for complex reasoning.
204- Batch similar requests with `.batch()` for throughput optimization.
205- Monitor token usage per request. Set budgets and alerts.
206- Use async operations throughout for concurrent processing.
207- Profile chain execution with LangSmith to identify bottlenecks.
208- Optimize RAG retrieval: tune chunk size, top-k, and similarity thresholds for your use case.
209- Use model routing: classify the query complexity and route to the appropriate model.
210