AGENTS.md

1# @second-brain/ingestion — Agent Instructions
2 
3LLM extraction and embedding pipeline for auto-growing the knowledge graph.
4 
5## Entry Point
6 
7`src/index.ts` re-exports the pipeline and extraction modules.
8 
9## Key Directories
10 
11| Directory | Purpose |
12|-----------|---------|
13| `src/extraction/` | LLM-based entity/relation extraction from text |
14| `src/pipeline/` | Orchestrates extraction → embedding → storage |
15| `src/net/` | HTTP/API client helpers for LLM providers |
16 
17## Key Files
18 
19- `src/content-hash.ts` — Deduplication via content hashing (avoids re-processing)
20 
21## How It Works
22 
231. Raw content arrives (from collectors or manual input)
242. Content is hashed for dedup (`content-hash.ts`)
253. LLM extracts entities and relations (`src/extraction/`)
264. Optionally generates embeddings for vector search
275. Results are stored via `@second-brain/core`
28 
29## Conventions
30 
31- All LLM calls go through the extraction module
32- Content hashing prevents duplicate processing
33- Embedding dimensions must match what `Brain` was constructed with
34