| Dimension | Shared | Only in A | Only in B | Overlap |
|---|---|---|---|---|
| Sections | 0 | 12 | 18 | 0% |
| Commands | 4 | 5 | 9 | 22% |
| Section tags | 4 | 3 | 1 | 50% |
What each file covers
Sections
0 shared · 12 only in A · 18 only in B- − Repository Guidelines
- − Managing AI-Generated Planning Documents
- − AI planning documents (ephemeral)
- − Important Rules
- − LLM Reference
- − Project Structure & Module Organization
- − Caching
- − Build, Test, and Development Commands
- − Coding Style & Naming Conventions
- − Testing Guidelines
- − Commit & Pull Request Guidelines
- − Landing the Plane (Session Completion)
- + CLAUDE.md
- + Key Commands
- + Architecture Overview
- + CLI Framework (Kong)
- + Enhance Command
- + Basic usage - enhance all notes in a directory
- + Recursive scan with TMDB content generation
- + Download cover images and generate content
- + Interactive mode for manual TMDB selection
- + Dry run to see what would be enhanced
- + Configuration
- + Development Patterns
- + Caching Patterns
- + Testing Requirements
- + Datasette Integration
- + Logging
- + Output Formats & Handling
- + Important Notes
Commands
4 shared · 5 only in A · 9 only in B- − git pull --rebase
- − git push
- − git status
- − go test -race -coverprofile=coverage/coverage.out ./...
- − go run ./cmd/root.go --help
- + task
- + task clean
- + task upgrade-deps
- + task build-linux
- + task test-ci
- + go run . import goodreads -f goodreads_library_export.csv
- + go run . import goodreads --automated --headful
- + go run . import goodreads --automated --dry-run
- + go run . enhance -d markdown/imdb --tmdb-generate-content
- task build
- task test
- task lint
- go test ./...
Section tags
4 shared · 3 only in A · 1 only in B- − build
- − git-pr
- − do-not
- + agent-behaviour
- test
- lint-format
- code-style
- architecture
Line diff
lepinkainen/hermes · AGENTS.md
@@ −1 @@
1# Repository Guidelines
2
3### Managing AI-Generated Planning Documents
4
5AI assistants often create planning and design documents during development:
6
7- PLAN.md, IMPLEMENTATION.md, ARCHITECTURE.md
8- DESIGN.md, CODEBASE_SUMMARY.md, INTEGRATION_PLAN.md
9- TESTING_GUIDE.md, TECHNICAL_DESIGN.md, and similar files
10
11**Best Practice: Use a dedicated directory for these ephemeral files**
12
13**Recommended approach:**
14
15- Create a `history/` directory in the project root
16- Store ALL AI-generated planning/design docs in `history/`
17- Keep the repository root clean and focused on permanent project files
18- Only access `history/` when explicitly asked to review past planning
19
20**Example .gitignore entry (optional):**
21
22```
23# AI planning documents (ephemeral)
24history/
25```
26
27**Benefits:**
28
29- ✅ Clean repository root
30- ✅ Clear separation between ephemeral and permanent documentation
31- ✅ Easy to exclude from version control if desired
32- ✅ Preserves planning history for archeological research
33- ✅ Reduces noise when browsing the project
34
35### Important Rules
36
37- ✅ Store AI planning docs in `history/` directory
38- ✅ Always run `task build` before claiming work is done
39- ❌ Do NOT clutter repo root with planning documents
40
41### LLM Reference
42
43Need a quick tour of the shared helpers under `internal/`? Read `docs/internal_llm_reference.md` for package-by-package guidance before writing new utilities.
44
45## Project Structure & Module Organization
46
47- `main.go` wires the CLI and dispatches importer subcommands.
48- `cmd/` hosts CLI entrypoints per provider (e.g. `cmd/goodreads`, `cmd/steam`).
49- `internal/` contains shared services: `cache` for local stores, `datastore` for SQLite/JSON writers, `config` for settings.
50- `docs/` is the canonical reference; update it alongside behaviour changes and new flags.
51- Generated build and coverage artifacts live in `build/` and `coverage/`; sample exports under `exports/` and `json/` support local runs but keep large fixtures out of commits.
52
53## Caching
54
55- Hermes caches provider responses in `cache.db` (SQLite) in the repo root; it is safe to delete and is separate from `hermes.db`.
56- Default TTL is `720h` (30 days); override with `--cache-db-file`, `--cache-ttl`, or env vars `CACHE_DBFILE`/`CACHE_TTL`.
57- Tables are created automatically per provider (`omdb_cache`, `openlibrary_cache`, `steam_cache`, `letterboxd_cache`, `tmdb_cache`); entries past TTL refresh on next use and malformed entries are retried.
58- Warm caches by running the relevant importer once; invalidate selectively with `hermes cache invalidate tmdb|omdb|steam|letterboxd|openlibrary` or delete `cache.db` to clear everything.
59- Legacy JSON caches under `cache/` are deprecated and can be removed; negative TMDB results are intentionally not cached to allow future discoveries.
60
61## Build, Test, and Development Commands
62
63- `task build` runs lint, tests, and produces `build/hermes` with the current Git SHA embedded.
64- `task test` executes `go test -race -coverprofile=coverage/coverage.out ./...` and emits `coverage/coverage.html` for review.
65- `task lint` wraps `golangci-lint run ./...`; resolve findings before opening a PR.
66- `go run ./cmd/root.go --help` is a quick sanity check for new flags; swap in a provider folder (e.g. `./cmd/goodreads`) to exercise importer flows.
67
68## Coding Style & Naming Conventions
69
70- Format Go sources with `gofmt` or goimports integrations; Go defaults to tab-indentation, so avoid manual overrides.
71- Keep package names lowercase and singular; exported identifiers use UpperCamelCase, unexported ones use lowerCamelCase.
72- Prefer context-aware logging through the `humanlog` helpers and centralize config lookups in `internal/config` to keep importer packages lean.
73
74## Testing Guidelines
75
76- Co-locate `_test.go` files with the code under test; favour table-driven cases and `testify` assertions for clarity.
77- Run `task test` (or `go test ./...` when iterating) before pushing; inspect `coverage/coverage.html` for critical paths such as `internal/datastore` or importer pipelines.
78- Store lightweight fixtures under package-level `testdata/` directories and avoid reusing the large exports shipped at the repo root.
79
80## Commit & Pull Request Guidelines
81
82- Follow the existing Title-Case, imperative commit style (`Refactor caching`, `Add Steam importer config`) and keep each commit focused.
83- PRs should explain the motivation, list manual verification steps, and link issues; attach screenshots or sample output when behaviour is user-visible.
84- Before requesting review, ensure lint/tests pass, docs in `docs/` reflect the change, and configuration updates reference `config.yml` or `.env` expectations.
85
86## Landing the Plane (Session Completion)
87
88**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.
89
90**MANDATORY WORKFLOW:**
91
921. **Note remaining work** - Capture anything that needs follow-up in the handoff
932. **Run quality gates** (if code changed) - Tests, linters, builds
943. **PUSH TO REMOTE** - This is MANDATORY:
95 ```bash
96 git pull --rebase
97 git push
98 git status # MUST show "up to date with origin"
99 ```
1004. **Clean up** - Clear stashes, prune remote branches
1015. **Verify** - All changes committed AND pushed
1026. **Hand off** - Provide context for next session
103
104**CRITICAL RULES:**
105- Work is NOT complete until `git push` succeeds
106- NEVER stop before pushing - that leaves work stranded locally
107- NEVER say "ready to push when you are" - YOU must push
108- If push fails, resolve and retry until it succeeds
109
lepinkainen/hermes · CLAUDE.md
@@ +1 @@
1# CLAUDE.md
2
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
5Refer to llm-shared/project_tech_stack.md for core technology choices, build system configuration, and library preferences.
6
7## Key Commands
8
9- `task` or `task build` - Build the application (runs tests and lint first)
10- `task test` - Run tests with coverage report (generates coverage/coverage.html)
11- `task lint` - Run golangci-lint
12- `task clean` - Clean build artifacts
13- `task upgrade-deps` - Update all dependencies
14- `task build-linux` - Cross-compile for Linux
15- `task test-ci` - Run tests for CI (with ci build tag)
16
17**Running the application:**
18
19- `./build/hermes --help` - View available commands
20- `./build/hermes import goodreads --help` - View importer-specific options
21- `./build/hermes enhance --help` - View enhance command options
22- `go run . import goodreads -f goodreads_library_export.csv` - Run directly without building
23- `go run . import goodreads --automated --headful` - Automated Goodreads export download (Chrome required)
24- `go run . import goodreads --automated --dry-run` - Test automation without import
25- `go run . enhance -d markdown/imdb --tmdb-generate-content` - Enhance existing notes with TMDB data
26
27**Development workflow:**
28
29- Tests must pass before builds (enforced by Taskfile dependencies)
30- Always run `goimports -w .` after modifying Go files
31- Use `go test ./...` for quick test runs during development
32
33## Architecture Overview
34
35Hermes is a data import/export tool that converts exports from various sources (Goodreads, IMDb, Letterboxd, Steam) into unified formats (JSON, Markdown, SQLite/Datasette). It also provides an `enhance` command to enrich existing markdown notes with TMDB data.
36
37**Key Components:**
38
39- `cmd/root.go` - Kong-based CLI structure with nested commands
40- `cmd/{source}/` - Each data source has its own package (goodreads, imdb, letterboxd, steam)
41- `cmd/enhance/` - Enhance existing markdown notes with TMDB data
42- `internal/` - Shared utilities and packages
43 - `cache/` - API response caching
44 - `cmdutil/` - Command setup helpers
45 - `config/` - Global configuration management
46 - `datastore/` - SQLite/Datasette integration
47 - `enrichment/` - TMDB enrichment functionality
48 - `errors/` - Custom error types
49 - `fileutil/` - File operations, markdown/JSON utilities
50 - `tmdb/` - TMDB API client
51 - `tui/` - Interactive terminal UI for TMDB selection
52
53**Standard Importer Structure:**
54
55```plain
56cmd/{source}/
57├── cmd.go # Command setup and execution
58├── parser.go # Input data parsing
59├── types.go # Data models
60├── {api}.go # External API integration (e.g., omdb.go, openlibrary.go)
61├── cache.go # API response caching
62├── json.go # JSON output formatting
63├── markdown.go # Markdown output formatting
64├── testdata/ # Test fixtures and expected outputs
65└── *_test.go # Unit tests
66```
67
68## CLI Framework (Kong)
69
70- Uses Kong for command-line parsing (defined in `cmd/root.go`)
71- CLI structure: `CLI` struct contains global flags and `ImportCmd` with subcommands
72- Each importer command (GoodreadsCmd, IMDBCmd, etc.) implements a `Run() error` method
73- Kong automatically handles help text, usage messages, and error handling
74- Commands read from config file if CLI flags not provided (CLI flags take precedence)
75
76**Adding a new importer command:**
77
781. Add command struct to `cmd/root.go` (e.g., `type NewSourceCmd struct`)
792. Add struct field to `ImportCmd` with cmd and help tags
803. Implement `Run() error` method that calls the importer package
814. Follow pattern of reading from config with CLI flag override
82
83## Enhance Command
84
85The `enhance` command enriches existing markdown notes with TMDB data without re-importing from original sources.
86
87**Usage:**
88
89```bash
90# Basic usage - enhance all notes in a directory
91./build/hermes enhance -d markdown/imdb
92
93# Recursive scan with TMDB content generation
94./build/hermes enhance -d markdown/letterboxd -r --tmdb-generate-content
95
96# Download cover images and generate content
97./build/hermes enhance -d markdown/imdb --tmdb-download-cover --tmdb-generate-content
98
99# Interactive mode for manual TMDB selection
100./build/hermes enhance -d markdown/letterboxd --tmdb-interactive
101
102# Dry run to see what would be enhanced
103./build/hermes enhance -d markdown/imdb --dry-run
104```
105
106**Key Features:**
107
108- Scans directory for markdown files with YAML frontmatter
109- Extracts title, year, and IMDB ID from existing notes
110- Searches TMDB for matching content
111- Updates frontmatter with TMDB ID, runtime, genres, etc.
112- Optionally downloads cover images
113- Optionally generates TMDB content sections (cast, crew, similar titles, etc.)
114- Supports interactive TUI for selecting from multiple TMDB matches
115- Skips notes that already have TMDB data (unless `--overwrite` flag is used)
116- Dry-run mode to preview changes without modifying files
117
118**Implementation:**
119
120- `cmd/enhance/cmd.go` - Command logic, file discovery, and processing
121- `cmd/enhance/parser.go` - YAML frontmatter parsing and markdown rebuilding
122- Uses `internal/enrichment` for TMDB API integration
123- Leverages existing TMDB client and TUI components
124
125## Configuration
126
127- Primary config: `config.yaml` (YAML format, auto-generated on first run)
128- CLI flags override config file values
129- Global settings in `cmd/root.go`: output directories, overwrite flag, datasette config
130- Command-specific config uses namespaced keys (e.g., `goodreads.csvfile`, `steam.apikey`)
131- Viper manages config loading with defaults in `initConfig()`
132
133## Development Patterns
134
135**Adding New Importers:**
136
1371. Create new package under `cmd/{source}/`
1382. Implement standard structure (cmd.go, parser.go, types.go, etc.)
1393. Add command struct and Run() method to `cmd/root.go`
1404. Add struct field to `ImportCmd` in `cmd/root.go`
1415. Follow existing patterns for API integration, caching, and output formatting
1426. Add tests in `testdata/` subdirectory
143
144**Common Utilities:**
145
146- `internal/cmdutil` - Command setup helpers
147- `internal/fileutil` - File operations, markdown/JSON utilities
148- `internal/config` - Global configuration management
149- `internal/datastore` - SQLite/Datasette integration
150- `internal/cache` - API response caching
151- `internal/errors` - Custom error types (e.g., RateLimitError)
152
153**API Integration:**
154
155- All API responses cached in SQLite database (`cache.db` in project root, configurable via `cache.dbfile`)
156- Cache TTL defaults to 720h (30 days), configurable via `cache.ttl`
157- Implement API client logic within relevant command package (e.g., `cmd/goodreads/openlibrary.go`)
158- Respect API rate limits using delays (`time.Sleep`) or by handling specific rate limit errors
159- Handle common API errors gracefully (log warnings for 404s, retry or fail on persistent errors)
160- Use existing patterns from OMDB or OpenLibrary integration
161
162**Error Handling:**
163
164- Use standard Go error handling (`errors.New`, `fmt.Errorf`)
165- Return errors up the call stack for handling by Kong's command execution
166- Wrap errors with context: `fmt.Errorf("failed to process item %s: %w", itemID, err)`
167- Use custom error types from `internal/errors/` for specific conditions
168- Log significant errors but return them to let top level handle exit codes
169
170## Caching Patterns
171
172Use the appropriate caching strategy from `internal/cache`:
173
174- **`cache.GetOrFetch()`** - Cache all responses with global TTL (default 30 days)
175 - Use for: Steam games, TMDB details by ID
176 - Example: `cache.GetOrFetch("steam_cache", appID, fetchFunc)`
177
178- **`cache.GetOrFetchWithPolicy()`** - Cache only certain responses (conditional)
179 - Use for: TMDB searches (don't cache empty results)
180 - Example: `cache.GetOrFetchWithPolicy("tmdb_cache", key, fetchFunc, shouldCache)`
181
182- **`cache.GetOrFetchWithTTL()`** - Different TTLs for different result types (negative caching)
183 - Use for: Goodreads books (7 days for "not found", 30 days for successful)
184 - Helper: `cache.SelectNegativeCacheTTL(func(r *CachedResult) bool { return r.NotFound })`
185 - Example: See `cmd/goodreads/cache.go` for reference implementation
186
187**When adding new cached operations:**
1881. Choose appropriate strategy (GetOrFetch, GetOrFetchWithPolicy, or GetOrFetchWithTTL)
1892. Design deterministic cache keys (normalize if needed)
1903. Add table schema to `internal/cache/schema.go`
1914. Add table name to `ValidCacheTableNames` map
1925. Add source to cache invalidation in `internal/cache/cmd.go`
193
194**For detailed guidance:**
195- Developer guide: `docs/cache-architecture.md`
196- User documentation: `docs/caching.md`
197- Architecture decisions: `docs/decisions/001-cache-architecture.md`
198
199## Testing Requirements
200
201- Write unit tests for parsing logic, API interaction, and output generation
202- Test files in same package with `_test.go` suffix
203- Tests must pass before builds (enforced by Taskfile)
204- Use `testdata/` subdirectory within each command package for fixtures
205- Employ table-driven tests for validating multiple input cases
206- Use `//go:build !ci` to skip tests in CI that require external dependencies
207
208## Datasette Integration
209
210- Supports both local SQLite and remote Datasette storage
211- `internal/datastore` provides unified interface
212- Configuration under `datasette:` key in config (enabled, mode, dbfile, remote_url, api_token)
213- Local mode writes to `hermes.db`, remote mode uses API
214- Enable via `--datasette` flag or config file
215
216## Logging
217
218- Uses `log/slog` with custom handler from `github.com/lepinkainen/humanlog`
219- Initialized in `cmd/root.go` with `slog.LevelInfo` default
220- `InfoLevel` for progress messages (starting import, items processed)
221- `DebugLevel` for verbose debugging info (API request/response details, cache hits/misses)
222- `WarnLevel` for recoverable issues (skipping items but continuing)
223- `ErrorLevel` for significant problems before returning an error
224
225## Output Formats & Handling
226
227- Default output directories (`markdown/`, `json/`) set in `cmd/root.go`, configurable via `config.yaml`
228- Commands allow specifying subdirectories for output via `-o` flag (e.g., `markdown/goodreads/`)
229- Use `internal/fileutil` for writing files (WriteMarkdownFile for Markdown, WriteJSONFile for JSON); compose markdown notes with `internal/obsidian` (Frontmatter, TagSet, BuildNoteMarkdown)
230- Markdown files use YAML frontmatter (Obsidian-compatible)
231- Follow existing patterns for Markdown frontmatter and JSON structure for each data type
232- Respect `--overwrite` flag logic
233
234## Important Notes
235
236- Go-only project (Go 1.24+)
237- Follows standard Go project layout and idioms
238- Each importer handles its own data enrichment and API integration
239- Use `goimports -w .` after making changes (not gofmt)
240- Use shared utility functions from `internal/` packages
241- Build artifacts go to `build/` directory
242- Cache stored in SQLite database (`cache.db`, safe to delete for cache invalidation)
243
@@ −1 +1 @@
1−# Repository Guidelines
1+# CLAUDE.md
22
3−### Managing AI-Generated Planning Documents
3+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
44
5−AI assistants often create planning and design documents during development:
5+Refer to llm-shared/project_tech_stack.md for core technology choices, build system configuration, and library preferences.
66
7−- PLAN.md, IMPLEMENTATION.md, ARCHITECTURE.md
8−- DESIGN.md, CODEBASE_SUMMARY.md, INTEGRATION_PLAN.md
9−- TESTING_GUIDE.md, TECHNICAL_DESIGN.md, and similar files
7+## Key Commands
108
11−**Best Practice: Use a dedicated directory for these ephemeral files**
9+- `task` or `task build` - Build the application (runs tests and lint first)
10+- `task test` - Run tests with coverage report (generates coverage/coverage.html)
11+- `task lint` - Run golangci-lint
12+- `task clean` - Clean build artifacts
13+- `task upgrade-deps` - Update all dependencies
14+- `task build-linux` - Cross-compile for Linux
15+- `task test-ci` - Run tests for CI (with ci build tag)
1216
13−**Recommended approach:**
17+**Running the application:**
1418
15−- Create a `history/` directory in the project root
16−- Store ALL AI-generated planning/design docs in `history/`
17−- Keep the repository root clean and focused on permanent project files
18−- Only access `history/` when explicitly asked to review past planning
19+- `./build/hermes --help` - View available commands
20+- `./build/hermes import goodreads --help` - View importer-specific options
21+- `./build/hermes enhance --help` - View enhance command options
22+- `go run . import goodreads -f goodreads_library_export.csv` - Run directly without building
23+- `go run . import goodreads --automated --headful` - Automated Goodreads export download (Chrome required)
24+- `go run . import goodreads --automated --dry-run` - Test automation without import
25+- `go run . enhance -d markdown/imdb --tmdb-generate-content` - Enhance existing notes with TMDB data
1926
20−**Example .gitignore entry (optional):**
27+**Development workflow:**
2128
29+- Tests must pass before builds (enforced by Taskfile dependencies)
30+- Always run `goimports -w .` after modifying Go files
31+- Use `go test ./...` for quick test runs during development
32+
33+## Architecture Overview
34+
35+Hermes is a data import/export tool that converts exports from various sources (Goodreads, IMDb, Letterboxd, Steam) into unified formats (JSON, Markdown, SQLite/Datasette). It also provides an `enhance` command to enrich existing markdown notes with TMDB data.
36+
37+**Key Components:**
38+
39+- `cmd/root.go` - Kong-based CLI structure with nested commands
40+- `cmd/{source}/` - Each data source has its own package (goodreads, imdb, letterboxd, steam)
41+- `cmd/enhance/` - Enhance existing markdown notes with TMDB data
42+- `internal/` - Shared utilities and packages
43+ - `cache/` - API response caching
44+ - `cmdutil/` - Command setup helpers
45+ - `config/` - Global configuration management
46+ - `datastore/` - SQLite/Datasette integration
47+ - `enrichment/` - TMDB enrichment functionality
48+ - `errors/` - Custom error types
49+ - `fileutil/` - File operations, markdown/JSON utilities
50+ - `tmdb/` - TMDB API client
51+ - `tui/` - Interactive terminal UI for TMDB selection
52+
53+**Standard Importer Structure:**
54+
55+```plain
56+cmd/{source}/
57+├── cmd.go # Command setup and execution
58+├── parser.go # Input data parsing
59+├── types.go # Data models
60+├── {api}.go # External API integration (e.g., omdb.go, openlibrary.go)
61+├── cache.go # API response caching
62+├── json.go # JSON output formatting
63+├── markdown.go # Markdown output formatting
64+├── testdata/ # Test fixtures and expected outputs
65+└── *_test.go # Unit tests
2266 ```
23−# AI planning documents (ephemeral)
24−history/
67+
68+## CLI Framework (Kong)
69+
70+- Uses Kong for command-line parsing (defined in `cmd/root.go`)
71+- CLI structure: `CLI` struct contains global flags and `ImportCmd` with subcommands
72+- Each importer command (GoodreadsCmd, IMDBCmd, etc.) implements a `Run() error` method
73+- Kong automatically handles help text, usage messages, and error handling
74+- Commands read from config file if CLI flags not provided (CLI flags take precedence)
75+
76+**Adding a new importer command:**
77+
78+1. Add command struct to `cmd/root.go` (e.g., `type NewSourceCmd struct`)
79+2. Add struct field to `ImportCmd` with cmd and help tags
80+3. Implement `Run() error` method that calls the importer package
81+4. Follow pattern of reading from config with CLI flag override
82+
83+## Enhance Command
84+
85+The `enhance` command enriches existing markdown notes with TMDB data without re-importing from original sources.
86+
87+**Usage:**
88+
89+```bash
90+# Basic usage - enhance all notes in a directory
91+./build/hermes enhance -d markdown/imdb
92+
93+# Recursive scan with TMDB content generation
94+./build/hermes enhance -d markdown/letterboxd -r --tmdb-generate-content
95+
96+# Download cover images and generate content
97+./build/hermes enhance -d markdown/imdb --tmdb-download-cover --tmdb-generate-content
98+
99+# Interactive mode for manual TMDB selection
100+./build/hermes enhance -d markdown/letterboxd --tmdb-interactive
101+
102+# Dry run to see what would be enhanced
103+./build/hermes enhance -d markdown/imdb --dry-run
25104 ```
26105
27−**Benefits:**
106+**Key Features:**
28107
29−- ✅ Clean repository root
30−- ✅ Clear separation between ephemeral and permanent documentation
31−- ✅ Easy to exclude from version control if desired
32−- ✅ Preserves planning history for archeological research
33−- ✅ Reduces noise when browsing the project
108+- Scans directory for markdown files with YAML frontmatter
109+- Extracts title, year, and IMDB ID from existing notes
110+- Searches TMDB for matching content
111+- Updates frontmatter with TMDB ID, runtime, genres, etc.
112+- Optionally downloads cover images
113+- Optionally generates TMDB content sections (cast, crew, similar titles, etc.)
114+- Supports interactive TUI for selecting from multiple TMDB matches
115+- Skips notes that already have TMDB data (unless `--overwrite` flag is used)
116+- Dry-run mode to preview changes without modifying files
34117
35−### Important Rules
118+**Implementation:**
36119
37−- ✅ Store AI planning docs in `history/` directory
38−- ✅ Always run `task build` before claiming work is done
39−- ❌ Do NOT clutter repo root with planning documents
120+- `cmd/enhance/cmd.go` - Command logic, file discovery, and processing
121+- `cmd/enhance/parser.go` - YAML frontmatter parsing and markdown rebuilding
122+- Uses `internal/enrichment` for TMDB API integration
123+- Leverages existing TMDB client and TUI components
40124
41−### LLM Reference
125+## Configuration
42126
43−Need a quick tour of the shared helpers under `internal/`? Read `docs/internal_llm_reference.md` for package-by-package guidance before writing new utilities.
127+- Primary config: `config.yaml` (YAML format, auto-generated on first run)
128+- CLI flags override config file values
129+- Global settings in `cmd/root.go`: output directories, overwrite flag, datasette config
130+- Command-specific config uses namespaced keys (e.g., `goodreads.csvfile`, `steam.apikey`)
131+- Viper manages config loading with defaults in `initConfig()`
44132
45−## Project Structure & Module Organization
133+## Development Patterns
46134
47−- `main.go` wires the CLI and dispatches importer subcommands.
48−- `cmd/` hosts CLI entrypoints per provider (e.g. `cmd/goodreads`, `cmd/steam`).
49−- `internal/` contains shared services: `cache` for local stores, `datastore` for SQLite/JSON writers, `config` for settings.
50−- `docs/` is the canonical reference; update it alongside behaviour changes and new flags.
51−- Generated build and coverage artifacts live in `build/` and `coverage/`; sample exports under `exports/` and `json/` support local runs but keep large fixtures out of commits.
135+**Adding New Importers:**
52136
53−## Caching
137+1. Create new package under `cmd/{source}/`
138+2. Implement standard structure (cmd.go, parser.go, types.go, etc.)
139+3. Add command struct and Run() method to `cmd/root.go`
140+4. Add struct field to `ImportCmd` in `cmd/root.go`
141+5. Follow existing patterns for API integration, caching, and output formatting
142+6. Add tests in `testdata/` subdirectory
54143
55−- Hermes caches provider responses in `cache.db` (SQLite) in the repo root; it is safe to delete and is separate from `hermes.db`.
56−- Default TTL is `720h` (30 days); override with `--cache-db-file`, `--cache-ttl`, or env vars `CACHE_DBFILE`/`CACHE_TTL`.
57−- Tables are created automatically per provider (`omdb_cache`, `openlibrary_cache`, `steam_cache`, `letterboxd_cache`, `tmdb_cache`); entries past TTL refresh on next use and malformed entries are retried.
58−- Warm caches by running the relevant importer once; invalidate selectively with `hermes cache invalidate tmdb|omdb|steam|letterboxd|openlibrary` or delete `cache.db` to clear everything.
59−- Legacy JSON caches under `cache/` are deprecated and can be removed; negative TMDB results are intentionally not cached to allow future discoveries.
144+**Common Utilities:**
60145
61−## Build, Test, and Development Commands
146+- `internal/cmdutil` - Command setup helpers
147+- `internal/fileutil` - File operations, markdown/JSON utilities
148+- `internal/config` - Global configuration management
149+- `internal/datastore` - SQLite/Datasette integration
150+- `internal/cache` - API response caching
151+- `internal/errors` - Custom error types (e.g., RateLimitError)
62152
63−- `task build` runs lint, tests, and produces `build/hermes` with the current Git SHA embedded.
64−- `task test` executes `go test -race -coverprofile=coverage/coverage.out ./...` and emits `coverage/coverage.html` for review.
65−- `task lint` wraps `golangci-lint run ./...`; resolve findings before opening a PR.
66−- `go run ./cmd/root.go --help` is a quick sanity check for new flags; swap in a provider folder (e.g. `./cmd/goodreads`) to exercise importer flows.
153+**API Integration:**
67154
68−## Coding Style & Naming Conventions
155+- All API responses cached in SQLite database (`cache.db` in project root, configurable via `cache.dbfile`)
156+- Cache TTL defaults to 720h (30 days), configurable via `cache.ttl`
157+- Implement API client logic within relevant command package (e.g., `cmd/goodreads/openlibrary.go`)
158+- Respect API rate limits using delays (`time.Sleep`) or by handling specific rate limit errors
159+- Handle common API errors gracefully (log warnings for 404s, retry or fail on persistent errors)
160+- Use existing patterns from OMDB or OpenLibrary integration
69161
70−- Format Go sources with `gofmt` or goimports integrations; Go defaults to tab-indentation, so avoid manual overrides.
71−- Keep package names lowercase and singular; exported identifiers use UpperCamelCase, unexported ones use lowerCamelCase.
72−- Prefer context-aware logging through the `humanlog` helpers and centralize config lookups in `internal/config` to keep importer packages lean.
162+**Error Handling:**
73163
74−## Testing Guidelines
164+- Use standard Go error handling (`errors.New`, `fmt.Errorf`)
165+- Return errors up the call stack for handling by Kong's command execution
166+- Wrap errors with context: `fmt.Errorf("failed to process item %s: %w", itemID, err)`
167+- Use custom error types from `internal/errors/` for specific conditions
168+- Log significant errors but return them to let top level handle exit codes
75169
76−- Co-locate `_test.go` files with the code under test; favour table-driven cases and `testify` assertions for clarity.
77−- Run `task test` (or `go test ./...` when iterating) before pushing; inspect `coverage/coverage.html` for critical paths such as `internal/datastore` or importer pipelines.
78−- Store lightweight fixtures under package-level `testdata/` directories and avoid reusing the large exports shipped at the repo root.
170+## Caching Patterns
79171
80−## Commit & Pull Request Guidelines
172+Use the appropriate caching strategy from `internal/cache`:
81173
82−- Follow the existing Title-Case, imperative commit style (`Refactor caching`, `Add Steam importer config`) and keep each commit focused.
83−- PRs should explain the motivation, list manual verification steps, and link issues; attach screenshots or sample output when behaviour is user-visible.
84−- Before requesting review, ensure lint/tests pass, docs in `docs/` reflect the change, and configuration updates reference `config.yml` or `.env` expectations.
174+- **`cache.GetOrFetch()`** - Cache all responses with global TTL (default 30 days)
175+ - Use for: Steam games, TMDB details by ID
176+ - Example: `cache.GetOrFetch("steam_cache", appID, fetchFunc)`
85177
86−## Landing the Plane (Session Completion)
178+- **`cache.GetOrFetchWithPolicy()`** - Cache only certain responses (conditional)
179+ - Use for: TMDB searches (don't cache empty results)
180+ - Example: `cache.GetOrFetchWithPolicy("tmdb_cache", key, fetchFunc, shouldCache)`
87181
88−**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.
182+- **`cache.GetOrFetchWithTTL()`** - Different TTLs for different result types (negative caching)
183+ - Use for: Goodreads books (7 days for "not found", 30 days for successful)
184+ - Helper: `cache.SelectNegativeCacheTTL(func(r *CachedResult) bool { return r.NotFound })`
185+ - Example: See `cmd/goodreads/cache.go` for reference implementation
89186
90−**MANDATORY WORKFLOW:**
187+**When adding new cached operations:**
188+1. Choose appropriate strategy (GetOrFetch, GetOrFetchWithPolicy, or GetOrFetchWithTTL)
189+2. Design deterministic cache keys (normalize if needed)
190+3. Add table schema to `internal/cache/schema.go`
191+4. Add table name to `ValidCacheTableNames` map
192+5. Add source to cache invalidation in `internal/cache/cmd.go`
91193
92−1. **Note remaining work** - Capture anything that needs follow-up in the handoff
93−2. **Run quality gates** (if code changed) - Tests, linters, builds
94−3. **PUSH TO REMOTE** - This is MANDATORY:
95− ```bash
96− git pull --rebase
97− git push
98− git status # MUST show "up to date with origin"
99− ```
100−4. **Clean up** - Clear stashes, prune remote branches
101−5. **Verify** - All changes committed AND pushed
102−6. **Hand off** - Provide context for next session
194+**For detailed guidance:**
195+- Developer guide: `docs/cache-architecture.md`
196+- User documentation: `docs/caching.md`
197+- Architecture decisions: `docs/decisions/001-cache-architecture.md`
103198
104−**CRITICAL RULES:**
105−- Work is NOT complete until `git push` succeeds
106−- NEVER stop before pushing - that leaves work stranded locally
107−- NEVER say "ready to push when you are" - YOU must push
108−- If push fails, resolve and retry until it succeeds
199+## Testing Requirements
200+
201+- Write unit tests for parsing logic, API interaction, and output generation
202+- Test files in same package with `_test.go` suffix
203+- Tests must pass before builds (enforced by Taskfile)
204+- Use `testdata/` subdirectory within each command package for fixtures
205+- Employ table-driven tests for validating multiple input cases
206+- Use `//go:build !ci` to skip tests in CI that require external dependencies
207+
208+## Datasette Integration
209+
210+- Supports both local SQLite and remote Datasette storage
211+- `internal/datastore` provides unified interface
212+- Configuration under `datasette:` key in config (enabled, mode, dbfile, remote_url, api_token)
213+- Local mode writes to `hermes.db`, remote mode uses API
214+- Enable via `--datasette` flag or config file
215+
216+## Logging
217+
218+- Uses `log/slog` with custom handler from `github.com/lepinkainen/humanlog`
219+- Initialized in `cmd/root.go` with `slog.LevelInfo` default
220+- `InfoLevel` for progress messages (starting import, items processed)
221+- `DebugLevel` for verbose debugging info (API request/response details, cache hits/misses)
222+- `WarnLevel` for recoverable issues (skipping items but continuing)
223+- `ErrorLevel` for significant problems before returning an error
224+
225+## Output Formats & Handling
226+
227+- Default output directories (`markdown/`, `json/`) set in `cmd/root.go`, configurable via `config.yaml`
228+- Commands allow specifying subdirectories for output via `-o` flag (e.g., `markdown/goodreads/`)
229+- Use `internal/fileutil` for writing files (WriteMarkdownFile for Markdown, WriteJSONFile for JSON); compose markdown notes with `internal/obsidian` (Frontmatter, TagSet, BuildNoteMarkdown)
230+- Markdown files use YAML frontmatter (Obsidian-compatible)
231+- Follow existing patterns for Markdown frontmatter and JSON structure for each data type
232+- Respect `--overwrite` flag logic
233+
234+## Important Notes
235+
236+- Go-only project (Go 1.24+)
237+- Follows standard Go project layout and idioms
238+- Each importer handles its own data enrichment and API integration
239+- Use `goimports -w .` after making changes (not gofmt)
240+- Use shared utility functions from `internal/` packages
241+- Build artifacts go to `build/` directory
242+- Cache stored in SQLite database (`cache.db`, safe to delete for cache invalidation)
109243
