CLAUDE.md

1# CLAUDE.md — Cost Optimized RAG
2 
3## What this project is
4A cost-optimized RAG pipeline for BSAN 765 group project. Classifies query complexity and uses compression to reduce LLM token usage before routing to the appropriate model tier.
5 
6## Current Status
7- ✅ Adaptive Retrieval — DONE (Karthik) — `src/adaptive_retriever.py`
8- ✅ Context Compression — DONE (Anh) — `src/context_compression.py`
9- ✅ Query Analyzer — IMPLEMENTED (rule + heuristic + FLAN fallback) — `src/query_analyzer.py`
10- ✅ Confidence Checker — IMPLEMENTED (multi-signal confidence + model retry) — `src/confidence_checker.py`
11- ✅ Pipeline Wiring — IMPLEMENTED (end-to-end in `src/pipeline.py`)
12- ✅ Model Router — UPDATED (simple/medium → local OpenAI-compatible API; complex → OpenAI) — `src/model_router.py`
13 
14## Pipeline Flow
15```
16User Query
17    ↓
18Query Analyzer (complexity_score + confidence + label)
19    ↓
20Adaptive Retrieval (continuous k from score, confidence-aware, coverage retry)
21    ↓
22Context Compression (adaptive sentence budget + hybrid TF-IDF / local BGE embeddings + redundancy pruning + answer-type boosts; optional fixed caps via env)
23    ↓
24Model Router (simple→local LM Studio model / medium→local LM Studio model / complex→OpenAI, e.g. gpt-4o-mini)
25    ↓
26Confidence Checker (retry with stronger model if confidence low)
27    ↓
28Final Answer
29```
30 
31## What Was Implemented (Latest Work)
32- Query analyzer upgraded from label-only to score-first output:
33  - `complexity_score` (0-1) as primary signal
34  - keeps `complexity_label` (`simple|medium|complex`) for compatibility
35  - includes `confidence`, `source`, `reason_codes`, `llm_status`
36- Hybrid query classification now in place:
37  - baseline: rule + heuristic scoring
38  - fallback: Hugging Face FLAN (`google/flan-t5-large`) for low-confidence cases
39  - guardrails: timeout, strict JSON parsing, fallback to heuristic on failure
40- Adaptive retriever upgraded to dynamic behavior:
41  - score-based `k_base = round(3 + complexity_score * 7)` (capped)
42  - low-confidence safety bump before retrieval
43  - keyword coverage check after retrieval
44  - one re-retrieval step when coverage is below complexity-aware threshold
45  - retry step tuned to `+1` and confidence-gated (`confidence < 0.75`)
46- Retrieval metadata now logged:
47  - `complexity_score_used`, `analyzer_confidence`
48  - `k_base`, `k_final`
49  - `coverage_score`, `coverage_threshold_used`
50  - `retrieval_retry`
51- Evaluation output expanded in `evaluate.py`:
52  - includes analyzer/retrieval tuning fields in CSV/JSON
53  - prints FLAN invocation/timeouts/errors
54  - prints retrieval retry counts and average coverage
55 
56## Current Tuned Retrieval Defaults
57- Complexity-aware coverage thresholds:
58  - simple: `0.45`
59  - medium: `0.55`
60  - complex: `0.60`
61- Retry step: `+1` (capped)
62- Retry confidence gate: retry only when analyzer confidence is below `0.75` (or missing)
63 
64## Last Validation Snapshot
65- Retrieval-only evaluation on 50 test queries:
66  - retry rate improved from `20/50` to `10/50`
67  - average coverage remained stable (~`0.63`)
68  - average final `k` decreased after tuning
69- Full end-to-end `evaluate.py`: **complex** queries need a valid `OPENAI_API_KEY`; **simple/medium** need LM Studio (or compatible) running with models loaded. See `.env.example`.
70 
71## Key Technical Decisions
72- Vector store: ChromaDB (local)
73- Dataset: Notre Dame documents
74- Test set: 50 queries in `data/test_queries.csv`
75- Model tiers: **simple/medium** → local OpenAI-compatible endpoint (`LOCAL_OPENAI_BASE_URL`, model IDs from `/v1/models`); **complex** → `OPENAI_COMPLEX_MODEL` (default `gpt-4o-mini`) via OpenAI API
76- Query analyzer strategy: heuristic-first, FLAN fallback for low confidence
77- Retrieval strategy: score-first continuous `k` with quality-aware retry
78 
79## File Structure
80```
81cost-optimized-rag/
82├── src/
83│   ├── adaptive_retriever.py     ✅ done
84│   ├── context_compression.py    ✅ done
85│   ├── model_router.py           ✅ local + OpenAI hybrid
86│   ├── confidence_checker.py     ✅ done
87│   ├── query_analyzer.py         ✅ done
88│   └── pipeline.py              ✅ done (orchestrator)
89├── data/
90│   ├── documents/
91│   └── test_queries.csv
92├── chroma_db/
93├── test_pipeline.py
94├── run_all_queries.py
95├── evaluate.py
96├── src/pipeline.py
97└── requirements.txt
98```
99 
100## How to Run
101```bash
102source venv/bin/activate
103python test_pipeline.py        # test single query
104python run_all_queries.py      # test all 50 queries
105python evaluate.py             # full evaluation (local server + OpenAI key for complex tier)
106```
107 
108## Rules
109- Never commit `.env`
110- Always `git pull` before starting work
111- Only `git add` your own file — never `git add .`
112 
113## Last Updated
114April 24, 2026 (see appended **Updates — Apr 24, 2026** sections below for the full chronological log; latest block: **Updates — Apr 24, 2026 (compression ↔ retrieval, metrics naming, hygiene, router decision)**)
115 
116## Updates — Apr 24, 2026
117- Moved pipeline orchestrator to `src/pipeline.py` and updated imports (`evaluate.py` now imports from `src.pipeline`).
118- Removed unused compatibility wrapper `src/adaptive_retrieval.py` to avoid duplicate orchestration files.
119- Added minimum useful stage observability in pipeline/evaluation:
120  - per-stage latency (`query_analyzer_ms`, `retrieval_ms`, `compression_ms`, `model_router_ms`, `confidence_checker_ms`)
121  - end-to-end latency (`total_pipeline_ms`)
122  - persisted to CSV/JSON through `evaluate.py`
123- Ran and stored OpenAI baseline run artifacts:
124  - `results/openai_v1/run_output.log`
125  - `results/openai_v1/evaluation_results_openai_v1.csv`
126  - `results/openai_v1/evaluation_results_openai_v1.json`
127  - `results/openai_v1/run_config.json`
128- Added experiment versioning structure under `results/`:
129  - `results/README.md`
130  - `results/openai_v2_hybrid_context/run_config.template.json`
131  - `results/local_v1/run_config.template.json`
132  - `results/local_v2_hybrid_context/run_config.template.json`
133  - `results/compare_runs.py` (delta comparison helper for two run CSVs)
134- Ran a seed-variation stability run (`seed=7`) and stored artifacts:
135  - `results/openai_v1_seed7/run_output.log`
136  - `results/openai_v1_seed7/evaluation_results_openai_v1_seed7.csv`
137  - `results/openai_v1_seed7/evaluation_results_openai_v1_seed7.json`
138  - `results/openai_v1_seed7/run_config.json`
139- Seed stability comparison (`openai_v1` vs `openai_v1_seed7`):
140  - accuracy unchanged (`56.67%`)
141  - retrieval retry rate unchanged (`20%`)
142  - coverage unchanged (`0.6338`)
143  - timing varied moderately, expected for network-bound LLM calls
144 
145## Updates — Apr 24, 2026 (later session)
146 
147### Model router (local + OpenAI hybrid)
148- `src/model_router.py` uses **LangChain `ChatOpenAI`** against:
149  - **Local**: `LOCAL_OPENAI_BASE_URL` (default `http://localhost:1234/v1` for LM Studio) + `LOCAL_OPENAI_API_KEY` (default `lm-studio`).
150  - **OpenAI** (complex only): standard OpenAI credentials; model from `OPENAI_COMPLEX_MODEL`.
151- **Tier mapping**: `simple` → `LOCAL_SIMPLE_MODEL`, `medium` → `LOCAL_MEDIUM_MODEL`, `complex` → `OPENAI_COMPLEX_MODEL`.
152- **Important**: local model names must match IDs returned by `GET {base}/models` (e.g. `tinyllama-1.1b-chat-v1.0`, `mistralai/ministral-3-3b`), not short aliases.
153- Optional **health check** before local calls: `LOCAL_HEALTHCHECK_ENABLED` (default `true`), `LOCAL_HEALTHCHECK_TIMEOUT_SECONDS` (default `3`). Lists available models if the requested id is missing.
154- `.env.example` documents all of the above.
155 
156### Resilience / offline-friendly bits
157- `src/context_compression.py` and `src/model_router.py`: if `tiktoken` cannot load `cl100k_base` (network/cache), token counts fall back to a simple whitespace split (metrics only; not OpenAI-accurate).
158- `src/confidence_checker.py`: if `SentenceTransformer("BAAI/bge-small-en-v1.5")` fails to load, embedding similarity returns `0.0` and logs once (pipeline keeps running).
159 
160### Query analyzer (Evelyn-inspired heuristics, same API)
161- `src/query_analyzer.py`: merged heuristic ideas from a former prototype folder (`evelyn_query/`, now removed) without adopting its separate runtime:
162  - extra complex/medium keywords;
163  - stronger multi-part signal (`" and "` in longer queries);
164  - boundary-aware confidence near label thresholds (`0.38` / `0.70`).
165- **Report artifacts** (for write-ups): `results/query_analyzer_report/`
166  - `REPORT.md`, `query_analyzer_predictions.csv`, `query_analyzer_summary.json`
167  - `BEFORE_AFTER_HEURISTIC_TEMPLATE.md` — narrative template for the final report.
168 
169### Evaluation / results layout
170- Hybrid local+OpenAI runs can be versioned like existing folders, e.g. `results/local_v2_lmstudio/`, `results/local_v3_lmstudio_ids/`, plus seed-7 pairs and `compare_vs_*.txt` from `results/compare_runs.py`.
171- **`evaluate.py` CSV/JSON** now carries per-stage observability: analyzer word count + reason codes + label match vs test row; retrieval coverage threshold + retry gate + mean Chroma chunk distance; compression sentence counts + tokens saved + pipeline token estimates; router source + fallback + reroute flag; confidence semantic score + threshold + embedding checker on/off. JSON adds `pipeline_stage_summary` (nested snapshot). End-of-run summary prints avg tokens saved, avg semantic confidence, and model reroute count.
172 
173### Readiness caveats (full pipeline)
174- **Router**: LM Studio must be running with models whose IDs match `.env` (`GET /v1/models`). **Complex** tier needs **`OPENAI_API_KEY`**.
175- **Retrieval / confidence / compression embeddings**: HF model `BAAI/bge-small-en-v1.5` must load (set **`HF_HOME`** / project `.hf_cache` if default cache is not writable). Set **`COMPRESSION_USE_EMBEDDINGS=false`** to skip compression-time sentence embeddings only.
176- **Confidence reroute**: `MODEL_TIERS` in `confidence_checker.py` still targets **OpenAI model names**; local-only model IDs usually **will not** trigger a stronger-model retry until tiers are extended for your local IDs.
177 
178### Context compression (selective Anh-style upgrade, local-only)
179- `src/context_compression.py`: **adaptive budget** `round(1 + complexity_score * 5)` (cap 6) when `COMPRESSION_ADAPTIVE_BUDGET=true`; else legacy fixed map `simple/medium/complex`.
180- **Hybrid ranking**: TF-IDF + **local** `SentenceTransformer` (default `BAAI/bge-small-en-v1.5`, same family as retrieval). Weights shift toward embeddings as `complexity_score` rises. If embeddings fail to load, falls back to **TF-IDF only** (no OpenAI embedding calls).
181- **Redundancy**: greedy selection by score; skip a candidate if cosine similarity ≥ `COMPRESSION_REDUNDANCY_THRESHOLD` (default `0.85`) to an already chosen sentence (embedding space, or TF-IDF if no embeddings).
182- **Answer-type boosts**: small score bumps for when/year, who/capitalized names, where/location cues, what-is definition patterns.
183- `src/pipeline.py` passes `complexity_score` into `compress(...)`. Return dict includes `compression_metadata` (budget, weights, `used_embeddings`, redundancy skips). `evaluate.py` adds flat CSV columns and full `compression_metadata` in JSON records.
184- Env knobs in `.env.example`: `COMPRESSION_USE_EMBEDDINGS`, `COMPRESSION_EMBEDDING_MODEL`, `COMPRESSION_ADAPTIVE_BUDGET`, `COMPRESSION_REDUNDANCY_THRESHOLD`.
185 
186### Performance note (debugging)
187- If runs feel slow, check per-query **stage timing** in logs/CSV: `model_router_ms` and `confidence_checker_ms` often dominate; `query_analyzer_ms` → `adaptive_retriever` wiring is verified consistent (`complexity_label`, `complexity_score`, `confidence` passed through `src/pipeline.py`).
188 
189## Updates — Apr 24, 2026 (evaluation interpretability + seed runs)
190 
191_Added after the sections above; earlier dated bullets are unchanged._
192 
193### Answer vs ground truth in `evaluate.py` (offline only; no OpenAI judge)
194- Per-row CSV/JSON columns: **`answer_gt_token_f1`** (multiset token F1 between `ground_truth` and `final_answer`), **`answer_gt_embedding_similarity`** (cosine sim of embeddings for the two strings; local **`sentence_transformers`**, model from **`EVAL_GT_EMBEDDING_MODEL`** in `.env.example`, default `BAAI/bge-small-en-v1.5`), **`answer_gt_appropriateness_score`** (blend when both signals exist).
195- End-of-run console block: **“ANSWER VS GROUND TRUTH (softer metrics)”** with row counts and averages; **substring** accuracy is labeled explicitly as **“Substring hit rate”** (`ground_truth` must appear in `final_answer`) so it is not confused with F1/embedding scores.
196- JSON **`pipeline_stage_summary.answer_vs_ground_truth`** holds substring hit, F1, embedding sim, and appropriateness for quick reporting.
197 
198### `confidence_checker.py` (runtime; unchanged role, richer exports)
199- Return dict now includes **`confidence_threshold`**, **`confidence_semantic`**, and **`confidence_checker_embedding_enabled`** so pipeline/`evaluate.py` can log whether embedding-based similarity ran.
200 
201### Concept split (for team write-ups)
202- **`confidence_checker`**: scores **answer ↔ compressed context** (grounding / self-consistency in the prompt). Does **not** use CSV `ground_truth`.
203- **`evaluate.py` soft metrics**: score **answer ↔ labeled `ground_truth`** for benchmarks. Complementary to the checker, not a replacement.
204 
205### Multi-seed comparison artifacts
206- Folder **`results/compare_seeds/`**: `seed42/` and `seed7/` each with `evaluation_results.csv`, `.json`, `run_output.log`; root **`compare_seed7_vs_seed42.txt`** from `python results/compare_runs.py ...`; **`RUN_SUMMARY.txt`** lists paths and the compare command.
207 
208### Prototype / noise cleanup
209- Removed duplicate **`evelyn_query/`** prototype (heuristics already merged into `src/query_analyzer.py`).
210- **`Anh_Context compression/`** was verified to be a **Windows venv only** (no app source to merge); safe to delete from the repo if still present—do not treat as `src/context_compression.py` replacement.
211 
212### Possible next step (not implemented; design only)
213- **Retrieval + compression-aware confidence**: pass e.g. `coverage_score`, `compression_ratio`, `retrieval_retry` from **`src/pipeline.py`** into **`check_confidence`** and adjust threshold/score interpretably (team slide spec). **`MODEL_TIERS`** remains OpenAI-name-oriented until extended for local LM Studio model IDs if you want confidence-tier retries there too.
214 
215## Updates — Apr 24, 2026 (preflight, `evaluate.py` CLI, `.env` load order, docs, smoke runs)
216 
217_Appended; all sections above remain as-is._
218 
219### `src/preflight.py` (new)
220- **`PreflightError`** for fail-fast checks before a long batch.
221- **`validate_queries_csv`**: file exists, required columns (`query_id`, `query`, `complexity`, `ground_truth`), at least one data row.
222- **`check_vector_store`**: `chroma_db/` under **project root** or a documents tree with ingestible `.txt` files.
223- **`check_local_openai_models`**: `GET` local `…/v1/models` and verify `LOCAL_SIMPLE_MODEL` / `LOCAL_MEDIUM_MODEL` unless **`--skip-local-model-check`**.
224- **`warn_openai_key_if_missing`** / optional **`--require-openai-key`** on `evaluate.py`.
225- **`log_effective_configuration` / `log_pipeline_startup`**: sanitized snapshot (paths, model IDs, compression flags, whether OpenAI key is set); **`src/pipeline.py`** calls startup log once after wiring components.
226- **Bugfix**: removed stray **`return n`** from `run_eval_preflight` after refactor (was causing `NameError`).
227 
228### `evaluate.py` — `.env` before preflight + CLI
229- **`load_dotenv(PROJECT_ROOT / ".env")`** runs at import (after `PROJECT_ROOT` is set) so **`LOCAL_*` / `OPENAI_*`** are visible to **`os.getenv` during preflight** (previously preflight ran before `RAGPipeline` constructed `ModelRouter`, so defaults like `tinyllama` / `ministral` were used and LM Studio ID checks failed even with a correct `.env`).
230- **Argparse**: `--queries`, `--out-csv`, `--out-json`, `--documents`, `--seed`, `--no-preflight`, `--skip-local-model-check`, `--require-openai-key`, `--log-file`, `--log-json`, `-v` / `-vv`, `-q`.
231- **Paths**: relative paths resolve under project root via **`src.preflight.resolve_under_project`**.
232- **`RAGPipeline` lazy-import** inside `run_evaluation()` so **`python evaluate.py --help`** does not load HF/Chroma.
233- **`try` / `finally`** closes optional JSONL handle; guard if zero rows processed.
234- **Per-query JSONL** (`--log-json`): `query_complete` lines with `query_id`, `total_pipeline_ms`, `substring_correct`, `failure_type`.
235- **Exit codes**: `2` = `PreflightError`, `130` = Ctrl+C, `1` = other exceptions.
236- **Wrapper script**: `scripts/run_eval.sh` — `cd` to repo root and `exec python evaluate.py "$@"`.
237 
238### `src/adaptive_retriever.py`
239- **`chroma_db`** path is **`{project_root}/chroma_db`** (not cwd-relative) so retrieval works regardless of shell working directory.
240 
241### `src/confidence_checker.py` (documentation)
242- Module docstring clarifies **`MODEL_TIERS`** keys are **OpenAI-style** names; **local LM Studio `id` strings do not match** → **no tier retry** (scores still returned).
243 
244### Repo docs
245- **`README.md`**: quick start, LM Studio `/v1/models` IDs, preflight/CLI, GT vs confidence checker, first-run notes (tiktoken fallback, embeddings).
246- **`TEAM_GUIDE.md`**: pipeline status, file tree, `evaluate.py` / `run_eval.sh` instead of removed `test_pipeline.py` / `run_all_queries.py`.
247 
248### `src/` module docstrings (light)
249- Added top-of-file / section headers in **`pipeline.py`**, **`model_router.py`**, **`adaptive_retriever.py`**, **`query_analyzer.py`**, **`context_compression.py`**, **`confidence_checker.py`**, **`seed_manager.py`**; **`evaluate.py`** module docstring updated for CLI and GT semantics.
250 
251### Fast iteration (smoke) — added in repo
252- **`data/smoke_queries.csv`**: first **10** rows from `data/test_queries.csv` (all **simple**) for quick end-to-end checks.
253- Example run (does not require editing `.env` permanently):  
254  `COMPRESSION_USE_EMBEDDINGS=false python3 evaluate.py --queries data/smoke_queries.csv --out-csv results/smoke_run/evaluation_smoke.csv --out-json results/smoke_run/evaluation_smoke.json`  
255  **Lighter compression** = TF-IDF-only for that process env; **fewer rows** = faster wall clock. Use full **`data/test_queries.csv`** + `COMPRESSION_USE_EMBEDDINGS=true` for comparable benchmark numbers.
256 
257### LM Studio / `.env` reminder (team)
258- **`LOCAL_OPENAI_BASE_URL`** should include **`/v1`** (e.g. `http://127.0.0.1:1234/v1`).
259- **`LOCAL_SIMPLE_MODEL`** and **`LOCAL_MEDIUM_MODEL`** must equal **API model identifiers** from LM Studio (or the same model id for both tiers if only one model is loaded).
260 
261## Updates — Apr 24, 2026 (compression ↔ retrieval, metrics naming, hygiene, router decision)
262 
263_Appended; all sections above remain as-is._
264 
265### `src/context_compression.py` + `src/pipeline.py`
266- **`coverage_score`** from **`adaptive_retriever.retrieve`** is passed into **`ContextCompressor.compress`**; sentence budget adds up to **+2** sentences when keyword coverage is low (capped at **6**); metadata includes `sentence_budget_base`, `coverage_score_used`, `coverage_budget_extra`.
267- Adaptive / coverage-adjusted budgets use a **minimum of 2** sentences (was 1) to reduce overly aggressive single-sentence context.
268- **`_hybrid_weights`** docstring corrected: higher **complexity** shifts weight toward **embedding** (semantic-heavy); simple stays more keyword/TF-IDF–heavy.
269 
270### `src/adaptive_retriever.py`
271- Chunk metadata adds **`chroma_distance`** (Chroma/LangChain **distance**, **lower = closer**). **`similarity_score`** kept as the same numeric value for older JSON consumers (name is misleading).
272- Removed unused **`k_map`** dict.
273- Renamed gate variable to **`low_confidence_for_retry`** (clearer than “confidence_for_retry”).
274 
275### `src/pipeline.py`
276- Prints chunk lines as **`chroma distance`**; **`retrieval_avg_chunk_distance`** is mean distance (unchanged column name, semantics documented in README).
277 
278### `evaluate.py`
279- Creates **parent directories** for `--out-csv` / `--out-json` before write.
280- Extra CSV columns: **`compression_sentence_budget_base`**, **`compression_coverage_score_used`**, **`compression_coverage_budget_extra`**.
281 
282### `src/query_analyzer.py`
283- FLAN **adoption** when overriding heuristics now uses **`self.high_conf_threshold`** (from **`QUERY_ANALYZER_HIGH_CONF_THRESHOLD`**, default **0.80**) instead of hardcoded **0.65** — fewer overrides unless FLAN is highly confident (tunable via env).
284 
285### `src/confidence_checker.py`
286- **`sentence_transformers`** load is **lazy** (import inside try on first use) so a missing/broken package does not break module import; module-level note above **`MODEL_TIERS`** about local LM Studio ids not matching OpenAI-style keys.
287 
288### `src/model_router.py` (explicit non-change)
289- A **Phase 2** experiment added per-complexity **temperature** and **prompt templates**; it was **reverted**. Current router: **single `self.temperature`** (default **0.0**) and **one** system/human prompt pair. No **`ROUTER_TEMP_*`** env vars in **`.env.example`**.
290 
291### Repo hygiene (`.gitignore`)
292- Extended with **`results/`**, **`.hf_cache/`**, **`venv/`**, **`*.docx`**, **`~$*`** (Office temp locks) among other patterns — reduces accidental commits of runs, caches, and local artifacts.
293 
294## Updates — Apr 25, 2026 (shared embeddings cache + router fallback discussion)
295 
296_Appended; all sections above remain as-is._
297 
298### `src/shared_embeddings.py` (new)
299- Added a shared in-process cache for sentence-transformer models: **`get_embedding_model(model_name)`**.
300- Cache is **per model name** (dict-backed), so one process reuses loaded weights instead of re-instantiating the same model in each component.
301- Added lightweight **`logging.debug(...)`** messages for:
302  - previously failed model load skip
303  - cached model reuse
304  - first successful load
305  - load failure (exception text)
306 
307### `src/context_compression.py`
308- Replaced direct `SentenceTransformer(...)` creation with shared **`get_embedding_model(self.embedding_model_name)`**.
309- If shared loader returns `None`, compressor marks embeddings unavailable and falls back safely.
310 
311### `src/confidence_checker.py`
312- Replaced local lazy `SentenceTransformer("BAAI/bge-small-en-v1.5")` init with shared cache loader using the same model id.
313- Keeps existing fail-open behavior (pipeline continues if embeddings are unavailable).
314 
315### `evaluate.py`
316- Replaced `_eval_embedding_model()` direct instantiation with shared cache loader for **`EVAL_GT_EMBEDDING_MODEL`**.
317- Preserves prior behavior when embedding model is unavailable (returns `None` for that metric path).
318 
319### Expected impact
320- Reduces duplicate RAM use when multiple modules need the same embedding model in one run.
321- Reduces repeated warm-up/loading overhead; throughput per `encode` call is unchanged after warm-up.
322- Improves stability for Colab/low-memory environments by avoiding multiple copies of the same model weights.
323 
324### `src/model_router.py` note (decision deferred)
325- Considered adding provider fallback via external cascade (complex path only).
326- Team decision for now: **defer** this change; keep current router behavior unchanged until later A/B testing.
327 
328## Updates — Apr 25, 2026 (semantic cache in pipeline)
329 
330_Appended; all sections above remain as-is._
331 
332### `src/semantic_cache.py` (new)
333- Added **`SemanticCache`** for in-memory near-duplicate query reuse.
334- Uses lazy-load **`SentenceTransformer("BAAI/bge-small-en-v1.5")`** (same fail-open pattern style as confidence checker).
335- Cache entries are dicts with keys: **`query`**, **`embedding`**, **`result`**, **`corpus_hash`**.
336 
337### Lookup / hit policy
338- **`lookup(query, corpus_hash)`** embeds the incoming query and runs cosine similarity vs cached embeddings (sklearn).
339- Cache hit requires:
340  - matching **`corpus_hash`**
341  - best similarity **>= 0.92**
342- On hit, returns cached result enriched with:
343  - **`cache_hit=True`**
344  - **`cache_similarity`** (rounded float for logging)
345 
346### Store / eviction policy
347- **`store(query, result, corpus_hash)`** writes only when result quality gates pass:
348  - `confidence_score_final > 0.75`
349  - `retried == False`
350  - `coverage_score > 0.55`
351  - `retrieval_retry == False`
352- FIFO eviction: if cache grows beyond **500** entries, oldest entry is removed.
353 
354### Corpus-change safety
355- **`SemanticCache.get_corpus_hash(chroma_dir)`** computes an MD5 over file modification times under the Chroma directory.
356- Returns empty string if directory does not exist.
357 
358### `src/pipeline.py` integration
359- In `__init__`:
360  - instantiate **`SemanticCache`**
361  - compute and store **`self.corpus_hash = SemanticCache.get_corpus_hash(CHROMA_DIR)`**
362- At start of `run()`:
363  - call cache lookup before analyzer/retriever/compressor/router
364  - on hit, prints cache-hit message with similarity and returns cached payload immediately
365- At end of `run()`:
366  - add **`cache_hit=False`** for normal non-cached path
367  - call cache `store(...)` just before return
368 
369## Updates — Apr 25, 2026 (alignment + safeguards follow-up)
370 
371_Appended; all sections above remain as-is._
372 
373### `src/semantic_cache.py` follow-up
374- Replaced local embedding loader with shared loader: **`from src.shared_embeddings import get_embedding_model`**.
375- **`SemanticCache._get_embedding_model()`** now reuses shared in-process model cache, preventing duplicate BGE-small instances on cold start.
376- Added **`logging.debug(...)`** rejection observability in `store()` for each gate (low confidence, retried, low coverage, retrieval retry, embedding unavailable, and non-dict payload), plus store/eviction events.
377 
378### `src/context_compression.py`
379- Aligned adaptive sentence budget formula to agreed spec:
380  - from `round(1 + complexity_score * 5)` (clamped)  
381  - to **`round(2 + complexity_score * 4)`** (clamped to `[2, 6]`).
382 
383### `src/preflight.py` + `evaluate.py`
384- Added CSV-aware fail-fast for OpenAI key:
385  - new helper **`csv_has_complex_queries(queries_file)`**
386  - `evaluate.py` computes this before preflight
387  - `run_eval_preflight(...)` now enforces missing-key failure when the benchmark includes any `complex` rows (or when `--require-openai-key` is set).
388- Result: avoids mid-batch failure on first complex query due to missing `OPENAI_API_KEY`.
389 
390### `src/adaptive_retriever.py`
391- Unified retrieval sizing around shared constants:
392  - `K_BASE_MIN=3`, `K_BASE_MAX=10`, **`K_HARD_CAP=10`**, `LOW_CONF_BONUS=2`, `RETRY_BONUS=1`.
393- Low-confidence bump and coverage-retry bump now both respect the same hard cap constant.
394- Removes prior split-cap behavior (`k_base` capped at 10 while bump/retry could reach 12).
395 
396## Updates — Apr 25, 2026 (general typo-robust query handling)
397 
398_Appended; all sections above remain as-is._
399 
400### `src/pipeline.py` generalization
401- Replaced narrow hardcoded typo replacements with a **corpus-driven spell-correction path**.
402- Built a lightweight in-memory vocabulary index from `data/documents/**/*.txt` and used fuzzy matching (`difflib`) to normalize noisy query tokens.
403- Added conservative token guardrails (stopword + short-token excludes) plus small edit heuristics (leading/trailing character noise) before fuzzy correction.
404- Query flow now uses **original + normalized + spell-corrected** variants for cache lookup and robustness retries.
405 
406### Retry behavior updates
407- Kept the low-quality fallback retry but made it **general** (not founder-specific):
408  - triggers on weak retrieval/answer quality signals,
409  - retries retrieval with stronger settings and candidate query variant,
410  - adopts retry output only when confidence improves.
411 
412### Scope cleanup
413- Removed the temporary founder-specific hard override path so behavior remains generic across intents.
414- Retained existing non-session semantic cache behavior and session-document cache bypass.
415 
416## Updates — Apr 25, 2026 (factual extraction precision + retrieval diagnostics UI + k override cleanup)
417 
418_Appended; all sections above remain as-is._
419 
420### `src/adaptive_retriever.py`
421- Added/kept the sentence-level factual index path and retrieval helper for extraction-first answering:
422  - factual sentence corpus built from `data/documents/**/*.txt`,
423  - title/section noise lines are stripped before sentence indexing,
424  - sentence ranking blends semantic TF-IDF similarity with query-token overlap.
425 
426### `src/pipeline.py` (precision + observability)
427- Improved `who founded/established/started ...` extraction with a **general subject-aware reranker**:
428  - alias generation for subject forms (`the ...`, `university of ...`, acronym variants, `and`/`&` variants),
429  - alias-strength scoring against candidate sentence text,
430  - proximity gate requiring subject alias to appear near founding verbs to avoid sub-entity false positives.
431- Extraction-first branch now falls back safely when confidence is weak (no hardcoded entity exceptions).
432- Added structured return payload: **`retrieval_diagnostics`** with:
433  - factual extraction attempted/used + hit counts + top factual hits,
434  - retrieval source diagnostics (`k_base`, `k_final`, coverage, retry, top sources/distances),
435  - compression mode, grounding gate outcome, OpenAI/deep fallback attempt and use flags.
436 
437### `ui.py` (retrieval diagnostics surfaced)
438- Chat metadata pills now include **compression mode**.
439- Added per-assistant-turn expandable **Retrieval diagnostics** panel in chat, showing:
440  - k progression, coverage, retry, factual extraction status/hits,
441  - grounding gate result, fallback usage, top retrieved sources, top factual sentence hits.
442- Dashboard gained a **retrieval diagnostics summary** section:
443  - avg final k, avg coverage, retrieval retry rate,
444  - factual extractive usage count, grounded response count, OpenAI fallback usage count.
445- Session logging now stores retrieval diagnostic booleans/fields for dashboard aggregation.
446 
447### Follow-up cleanup (requested)
448- Removed the special-circumstance forced retrieval depth override from deep OpenAI retry path:
449  - deleted `force_k=20` usage in `src/pipeline.py`,
450  - removed forced-k wording from logs/metadata,
451  - deep retry now uses normal adaptive retriever behavior.
452 
453## Updates — Apr 25, 2026 (semantic chunker + HyDE toggle + corpus curation)
454 
455_Appended; all sections above remain as-is._
456 
457### `src/adaptive_retriever.py` (chunking upgrade)
458- Added optional semantic chunking path using **`SemanticChunker`** from `langchain_experimental` with safe fallback to `RecursiveCharacterTextSplitter`.
459- New env controls:
460  - `USE_SEMANTIC_CHUNKER` (default `true`)
461  - `SEMANTIC_BREAKPOINT_TYPE` (default `percentile`)
462  - `SEMANTIC_BREAKPOINT_AMOUNT` (default `85`)
463- Chunking config/version now includes semantic-chunker settings (`CHUNKING_VERSION = "v3_semantic_chunker_toggle"`), so Chroma rebuild triggers automatically when toggled.
464- Added dependency: **`langchain-experimental`** in `requirements.txt`.
465 
466### `src/pipeline.py` (HyDE retrieval, flag-gated)
467- Added lightweight **HyDE** at query time (ingest unchanged):
468  - env: `USE_HYDE` (default `false`)
469  - generates one concise hypothetical retrieval text using local OpenAI-compatible model
470  - runs alternate retrieval with that text
471  - adopts HyDE retrieval only when coverage improves by at least `HYDE_MIN_COVERAGE_GAIN` (default `0.03`)
472- Added related env/tuning fields:
473  - `HYDE_MIN_COVERAGE_GAIN` (default `0.03`)
474  - `HYDE_MAX_CHARS` (default `700`)
475- Added observability fields in result payload:
476  - `hyde_attempted`, `hyde_used`, `hyde_coverage_score`, `hyde_query_preview`, `retrieval_query_used`
477 
478### `ui.py` (HyDE diagnostics visibility)
479- Retrieval diagnostics panel now shows HyDE status:
480  - attempted/used flags
481  - alternate retrieval coverage score
482 
483### Corpus curation fix for founder query
484- Added one canonical fact sentence to `data/documents/doc_18.txt`:
485  - “Father Edward Sorin of the Congregation of Holy Cross founded the University of Notre Dame on November 26, 1842.”
486- Rebuilt index (`chroma_db`) after update.
487- Post-rebuild validation confirms `Who founded Notre Dame?` now returns the canonical grounded sentence via `extractive-factual`.
488 
489## Updates — Apr 26, 2026 (numeric factual formatting + Streamlit stability)
490 
491_Appended; all sections above remain as-is._
492 
493### `src/pipeline.py` (`how many` extractive answer refinement)
494- Improved extractive formatting for count questions:
495  - maps number words (`one`..`twenty`) to numeric values,
496  - scans all numeric mentions in top factual sentence hits,
497  - selects the strongest numeric signal (max value) to avoid incorrect first-match picks like “one of the five...”.
498- Output now returns concise numeric form when possible (e.g., `5 undergraduate colleges.`) instead of full sentence-only phrasing.
499 
500### Deep fallback guard for valid numeric extractive answers
501- Added a guard to prevent deep OpenAI fallback from overriding already-valid extractive numeric answers.
502- Condition: when factual extraction is used and answer starts with a number, skip deep OpenAI retry.
503 
504### Runtime validation snapshot
505- Verified query:
506  - `How many undergraduate colleges are at Notre Dame?`
507  - final answer now resolves to numeric extractive output (`5 undergraduate colleges.`) on `extractive-factual` path.
508- Also resolved Streamlit runtime startup instability by launching from project `venv` with:
509  - `--server.fileWatcherType none`
510  - this avoids watcher-related crashes seen under the global/anaconda environment.
511 
512## Updates — May 1, 2026 (retrieval reranker + confidence checker reliability tuning)
513 
514_Appended; all sections above remain as-is._
515 
516### `src/adaptive_retriever.py` (cross-encoder second-stage reranking)
517- Added optional cross-encoder reranking after hybrid merge and before compression input:
518  - model: `cross-encoder/ms-marco-MiniLM-L-6-v2`
519  - candidate cap: `CROSS_ENCODER_CANDIDATES` (default `20`)
520  - flag: `USE_CROSS_ENCODER_RERANK` (default `true`)
521- Kept current dense+lexical hybrid as candidate generation and bounded reranking to top candidates only.
522- Added safe fallback behavior when cross-encoder load/inference fails.
523- Added retrieval metadata for observability:
524  - `rerank_enabled`, `rerank_used`, `rerank_model`, `rerank_latency_ms`
525  - chunk-level `cross_encoder_score` + `retrieval_mode` suffix `+cross_encoder`.
526 
527### `src/pipeline.py` (rerank observability propagation)
528- Extended pipeline result and `retrieval_diagnostics` to include reranker fields:
529  - `rerank_enabled`, `rerank_used`, `rerank_model`, `rerank_latency_ms`.
530 
531### `src/confidence_checker.py` (retry behavior fixes + scoring fairness)
532- Fixed local-model retry dead-path by expanding model tier mapping:
533  - dynamically map `LOCAL_SIMPLE_MODEL` and `LOCAL_MEDIUM_MODEL` to `gpt-4o-mini`,
534  - keep OpenAI escalation path (`gpt-3.5-turbo -> gpt-4o-mini -> gpt-4o`),
535  - include short alias support from model IDs.
536- Added threshold configurability with local/openai split:
537  - `CONFIDENCE_THRESHOLD_LOCAL` (default `0.50`)
538  - `CONFIDENCE_THRESHOLD_OPENAI` (default `0.65`)
539  - backward-compatible `CONFIDENCE_THRESHOLD` override still supported.
540- Implemented complexity-aware heuristic length normalization:
541  - simple/medium/complex targets: `10/20/40`.
542- Upgraded embedding similarity from global context vector to sentence-level max pooling:
543  - answer embedding is compared against each context sentence embedding,
544  - final embedding score uses max sentence match for factual grounding.
545 
546### `src/model_router.py` (local model cache hygiene)
547- Added TTL-based refresh for local model cache to avoid stale `/v1/models` state when LM Studio hot-reloads:
548  - `LOCAL_MODELS_CACHE_TTL_SECONDS` (default `60`).
549 
550### Runtime checks performed
551- Compile + lint checks passed for touched files.
552- Smoke tests confirmed:
553  - local-model confidence retry can now escalate to OpenAI tier when needed,
554  - local/openai thresholds are selected correctly,
555  - cross-encoder reranker is active and observable in retrieval metadata.
556 
557## Updates — May 1, 2026 (analyzer-driven factual fast path + compression generalization)
558 
559_Appended; all sections above remain as-is._
560 
561### `src/pipeline.py` (QueryAnalyzer-triggered factual fast path)
562- Added a first-class factual fast path that can bypass Chroma retrieval and go directly to sentence-index extraction when all of the following hold:
563  - `simple_starter` present in `analysis.reason_codes`,
564  - `complexity_score < 0.30`,
565  - factual-query pattern check passes,
566  - no session-uploaded documents are attached.
567- Behavior:
568  - if strong extractive answer is found from `retrieve_factual_sentences(...)`, pipeline skips vector retrieval and proceeds with extractive-factual path;
569  - otherwise falls back safely to normal hybrid retrieval.
570- Added result + diagnostics fields:
571  - `factual_fastpath_attempted`
572  - `factual_fastpath_used`
573 
574### `src/context_compression.py` (`_answer_boost` de-corpus-ified)
575- Replaced corpus-specific location keywords with generic/default location cues.
576- Added env override:
577  - `COMPRESSION_LOCATION_KEYWORDS` (comma-separated) for corpus-specific customization when needed.
578- Added generic place-pattern detection (proper-noun phrase after location prepositions) for `where`-type boosts.
579 
580### `src/pipeline.py` robustness fix (retrieval distance metric)
581- Fixed `retrieval_avg_chunk_distance` aggregation to skip non-numeric chunk distance values (e.g., lexical chunks with empty distance fields), avoiding `float('')` errors in mixed retrieval mode.
582 
583### Validation snapshot
584- Compile + lint passed for updated files.
585- Smoke checks confirm:
586  - founder query uses factual fast path and returns grounded canonical sentence with retrieval bypass,
587  - `how many` query still returns numeric extractive answer,
588  - non-factual comparative queries continue through standard retrieval/compression/router path.
589 
590## Updates — May 1, 2026 (benchmark instrumentation + factual fastpath impact validation)
591 
592_Appended; all sections above remain as-is._
593 
594### `evaluate.py` instrumentation updates
595- Added new exported CSV/JSON fields for analysis:
596  - `factual_fastpath_attempted`
597  - `factual_fastpath_used`
598  - `rerank_enabled`, `rerank_used`, `rerank_model`, `rerank_latency_ms`
599- Extended end-of-run summary to print:
600  - factual fastpath usage (used/attempted counts),
601  - cross-encoder rerank usage count.
602 
603### Full 50-query benchmark run
604- Executed benchmark with:
605  - `evaluate.py --queries data/test_queries.csv`
606  - outputs:
607    - `results/may1_fastpath/evaluation_results.csv`
608    - `results/may1_fastpath/evaluation_results.json`
609- Run summary highlights:
610  - total queries: `50`
611  - factual fastpath: `used 12/50` (attempted `12`)
612  - cross-encoder rerank used: `38/50`
613  - retrieval retries: `4/50`
614  - model reroutes: `15/50`
615 
616### Factual fastpath impact (computed from benchmark CSV)
617- Average total pipeline latency:
618  - fastpath used: `171.38 ms`
619  - fastpath not used: `11211.90 ms`
620- Average retrieval latency:
621  - fastpath used: `3.29 ms`
622  - fastpath not used: `572.60 ms`
623- Average confidence:
624  - fastpath used: `0.79`
625  - fastpath not used: `0.54`
626- Fastpath usage concentrated on simple queries:
627  - simple: `12`
628  - medium: `0`
629  - complex: `0`
630 
631## Updates — May 1, 2026 (fallback-chain cap + timing split + starter protection + rerank policy)
632 
633_Appended; all sections above remain as-is._
634 
635### `src/pipeline.py` (confidence-stage timing split)
636- Split confidence-stage timing into explicit sub-metrics:
637  - `confidence_scoring_ms`
638  - `robust_retry_ms`
639  - `deep_openai_ms`
640  - `grounding_fallback_ms`
641- Kept `confidence_checker_ms` as aggregate of the four sub-metrics for backward-compatible dashboards/reports.
642- Updated stage timing log line to print confidence sub-breakdown.
643 
644### `src/pipeline.py` (protect factual starters before spell-correction)
645- Added starter protection so spell-correction does not alter factual query prefixes:
646  - `how many`, `who`, `when`, `where`, `which`, `what is`, `what was`.
647- Behavior change: only the suffix after a protected factual prefix is spell-corrected.
648 
649### `src/adaptive_retriever.py` (selective cross-encoder policy, env-gated)
650- Added per-query rerank gate:
651  - rerank enabled for `medium/complex`,
652  - simple queries rerank only when `RERANK_SIMPLE_QUERIES=true`.
653- New env:
654  - `RERANK_SIMPLE_QUERIES` (default `false`).
655- Retrieval metadata now reflects effective per-query rerank decision (`rerank_enabled`, `rerank_model`, etc.).
656 
657### `src/pipeline.py` (fallback-chain depth cap, env-gated)
658- Added global heavy-fallback cap across:
659  - typo-tolerant robust retry,
660  - deep OpenAI fallback,
661  - grounding-gate OpenAI fallback.
662- New env:
663  - `PIPELINE_MAX_HEAVY_FALLBACKS` (default `1`).
664- Added observability fields:
665  - `max_heavy_fallbacks`
666  - `heavy_fallbacks_used`.
667 
668### Validation — fallback cap A/B (cap=1 vs cap=2)
669- **Slice A (50-query subset)**:
670  - accuracy (evaluable): unchanged (`50.0%` vs `50.0%`),
671  - paired average latency delta (`cap2 - cap1`): approximately `-171.79 ms` on this run (no accuracy gain).
672- **Slice B (hard subset, 29 queries)**:
673  - hard subset built from reroute/retry/non-simple signals;
674  - single-run result: accuracy unchanged (`88.89%` evaluable), paired average latency delta (`cap2 - cap1`) approximately `+2755.87 ms`.
675- **Stability check (3 repeated hard-slice runs per cap)**:
676  - accuracy unchanged in all runs (`88.89%` evaluable for both caps),
677  - paired mean latency delta (`cap2 - cap1`): approximately `+931.34 ms` (std `1582.09`),
678  - `cap=2` showed higher confidence-stage latency variance.
679 
680### Decision from validation
681- Keep default:
682  - `PIPELINE_MAX_HEAVY_FALLBACKS=1`
683- Rationale:
684  - no observed accuracy lift from cap `2`,
685  - higher/less stable latency and potential extra fallback cost with cap `2`.
686 
687## Updates — May 1, 2026 (deep OpenAI fallback coverage gate)
688 
689_Appended; all sections above remain as-is._
690 
691### `src/pipeline.py` (coverage-aware deep fallback gate)
692- Added env-gated minimum coverage requirement before deep OpenAI fallback can run:
693  - `DEEP_OPENAI_MIN_COVERAGE` (default `0.40`).
694- Deep fallback now requires:
695  - existing low-quality eligibility + API/cap checks, and
696  - `retrieval.coverage_score >= DEEP_OPENAI_MIN_COVERAGE`.
697- Added explicit skip log when deep fallback is otherwise eligible but blocked by low coverage:
698  - includes current `coverage_score` and configured threshold.
699 
700### `.env.example` update
701- Added:
702  - `DEEP_OPENAI_MIN_COVERAGE=0.40`
703- Comment clarifies behavior:
704  - skip deep OpenAI fallback when corpus coverage is below threshold.
705 
706### Quick verification
707- `src/pipeline.py` compiles successfully (`py_compile`).
708- Lint check passed for touched files.
709 
710## Updates — May 1, 2026 (embedding reuse in confidence checker + robust retry scope cap)
711 
712_Appended; all sections above remain as-is._
713 
714### `src/context_compression.py` + `src/confidence_checker.py` (embedding reuse)
715- Added carry-through of selected sentence embeddings from compression output:
716  - `_selected_sentence_embeddings` is now included in compression results when embedding scoring is active.
717- Extended confidence-check embedding API to accept precomputed context sentence embeddings:
718  - `embedding_similarity(..., context_sentences, context_sentence_embeddings)`
719  - `check_confidence(..., context_sentences, context_sentence_embeddings)`
720- Behavior:
721  - confidence checker reuses compressor-provided sentence embeddings and encodes only the answer embedding for similarity,
722  - fallback path still works if cached embeddings are unavailable/mismatched.
723 
724### `src/pipeline.py` (wire embedding reuse through all confidence paths)
725- Updated confidence-check calls to pass selected sentences + carried embeddings for:
726  - initial confidence scoring,
727  - robust retry confidence scoring,
728  - grounding-triggered OpenAI fallback confidence scoring.
729 
730### `src/pipeline.py` (cap robust retry to non-simple queries)
731- Restricted typo-tolerant robust retry to medium/complex requests only:
732  - added `complexity != "simple"` condition to `should_retry_robust`.
733- Goal:
734  - avoid extra retrieval/compression/router passes on simple low-coverage questions where refusal/grounding guard is usually preferable.
735 
736### Quick verification
737- Lint check passed for:
738  - `src/context_compression.py`
739  - `src/confidence_checker.py`
740  - `src/pipeline.py`
741- Compile checks passed for the same files (`py_compile`).
742 
743## Updates — May 1, 2026 (session-only multimodal PDF retrieval + UI upload/title enhancements)
744 
745_Appended; all sections above remain as-is._
746 
747### `src/session_multimodal_retriever.py` (new module, in-memory only)
748- Added session-only multimodal retriever for uploaded visual PDFs:
749  - no persistence,
750  - no ChromaDB writes,
751  - no changes to main corpus index.
752- Implemented chunking spec:
753  - primary sentence split: `re.split(r"(?<=[.!?])\\s+", page_text)`,
754  - fallback paragraph split (`\\n\\n`) when primary yields fewer than 2 usable chunks,
755  - post-filter: trim, drop `<30` chars, hard-cap to `700` chars.
756- Added `SessionMultimodalError` for explicit failure signaling.
757- Added `detect_visual_pdf(pdf_bytes)` using `fitz` image detection (`page.get_images()`).
758- Gemini embedding integration:
759  - model: `models/gemini-embedding-2-preview`,
760  - `output_dimensionality=768`,
761  - `task_type="retrieval_document"` for chunks,
762  - `task_type="retrieval_query"` for query.
763- Retrieval behavior:
764  - cosine similarity,
765  - fixed `top_k=3`,
766  - returns `text`, `page_num`, `score`.
767 
768### `src/pipeline.py` (conditional session multimodal routing)
769- Extended `run(...)` signature with optional `session_uploads` metadata while preserving existing `session_documents` behavior.
770- Added multimodal activation gate (all must pass):
771  - session uploads present,
772  - `USE_SESSION_MULTIMODAL_EMBEDDING=true`,
773  - `GEMINI_API_KEY` set,
774  - uploaded PDF contains images.
775- On activation success:
776  - builds session multimodal index,
777  - retrieves top multimodal chunks and appends texts to context before compression.
778- On failure (`SessionMultimodalError`):
779  - logs warning and falls back to text-only session path.
780- Added observability fields:
781  - `multimodal_embedding_used`,
782  - `multimodal_embedding_reason`,
783  - `multimodal_hits_count`.
784 
785### `ui.py` (upload metadata path + dynamic title)
786- Upgraded upload extraction to produce structured session payload:
787  - filename/mime,
788  - raw PDF bytes,
789  - per-page text map (`pdf_text_by_page`),
790  - combined extracted text.
791- Pipeline calls now pass both:
792  - `session_documents` (text path),
793  - `session_uploads` (metadata path for multimodal detection/retrieval).
794- Header title now updates from uploaded document name (stem) for session context visibility; defaults back to `Notre Dame Assistant` when no file is loaded.
795 
796### Config and dependencies
797- `.env.example` additions:
798  - `GEMINI_API_KEY=`
799  - `USE_SESSION_MULTIMODAL_EMBEDDING=false`
800  - comments clarifying optional feature behavior and fallback.
801- `requirements.txt` additions:
802  - `google-generativeai`
803  - `pymupdf`
804 
805### Runtime checks performed
806- Lint checks passed for touched files.
807- Compile checks passed for:
808  - `src/session_multimodal_retriever.py`
809  - `src/pipeline.py`
810  - `ui.py`
811- Streamlit restarted successfully and verified reachable at `http://localhost:8501`.
812 
813## Updates — May 1, 2026 (session multimodal index cache + reuse)
814 
815_Appended; all sections above remain as-is._
816 
817### `src/session_multimodal_retriever.py` (session-scoped in-memory Chroma cache)
818- Upgraded from per-query in-memory list scoring to a session-scoped in-memory Chroma collection:
819  - session key: `md5(pdf_bytes)[:12]`,
820  - collection name: `session_<session_id>`,
821  - single shared in-memory Chroma client per process.
822- Added `GeminiEmbeddingFunction` wrapper for Chroma-compatible embeddings:
823  - Gemini model: `models/gemini-embedding-2-preview`,
824  - `output_dimensionality=768`,
825  - query path still uses `task_type="retrieval_query"`.
826- `build_index()` now caches embeddings in collection:
827  - if collection already has documents, it skips re-embedding (cache hit),
828  - otherwise it chunks, embeds once, and stores ids/documents/embeddings/metadata.
829- `retrieve()` now queries Chroma directly with `query_embeddings=[...]` and returns top-3 hits.
830- Added retriever stats used for observability:
831  - `chunk_count`,
832  - `avg_chunk_tokens`,
833  - `last_build_reused_cache`.
834 
835### `src/pipeline.py` (retriever reuse across follow-up queries)
836- Added pipeline-level retriever cache:
837  - `self._session_multimodal_retrievers: dict[str, SessionMultimodalRetriever]`.
838- For visual uploads:
839  - computes `session_id`,
840  - reuses existing retriever instance when same PDF is queried again,
841  - falls back to create/build only on first encounter.
842- Added cache-hit logging:
843  - `[SessionMultimodal] reusing index for session_<id>, skipping <n> chunk embeddings`.
844 
845### New observability fields (result + retrieval diagnostics)
846- Added:
847  - `session_index_rebuilt` (True on first build, False on cache hit),
848  - `session_index_chunk_count`,
849  - `session_embedding_tokens_saved` (estimated via `chunk_count * avg_chunk_tokens` on cache hit).
850- Existing multimodal fields remain unchanged:
851  - `multimodal_embedding_used`,
852  - `multimodal_embedding_reason`,
853  - `multimodal_hits_count`.
854 
855### Quick verification
856- Compile checks passed:
857  - `src/session_multimodal_retriever.py`
858  - `src/pipeline.py`
859- Lint checks passed for both touched files.
860 
861## Updates — May 1, 2026 (session retrieval mode toggle + upload-session fast path + generic UI cleanup)
862 
863_Appended; all sections above remain as-is._
864 
865### `src/pipeline.py` (session retrieval policy controls + latency improvements)
866- Added `session_retrieval_mode` to `RAGPipeline.run(...)`:
867  - supported values: `upload_only`, `hybrid`,
868  - default fallback from env (`SESSION_RETRIEVAL_MODE`) or `hybrid`.
869- Retrieval behavior now mode-aware:
870  - `upload_only`: skips corpus retrieval (`k=0`) and uses uploaded/session context only,
871  - `hybrid`: uses corpus retrieval + session context merge.
872- Added session upload-only extractive fast path:
873  - for factual-style prompts (e.g., short `what is ...`),
874  - extracts answer from session sentences directly before router generation.
875- Added session-scoped exact-match query cache:
876  - key: `(session_scope_hash, normalized_query)`,
877  - `session_scope_hash` derives from uploaded file bytes/text + retrieval mode,
878  - prevents cross-file contamination while enabling repeat-query speedups in the same upload session.
879- Added observability fields:
880  - `session_retrieval_mode`,
881  - `session_corpus_supplement_used`,
882  - `session_corpus_supplement_count`,
883  - plus cache visibility via result-level `cache_hit` / `cache_similarity`.
884 
885### `ui.py` (mode toggle + cache visibility + generic assistant copy)
886- Added visible session retrieval mode toggle in chat UI when files are uploaded:
887  - `Upload only (ignore base corpus)`,
888  - `Hybrid (uploaded files + base corpus)`.
889- Passed `session_retrieval_mode` into `pipeline.run(...)`.
890- Added visible `cache hit` pill in chat metadata strip:
891  - shows `cache hit` and, when available, similarity score.
892- Generalized assistant branding and messaging:
893  - replaced Notre-Dame-specific title/subtitle and prompts with generic assistant language,
894  - updated uploader label to `Upload file (PDF/TXT)`,
895  - generalized fallback/off-topic messaging to context-based wording.
896- Updated visual styling:
897  - switched to generic robot icon (`🤖`),
898  - increased icon size and applied high-contrast, bold header color for better visibility.
899 
900### Runtime behavior changes observed
901- Repeated identical queries in the same upload session now short-circuit on session cache.
902- Simple definition-style questions in `upload_only` mode can avoid local LLM generation via extractive answer.
903- Previous startup `403` symptoms traced to proxy/HF metadata requests; runtime stabilized by launching with:
904  - `HF_HUB_OFFLINE=1`,
905  - `TRANSFORMERS_OFFLINE=1`,
906  - and (for responsiveness during tests) `USE_SESSION_MULTIMODAL_EMBEDDING=false`.
907 
908### Quick verification
909- Compile checks passed:
910  - `src/pipeline.py`
911  - `ui.py`
912- Lint checks passed for touched files.
913- Streamlit restarted successfully and reachable at `http://localhost:8501`.
914 
915## Updates — May 1, 2026 (upload-session answer isolation + grounding allowlist + text-only session embeddings)
916 
917_Appended; all sections above remain as-is._
918 
919### `ui.py` (prevent answer bleed in upload sessions)
920- Updated short-query contextual stitching behavior:
921  - when `session_uploads` is present, skip `build_contextual_query(...)` stitching entirely,
922  - each uploaded-file query is treated independently to prevent prior-turn topic contamination.
923- Existing stitching behavior remains unchanged when no uploads are present.
924 
925### `src/pipeline.py` (grounding false-positive reduction for technical terms)
926- Added a technical entity allowlist used by `_grounding_gate(...)` filtering before unsupported-entity checks:
927  - `BM25`, `TF-IDF`, `BERT`, `embeddings`, `cosine`, `LangChain`, `LlamaIndex`, `ChromaDB`, `RAG`, `LLM`, `API`.
928- Result: standard technical acronyms from uploaded docs no longer trigger avoidable `unsupported_entities` failures.
929 
930### `src/session_multimodal_retriever.py` + `src/pipeline.py` (all-upload activation with Gemini/BGE routing)
931- Added `use_gemini: bool` to `SessionMultimodalRetriever`:
932  - `use_gemini=True`: Gemini embeddings path (`session_<session_id>` collection),
933  - `use_gemini=False`: local BGE path via `get_embedding_model("BAAI/bge-small-en-v1.5")` (`session_text_<session_id>` collection), zero Gemini API cost.
934- Pipeline session retrieval activation now runs for all uploads (not only visual+key):
935  - visual PDF + Gemini key → `use_gemini=True`,
936  - text-only upload (or visual upload without key) → `use_gemini=False`.
937- Kept manual query embedding behavior (`query_embeddings`) unchanged.
938- Removed pipeline-level session retriever object cache usage so in-memory session collection reuse is owned by the retriever module cache.
939 
940### Quick verification
941- Compile checks passed:
942  - `src/session_multimodal_retriever.py`
943  - `src/pipeline.py`
944  - `ui.py`
945- Lint checks passed for touched files.
946 
947## Updates — May 1, 2026 (complex-query grounding gate coverage-gated bypass)
948 
949_Appended; all sections above remain as-is._
950 
951### `src/pipeline.py` (call-site only, no grounding core changes)
952- Updated the main `run()` grounding-gate call site to support a coverage-gated bypass for analytical complex queries:
953  - if `complexity == "complex"` and `retrieval.coverage_score >= 0.55`, skip strict entity grounding for that turn,
954  - set `grounded_ok = True` with structured `grounding_meta`:
955    - `reason: "skipped_for_complex_query"`
956    - `unsupported_entities: []`
957    - `coverage_score: <value>`
958  - emit explicit runtime log via `print(...)` including coverage value.
959- All other behavior remains unchanged:
960  - complex queries with weak retrieval (`coverage < 0.55`) still run `_grounding_gate(...)`,
961  - simple/medium queries always run `_grounding_gate(...)`,
962  - `_grounding_gate()` implementation itself was not modified.
963 
964### Quick verification
965- Compile check passed:
966  - `src/pipeline.py`
967- Lint checks passed for touched files.
968