AGENTS.md

1# last30days Skill
2 
3Agent Skills package for researching any topic across Reddit, X, YouTube, and web. Installable across Claude Code (most common host), Codex, Cursor, GitHub Copilot, Gemini CLI, Grok (xAI), and 50+ other [Agent Skills](https://agentskills.io) hosts. Python scripts with multi-source search aggregation.
4 
5## Structure
6- `skills/last30days/SKILL.md` — canonical skill definition / runtime spec the model reads when the slash command fires
7- `skills/last30days/scripts/last30days.py` — main research engine
8- `skills/last30days/scripts/lib/` — search, enrichment, rendering modules
9- `skills/last30days/scripts/lib/vendor/bird-search/` — vendored X search client
10- `docs/solutions/` — documented solutions to past problems (bugs, best practices, workflow patterns), organized by category with YAML frontmatter (`module`, `tags`, `problem_type`)
11- `CONCEPTS.md` — shared domain vocabulary (Skill, Engine, Harness, Beta channel) — relevant when orienting to the codebase or discussing project terminology
12- `CONFIGURATION.md` — user-facing knobs (env vars, flags, per-host install patterns); keep in sync per the rules below
13- `CHANGELOG.md` — structured release history built by towncrier at release time (launch copy lives in GitHub Releases)
14- `changelog.d/` — per-PR news fragments; feature PRs write here, never edit `CHANGELOG.md` directly
15- `CONTRIBUTING.md` — setup, fragments, and release notes for humans and agents (towncrier is release-only)
16- `.github/scripts/prepare_release.py` — lockstep version bump + towncrier build (release PRs only)
17- `HERMES_SETUP.md` — install instructions for the Hermes harness specifically
18 
19## Orientation
20- This is an Agent Skills package, not a CLI tool. The product is the slash-command-invoked skill (`/last30days <topic>` in most harnesses); `scripts/last30days.py` is implementation. Claude Code is the most common host but not the only one — features must work across every harness the skill installs into.
21- Feature design starts from the slash-command UX. A new engine flag with no SKILL.md integration is incomplete — the model invoking the skill won't know the flag exists.
22- README and PR examples show `/last30days <topic>` first. Direct CLI invocation (`python3 scripts/last30days.py ...`) is a fallback for scripting, cron, and dev-time engine testing; label it as such, never as the primary path.
23- Slash commands don't pass shell mechanics through. `/last30days OpenClaw --emit=html | pbcopy` is invalid in any harness — either use the slash form (no flags or pipes; let the model translate user intent into engine flags) or use the direct CLI form (full `python3 ...` with explicit flags and a real shell).
24 
25## Commands
26```bash
27# Dev/fallback: direct engine invocation (scripting, cron, or engine testing only).
28# Saves to $LAST30DAYS_MEMORY_DIR when set in shell or ~/.config/last30days/.env;
29# add --save-dir <path> for a one-off override. Mirrors LAST30DAYS_STORE convention.
30python3 skills/last30days/scripts/last30days.py &quot;test query&quot; --emit=compact
31npx skills add . -g -y   # copies skill into ~/.agents/skills/<name>/ (frozen at install time); re-run to sync working-tree edits — see Rules below
32 
33# Tests (pytest, ~89 files under tests/, configured in pyproject.toml)
34uv run pytest                              # full suite
35uv run pytest tests/test_dedupe_v3.py      # single file
36uv run pytest tests/test_dedupe_v3.py -k some_case   # single case
37uv run pytest --cov                        # with coverage (skips lib/vendor/)
38 
39# Release prep (maintainers / release automation — not feature PRs):
40# Prefer GitHub Actions → "Prepare release". Local equivalent:
41uv run python .github/scripts/prepare_release.py --bump patch   # or --version X.Y.Z
42```
43 
44Python 3.12+ required. Use `uv` for the env; the venv lives at `.venv/`.
45 
46## Changelog and releases (agents)
47 
48Agents open most PRs. Follow this so `CHANGELOG.md` stops conflicting and versions stay lockstep:
49 
501. **Feature/fix PRs:** add `changelog.d/<pr-or-issue>.<type>.md` (`added` / `changed` / `fixed` / `removed` / `deprecated` / `security`) when the change belongs in the next release notes. See `changelog.d/README.md` and `CONTRIBUTING.md`. Fill the PR template’s Summary, Agent disclosure, and Relationship sections.
512. **Never** edit `CHANGELOG.md` in a feature PR. **Never** bump version strings in `pyproject.toml`, `SKILL.md`, plugin/marketplace JSON, or `uv.lock` outside a release PR. CI (`changelog-guard.yml`) enforces this.
523. **Nothing for release notes:** omit the fragment, check Skip changelog in the template, and add the `skip-changelog` label.
534. **Cutting a release:** run Actions → **Prepare release** (patch/minor/major). That opens a `chore(release): bump version to X.Y.Z` PR which runs towncrier and bumps every lockstep surface. Merging to `main` triggers **Tag release**, which pushes `vX.Y.Z` and existing `release.yml` publishes `.skill` / `.mcpb` artifacts. Do not hand-edit ten version files. Contributors do not need a global towncrier install — `uv sync --group dev` (or the Action) provides it for release prep only.
545. Lockstep gate remains `tests/test_plugin_contract.py::test_versions_match_across_manifests`. Workflow contract: `tests/test_changelog_workflow.py`.
55 
56## Rules
57- `lib/__init__.py` must be bare package marker (comment only, NO eager imports)
58- One-time setup: `npx skills add . -g -y` copies the skill into `~/.agents/skills/<name>/` (real directory) and, for harnesses that support symlinked skill dirs, drops a per-host symlink pointing at that copy. **Working-tree edits do NOT propagate automatically** — the `~/.agents/skills/<name>/` copy is frozen at install time. To sync after edits, re-run `npx skills add . -g -y`. For live-edit on a dev machine, replace the install copy with a symlink to the working tree: `ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days` (run from the repo root).
59- Git remote: origin = public (`mvanhorn/last30days-skill`)
60- Do not reduce `fail_under` in `pyproject.toml` (`[tool.coverage.report]`) without documenting why in the PR. The coverage gate is a floor meant to rise over time, not to be relaxed when new code is under-tested.
61- Every `lib/*.py` call to `log.source_log(...)` must pass `tty_only=False`. The default is `True`, which silently drops every line when stderr isn't a TTY (Claude Code, Codex, CI, captured output) — turning source observability into invisible failure. Enforced by `tests/test_source_log_visibility.py`.
62- **CLI-gated optional sources** (Digg via `digg-pp-cli`, YouTube via `yt-dlp`) activate only when `shutil.which` resolves the binary on the **agent subprocess PATH** — not merely when the file exists on disk. First-run setup installs Digg through `@mvanhorn/printing-press-library` (default `$HOME/.local/bin`); Hermes/OpenClaw gateways often need that directory on PATH. Setup must distinguish PATH-visible installs from off-PATH binaries and must not claim "now active" unless the engine gate would pass. See `docs/solutions/integration-issues/digg-cli-agent-path-setup-wizard.md`.
63- **First-run onboarding is consent-driven, model-led, and host-split.** The setup subprocess does only mechanical work (cookie reads, tool installs, GitHub device-auth, and emitting the engine-owned welcome via `--welcome`) — it cannot prompt, so consent lives in `SKILL.md` Step 0. Two flows avoid model-authored prose that Claude Code folds or the model skips: in the **Modal Flow** the welcome pitch is embedded in the setup modal's question (the AskUserQuestion modal is the only always-fully-visible surface — a separate welcome message or `--welcome` Bash run gets buried behind "ctrl+o to expand"); the **Non-Modal Prose Flow** still uses `last30days.py --welcome` (relayed verbatim) since it has no modal. The GitHub device code is surfaced by a two-command split — `setup --github-start` returns the code fast (foreground, copies to clipboard) and `setup --github-poll` waits for authorization (`setup --github` still chains both for back-compat). Step 0 has TWO branches: a **Claude Code Modal Flow** (the restored v3.0.0 `AskUserQuestion`-driven NUX — welcome, Auto/Manual/Skip, cookie consent, ScrapeCreators offer, `INCLUDE_SOURCES` opt-in, first-topic picker) for hosts with modals, and a **Non-Modal Prose Flow** for hosts without (OpenClaw, Codex, Cursor, Gemini CLI, Grok). Both ask before reading cookies, surface the macOS Full Disk Access fix on permission-denied, and offer the ScrapeCreators GitHub signup (10,000 free calls) on every first run. A successful `setup --github` persists `SCRAPECREATORS_API_KEY` automatically (via `setup_wizard.write_api_key`, 0o600) and masks the key in stdout. Do NOT collapse the modal flow back into a bare silent `setup` call or flatten it to prose-only — the guided modals are the feature (they eroded once and were restored). The onboarding contract is locked by `tests/test_onboarding_contract.py`. The Step 5 source opt-in is two tiers, both comment-enabled: **Recommended** (TikTok + Instagram posts AND top comments, plus YouTube comments — `INCLUDE_SOURCES=tiktok,instagram,youtube_comments,tiktok_comments,instagram_comments`) and **Everything** (also Threads + Pinterest). Comments are on by default (posts on → comments on for all three platforms); **Threads and Pinterest are the only opt-in extras**, appearing only in the Step 5 Everything option, never in the welcome or the Step 4 offer. Instagram comments are fetched via ScrapeCreators (`/v2/instagram/post/comments`, ranked by `comment_like_count`) with full vote-weighting parity to YouTube/TikTok (a dedicated `_instagram_engagement` carve-out, the `_VOTE_LOG_REFERENCE`/label/threshold entries). The cross-platform "Top Community Comments" list (`render._render_top_comments`) selects **round-robin by within-platform rank** (every platform's #1, then #2, then #3) so a viral platform can't crowd out a smaller one, and drops the per-platform absolute floor so a less-watched video's killer low-vote comment still surfaces.
64 
65## Security hygiene
66- Never commit real API keys, browser cookies, auth tokens, app passwords, access tokens, or `.env` contents.
67- Use the env-based auth patterns in `skills/last30days/scripts/lib/env.py`; tests and fixtures must use obvious dummy values only.
68- Keep examples safe by redacting secrets and avoiding copy/pasteable live credentials in docs, fixtures, and test data.
69- Do not weaken or disable the advisory security workflow (`.github/workflows/security.yml`) without explaining why in the PR description or review thread.
70 
71## Maintaining README translations
72 
73`README.md` is the canonical English README. When changing it, reflect the same substantive updates in `README.fr.md`, `README.de.md`, `README.es.md`, `README.pt-BR.md`, `README.ja.md`, and `README.zh-CN.md`, preserving commands, links, tables, and reciprocal language navigation.
74 
75## Maintaining CONFIGURATION.md
76 
77`CONFIGURATION.md` is the user-facing configuration reference — save paths, per-source API keys, web-search backend priority, trend-monitoring stack, per-client install patterns. Distinct from `SKILL.md` (the canonical runtime spec).
78 
79Update `CONFIGURATION.md` when:
80 
81- adding a new env var (e.g. `LAST30DAYS_*`, `BSKY_*`, `*_API_KEY`)
82- adding a new CLI flag that affects configuration (e.g. `--store`, `--web-backend`)
83- adding a new per-client install pattern (Claude Code, Gemini, Codex, Cursor, Grok, Hermes…)
84- adding a new optional source that requires its own credential
85- changing the priority order of config layers (per-run flag > env > `.env` file > defaults)
86 
87Keep the existing structure organized by how often each layer is touched: per-run flags → env vars / `.env` → optional trend-monitoring stack → per-client patterns. Add new content into the right section rather than appending at the end.
88 
89When a new config concept lands in `SKILL.md` or `AGENTS.md`, mirror the user-facing knob in `CONFIGURATION.md` so non-agent readers can configure the skill without reverse-engineering it from the runtime spec.
90 
91## Plugin manifests (Grok)
92 
93The repo doubles as a native Grok Build plugin via `.grok-plugin/plugin.json` + `.grok-plugin/marketplace.json`. Grok also reads `.claude-plugin/*` for compatibility; the native pair is the first-class lane and what an official xAI marketplace listing points at. The self-hosted catalog uses a bare Git URL source (`{"source":"url","url":"https://github.com/mvanhorn/last30days-skill.git"}`) so `grok plugin marketplace add mvanhorn/last30days-skill` tracks HEAD — not a self-referential local `path: "."` (Grok does not enumerate those). Version lockstep with Claude/Codex/Gemini manifests is enforced by `tests/test_plugin_contract.py`. Validate with `grok plugin validate .`.
94 
95## Submitting to the xAI plugin marketplace
96 
97Getting last30days into xAI's official catalog (`xai-org/plugin-marketplace`) is an outbound PR to *their* repo — an index that only points at our source, so nothing of last30days is vendored there. Do this **after** the change you want to ship has merged to `main`: the entry pins a commit that must already exist.
98 
991. Fork `xai-org/plugin-marketplace` and branch from `main`.
1002. Get the commit to pin — a full 40-char lowercase SHA; a branch, tag, or short SHA is rejected by their validator:
101```bash
102   git ls-remote https://github.com/mvanhorn/last30days-skill.git HEAD
103```
1043. Add one entry to their `.grok-plugin/marketplace.json` under `plugins[]`, a remote source pinned to that SHA:
105```json
106   {
107     "name": "last30days",
108     "description": "Research any topic across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, and 5+ more sources. AI agent scores by upvotes, likes, and real money - not editors.",
109     "category": "productivity",
110     "source": {
111       "source": "url",
112       "url": "https://github.com/mvanhorn/last30days-skill.git",
113       "sha": "<full-40-char-sha-from-step-2>"
114     },
115     "homepage": "https://github.com/mvanhorn/last30days-skill",
116     "keywords": ["last30days", "last 30 days"]
117   }
118```
1194. Regenerate their component index (never hand-edit it) and validate exactly as their CI does:
120```bash
121   python3 scripts/generate-plugin-index.py
122   python3 scripts/validate-catalog.py
123   python3 scripts/generate-plugin-index.py --check
124```
1255. Open the PR, fill in their template, and wait for code-owner review.
126 
127To roll out a later update in their catalog, bump the pinned `sha` in the existing entry — never open a second, parallel entry.
128 
129Do not confuse this with our own `.grok-plugin/marketplace.json`: that file makes this repo directly addable as a Grok marketplace (`grok plugin marketplace add mvanhorn/last30days-skill`) and uses a **bare URL** source (no SHA) so it tracks HEAD; the xAI entry above lives in *their* repo and uses a **remote** source pinned to a SHA.
130 
131## Beta channel
132 
133Experimental changes get tested on `mvanhorn/last30days-skill-private`, which installs as a parallel `/last30days-beta` slash command. Beta-only changes never ship to public without a review PR here. Workflow guide lives at `BETA.md` in the private repo. Plan that established this setup: `docs/plans/2026-04-17-005-feat-beta-skill-from-private-repo-plan.md`.
134