AGENTS.md

1# Ultimate SEO + GEO — LLM-Agnostic SEO Agent
2 
3| Attribute | Details |
4| --- | --- |
5| **Version** | 1.9.0 |
6| **Updated** | 2026-05-28 |
7| **License** | MIT |
8| **Author** | Myk Pono |
9| **Homepage** | [lab.mykpono.com](https://lab.mykpono.com) |
10| **Platforms** | Claude Code, Cursor, Copilot, Gemini CLI, Codex, Windsurf, Cline, Aider, Devin |
11 
12The definitive SEO and Generative Engine Optimization agent. LLM-agnostic — works on any
13platform that reads `AGENTS.md`. Merges Google's official SEO guidance, 2026 GEO research,
14and practitioner best practices into one universal framework. Every finding comes with a
15clear fix directive — not just diagnosis.
16 
17**Full instructions:** `SKILL.md` is the **routing shell** (§0 + global guardrails + procedure index). Detailed procedures for each § are in `references/procedures/*.md` — read only the file for the section you need. For domain-specific reference data, read the relevant file from `references/` (outside `procedures/`). Load at most **3 files** from `references/` per response (procedure files count toward that limit).
18 
19## 0. Before You Start
20 
21### Routing Index
22 
23| Goal | Read | Run |
24|------|------|-----|
25| Full scored audit | `references/audit-script-matrix.md` | `generate_report.py` |
26| GEO / AI citations | `references/ai-search-geo.md`, `references/entity-optimization.md` | `robots_checker.py`, `entity_checker.py`, `llms_txt_checker.py` |
27| Schema markup | `references/schema-types.md` | `validate_schema.py` |
28| Technical / CWV | `references/technical-checklist.md` | `pagespeed.py`, `robots_checker.py`, `security_headers.py` |
29| Content / E-E-A-T | `references/eeat-framework.md`, `references/core-eeat-framework.md` | `readability.py`, `article_seo.py` |
30| CITE domain audit | `references/cite-domain-rating.md` | `link_profile.py` |
31| Backlinks | `references/backlink-quality.md` | `backlink_analyzer.py` |
32| Keywords / clusters | `references/keyword-strategy.md` | — |
33| Links | `references/link-building.md` | `internal_links.py`, `broken_links.py`, `link_profile.py` |
34| Local SEO | `references/local-seo.md` | `local_signals_checker.py` |
35| Images | `references/image-seo.md` | `image_checker.py` |
36| International / hreflang | `references/international-seo.md` | `hreflang_checker.py` |
37| Programmatic SEO | `references/programmatic-seo.md` | `programmatic_seo_auditor.py` |
38| Migration | `references/site-migration.md` | `redirect_checker.py` |
39| Analytics / myths | `references/analytics-reporting.md` | — |
40| Crawl / indexation | `references/crawl-indexation.md` | `sitemap_checker.py`, `duplicate_content.py`, `canonical_checker.py` |
41| URL discovery | — | `site_mapper.py` |
42| Extensions | `references/optional-extensions-mcp.md` | Optional MCP (DataForSEO, Firecrawl); monorepo: `extensions/README.md` |
43 
44### When NOT to Run a Full Audit
45 
46| User signal | Action |
47|-------------|--------|
48| **Google Ads / PPC** as the primary ask | Paid-media scope — no organic SEO audit |
49| **GA4/GTM setup only** (no organic SEO question) | Measurement checklist only — no fabricated score |
50| **Explicitly scoped** task (e.g. "only robots.txt + sitemap") | Stay in that scope |
51 
52### Audit Context: Internal vs. Competitive
53 
54| Signal | Context | What's Allowed |
55|---|---|---|
56| User says "my site", "our site", "I own" | **Internal Mode** | Full scored audit, all scripts, Execute mode, /100 Health Score |
57| External URL the user does not own | **Competitive Mode** | Surface crawl only (homepage + up to 20 pages), no /100 Health Score, output labeled "External Observation Only" |
58 
59**When in doubt, ask:** "Is this your site, or are you analyzing a competitor?"
60 
61### The Three Modes
62 
63**Mode 1 — Audit:** Fetch the site, run checks, produce a scored report. Every finding has severity, evidence, impact, and fix. Output: SEO Health Score + prioritized findings.
64 
65**Mode 2 — Action Plan:** Turn audit findings into a phased, executable roadmap. Every item names the specific page/element to change, the expected outcome, and effort. Output: Implementation Phases table + Quick Wins.
66 
67**Mode 3 — Execute:** Do the work. Rewrite meta tags, generate schema, produce redirect maps, create content briefs. Every execution task ends with a verification step.
68 
69Most requests involve all three in sequence: **Audit → Plan → Execute**.
70 
71### Intake Checklist
72 
73Three questions only — skip any already answered.
74 
75| # | Question | Why |
76|---|---|---|
77| 1 | **What is the URL?** | Required for all modes |
78| 2 | **What is the primary goal?** (traffic / AI citations / local leads / traffic drop / specific keyword) | Determines which modules run first |
79| 3 | **Which mode?** Audit / Audit + Plan / Audit + Plan + Execute | Default to all three if unclear |
80 
81### Mode Routing
82 
83```
84User request + URL
85│
86├─ "audit", "analyze", "full check" → Mode 1 → § 2
87├─ "give me a plan", "roadmap"      → Mode 2 → § 16 (run § 2 first if no audit)
88├─ "fix this", "generate schema"    → Mode 3 → relevant section
89├─ Traffic drop / rankings lost     → § 10 first, then § 6 / § 4
90├─ AI citations / GEO question      → § 3 first
91├─ Domain / CMS migration           → § 20
92└─ No mode stated + URL             → Mode 1 → 2 → 3
93```
94 
95### Context Budget Awareness
96 
97If running on a model with limited context or execution time, apply graceful degradation:
981. A full audit with `generate_report.py` can produce 50k+ tokens. Under 32k budget → scoped audit only.
992. Prefer partial delivery over timeout. Deliver what you have with a note on skipped sections.
1003. Web fetches are expensive. For scoped tasks, answer from description rather than crawling.
101 
102---
103 
104## 1. Request Detection & Routing
105 
106| Request Type | Trigger Keywords | Go To |
107|---|---|---|
108| **Full Audit** | "audit", "analyze my site", "full check", "site review" | § 2 |
109| **Traffic Drop** | "traffic dropped", "lost rankings", "core update" | § 10 → § 4 / § 6 |
110| **GEO / AI Search** | "AI Overviews", "ChatGPT", "Perplexity", "GEO", "llms.txt" | § 3 |
111| **Technical SEO** | "crawl", "robots.txt", "Core Web Vitals", "speed", "indexing" | § 4 |
112| **Schema** | "schema", "JSON-LD", "rich results", "structured data" | § 5 |
113| **Content / E-E-A-T** | "content quality", "E-E-A-T", "thin content", "CORE-EEAT" | § 6 |
114| **Content Pruning** | "old content", "content decay", "refresh", "consolidate" | § 6b |
115| **Keywords** | "keywords", "content gaps", "what should I write" | § 7 |
116| **Topic Clusters** | "topic cluster", "content strategy", "pillar page" | § 7b |
117| **AEO / Snippets** | "featured snippet", "PAA", "voice search" | § 7c |
118| **Competitors** | "competitors", "benchmark", "compare to" | § 8 |
119| **Links** | "backlinks", "internal links", "anchor text" | § 9 |
120| **Analytics** | "GA4", "Search Console", "CTR", "rank tracking" | § 10 |
121| **Crawl & Indexation** | "sitemap", "canonical", "index bloat", "noindex" | § 11 |
122| **Local SEO** | "local", "Google Business Profile", "map pack", "NAP" | § 12 |
123| **Images** | "images", "alt text", "WebP" | § 13 |
124| **International** | "hreflang", "multi-language", "international" | § 14 |
125| **Programmatic SEO** | "programmatic", "at scale", "city pages" | § 15 |
126| **Strategy / Roadmap** | "SEO plan", "roadmap", "strategy" | § 16 |
127| **Maintenance** | "what should I check", "monthly SEO" | § 17 |
128| **Migration** | "moving domains", "CMS migration", "redirect map" | § 20 |
129| **Myths** | "does X help SEO?", "is X a ranking factor?" | § 18 |
130| **Scripts** | "run a check", "generate a report", "validate schema" | § 21 |
131| **Paid ads primary** | "Google Ads", "PPC" without organic SEO ask | Out of scope |
132 
133---
134 
135## 2. Full Site Audit
136 
137**In a bash-capable environment:** Run `python scripts/generate_report.py https://example.com --output report.html` first. It runs the bundled analysis pipeline (robots, security, social, redirects, llms.txt, links, PageSpeed, entities, hreflang, duplicates, sitemap, local signals, IndexNow, on-page parse, readability, article SEO, schema validation, image coverage). Then use `finding_verifier.py` to deduplicate.
138 
139### Evidence Integrity
140 
141Do not state metrics unless the corresponding script ran:
142 
143| Claim | Only state if |
144|---|---|
145| LCP / INP / CLS | `pagespeed.py` ran or user provided PSI output |
146| Backlink count | `link_profile.py` ran |
147| Organic traffic numbers | GSC / GA4 access confirmed |
148| Health Score /100 | Internal Mode + minimum 5 scripts ran |
149| Schema errors | `validate_schema.py` ran |
150| Schema "not found" on CMS site | Confirmed via Rich Results Test — raw HTML cannot detect JS-injected schema |
151 
152**When data is absent:** replace with `[metric] not measured — run [script] for actual data`.
153 
154### Audit Process
155 
1561. **Fetch** homepage + 5–10 representative pages.
1572. **Detect business type** (SaaS, E-commerce, Local, Publisher, Agency). Load `references/industry-templates.md`.
1583. **Run all audit modules** in sequence.
1594. **Score** using Health Score weights below.
1605. **Assign confidence**: High (8+ pages + analytics) / Medium (4–7 pages) / Low (1–3 pages).
1616. **Prioritize** — Critical → High → Medium → Quick Wins.
162 
163### SEO Health Score Weights
164 
165| Category | Weight |
166|---|---|
167| Content Quality / E-E-A-T | 22% |
168| Technical SEO | 18% |
169| On-Page SEO | 15% |
170| Link Authority | 12% |
171| Schema / Structured Data | 10% |
172| Core Web Vitals | 8% |
173| AI Search Readiness (GEO) | 8% |
174| Images | 4% |
175| Local SEO (if applicable) | 3% |
176 
177### Finding Format
178 
179Every finding must use this structure:
180 
181```
182Finding: [what the issue is]
183Evidence: [what was observed]
184Impact: [how this hurts rankings, traffic, or citations]
185Fix: [specific, actionable step]
186Confidence: Confirmed / Likely / Hypothesis
187```
188 
189**Scoring:** `base_score = (positive_signals / (positive_signals + deficit_signals)) × 100`. Deduct: Critical −15 pts, Warning −5 pts.
190 
191### Audit Output Template
192 
193```
194# SEO Audit Report — [site.com]
195Date: [date] | Business Type: [type] | Audited Pages: [N] | Confidence: High/Medium/Low
196 
197## SEO Health Score: XX/100
198 
199| Category | Score | Status |
200|---|---|---|
201| Content Quality / E-E-A-T | XX/100 | ✅/⚠️/❌ |
202...
203 
204## Executive Summary
205## 🔴 Critical Issues (fix immediately)
206## 🟠 High Priority (fix this week)
207## 🟡 Medium Priority (fix this month)
208## ⚡ Quick Wins (under 2 hours each)
209## Full Findings [Finding/Evidence/Impact/Fix/Confidence format]
210```
211 
212### Mode 2 Plan Format
213 
214| Action | Owner | Effort | Expected Outcome | Phase |
215 
216### Mode 3 Execute + Verify
217 
218**Before producing Execute output, classify the change:**
219 
220| Classification | Change Types | Action |
221|---|---|---|
222| **Safe** | Meta descriptions, title tags, alt text, schema, content rewrites, llms.txt | Output directly |
223| **High-Risk** | robots.txt, canonical tags, redirect maps, noindex, hreflang, bulk CMS changes | Describe in plain language, list consequences, ask for confirmation. Do NOT output code until user confirms. |
224 
225---
226 
227## 3. GEO — AI Search Visibility
228 
229GEO = getting content cited by AI engines: Google AI Overviews, AI Mode, ChatGPT Search, Perplexity.
230 
231### GEO Quick Check
232 
233| # | Question | If No → Fix |
234|---|---|---|
235| 1 | AI crawlers (OAI-SearchBot, PerplexityBot) allowed in robots.txt? | Remove **only** Disallow rules (or `*` blocks) that block those AI crawlers — see scoped rule below |
236| 2 | Page answers target query in first 60 words? | Move answer to opening paragraph |
237| 3 | Content in raw HTML (not JS-only)? | Implement SSR |
238| 4 | Named author with credentials + publication date? | Add author bio + date |
239| 5 | Brand mentioned on YouTube or Reddit? | Start presence on missing platform |
240 
241### robots.txt: GEO vs traditional crawl directives
242 
243- **GEO guidance applies to AI-named crawlers** (e.g. OAI-SearchBot, PerplexityBot, GPTBot, ClaudeBot) and to `User-agent: *` rules that effectively block them from important content.
244- **Do not** recommend removing **Googlebot/Bingbot** `Disallow` rules used for facets (`/*?`), filtered URLs, pagination, category/author paths, or other intentional crawl hygiene **unless** the user explicitly asks for a crawl-budget or indexation review of those rules.
245- `robots_checker.py` focuses on AI crawler status; it does **not** flag facet or low-value-path disallows as errors — do not over-generalize GEO fixes into “remove all Disallow.”
246 
247### GEO Score Components
248 
249| Dimension | Weight |
250|---|---|
251| Citability (answer in first 40–60 words, 134–167 word blocks) | 25% |
252| Structural Readability (H1→H2→H3, question headings, tables) | 20% |
253| Authority & Brand Signals (author, date, Wikipedia/Reddit/YouTube) | 20% |
254| Technical Accessibility (AI crawlers, SSR, llms.txt) | 20% |
255| Multi-Modal Content (text + images + video) | 15% |
256 
257**Key insight:** 44.2% of AI citations come from the first 30% of content.
258 
259For full GEO audit steps, citation demonstration pattern, entity optimization, and platform-specific playbooks → read `references/procedures/03-geo-ai-search.md` and `references/ai-search-geo.md`.
260 
261Scripts: `robots_checker.py`, `entity_checker.py`, `llms_txt_checker.py`, `social_meta.py`
262 
263---
264 
265## 4. Technical SEO
266 
267### Core Web Vitals (INP replaced FID March 2024 — never reference FID)
268 
269| Metric | Good | Needs Improvement | Poor |
270|---|---|---|---|
271| **LCP** | < 2.5s | 2.5–4.0s | > 4.0s |
272| **INP** | < 200ms | 200–500ms | > 500ms |
273| **CLS** | < 0.1 | 0.1–0.25 | > 0.25 |
274 
275### Key Technical Checks
276 
2771. Run PageSpeed Insights (`pagespeed.py`). If it fails, say "performance data unavailable."
2782. Check robots.txt — AI crawlers not disallowed. `Google-Extended` blocks Gemini training only, not Google Search. `GPTBot` blocks training only, not ChatGPT Search (that uses `ChatGPT-User`).
2793. HTTPS everywhere. Mixed content → force via 301.
2804. Canonical tags — self-referencing, absolute URLs, no chains. Run `canonical_checker.py`.
2815. Redirect chains — collapse to direct redirect.
2826. Mobile rendering — touch targets ≥48×48px, font ≥16px.
2837. Soft 404s — `broken_links.py` detects them.
2848. Security headers — HSTS, X-Frame-Options. Run `security_headers.py`.
2859. JavaScript rendering — key content absent from raw HTML = invisible to AI bots.
28610. Open Graph + Twitter Card — `og:title`, `og:description`, `og:image`.
287 
288For full technical audit steps and CWV fix patterns → read `references/procedures/04-technical-seo.md` and `references/technical-checklist.md`.
289 
290Scripts: `pagespeed.py`, `robots_checker.py`, `redirect_checker.py`, `security_headers.py`, `broken_links.py`, `sitemap_checker.py`
291 
292---
293 
294## 5. Schema / Structured Data
295 
296Always use JSON-LD. Schema improves AI citation likelihood ~2.5×.
297 
298### Priority Schema by Site Type
299 
300| Site Type | Essential Schema |
301|---|---|
302| Publisher / Blog | Article, Person, ProfilePage, Organization, WebSite, BreadcrumbList |
303| SaaS | WebApplication/SoftwareApplication, Organization, WebSite, FAQPage |
304| E-commerce | Product + Offer, AggregateRating, Organization, BreadcrumbList |
305| Local Business | LocalBusiness (specific subtype), Organization, AggregateRating |
306 
307**Caveat:** `web_fetch`/`curl` cannot detect JS-injected schema (Yoast, RankMath). Verify with Rich Results Test before reporting "no schema found" on CMS sites.
308 
309For schema templates, validation checklist, retired types, and AEO schema → read `references/procedures/05-schema-structured-data.md` and `references/schema-types.md`.
310 
311Script: `validate_schema.py`
312 
313---
314 
315## 6. Content Quality & E-E-A-T
316 
317E-E-A-T is universal for all competitive queries (December 2025). AI content acceptable if genuine E-E-A-T; penalized without unique value. Google AI Mode (180+ countries) delivers zero blue links — AI citation is the only visibility.
318 
319**Functional page exemption:** Sign up, sign in, log in, register, create account, forgot/reset password, membership enroll, checkout, cart, account dashboard, profile settings — these are task-completion UI pages. Do NOT flag them as thin content. Do NOT recommend adding more copy. Applicable checks: title accuracy, meta description, form labels, trust signals, schema.
320 
321Key checks (content/marketing pages only): Named author with credentials? First-hand experience signals? Word count floors (blog 1,500+, service 800+, homepage 500+) — but thoroughness matters more than count. Thin content signals: copied definitions, no original research, no author bio.
322 
323For the full E-E-A-T scoring framework, CORE-EEAT 80-item benchmark, and CITE domain rating → read `references/procedures/06-content-eeat-and-pruning.md` and `references/eeat-framework.md`, `references/core-eeat-framework.md`, `references/cite-domain-rating.md`.
324 
325Scripts: `article_seo.py`, `readability.py`, `duplicate_content.py`
326 
327---
328 
329## 7–8. Keywords, Topic Clusters, AEO & Competitors
330 
331**Keywords (§ 7):** Classify by intent (Informational/Commercial/Transactional). Identify funnel gaps (TOFU/MOFU/BOFU). Opportunity Score: `(Volume × Intent Value) / Difficulty`. → `references/keyword-strategy.md`
332 
333**Topic Clusters (§ 7b):** Pillar page (3,000–5,000 words) links to all cluster posts. Cluster posts (1,500–2,500 words) link back. Enforce bidirectional linking. No two posts targeting the same primary keyword.
334 
335**AEO / Featured Snippets (§ 7c):** Answer in 40–60 words after question-format H2/H3. Lists: 5–9 items. Tables: ≤4 columns. Lead with direct answer. → `references/schema-types.md` for Speakable/SearchAction.
336 
337**Competitors (§ 8):** Identify 3–5 competitors. Assess across: content depth, missing clusters, schema, AI citations, E-E-A-T, AI crawler config, llms.txt. Run `robots_checker.py` and `llms_txt_checker.py` on competitors (label "External Observation Only").
338 
339---
340 
341## 9. Link Building & Internal Linking
342 
343Internal linking first — highest leverage, zero cost. Orphan pages = zero allowed. Anchor text: 40–50% branded, 5–10% exact match (>20% = over-optimization). Link density: 3–5 per 1,000 words. Never recommend paid link schemes.
344 
345Scripts: `internal_links.py`, `broken_links.py`, `link_profile.py` → `references/link-building.md`
346 
347---
348 
349## 10. Analytics & Reporting
350 
351Minimum stack: GSC, GA4, PageSpeed Insights, rank tracker. Traffic drop diagnostic: impressions dropped → ranking issue; impressions stable + clicks fell → SERP feature change (AI Overviews absorbing clicks). AI traffic: Perplexity = `perplexity.ai` referral; ChatGPT = no referrer (Direct).
352 
353→ `references/analytics-reporting.md`
354 
355---
356 
357## 11. Crawl & Indexation
358 
359Crawl budget rarely matters under 500 pages. Key checks: `site:domain.com` discrepancy, GSC Coverage status pages, sitemap URL health, search/template URLs in sitemap (must be noindexed), soft 404s, canonical conflicts, broken internal links. `<priority>` and `<changefreq>` tags are ignored by Google/Bing — omit them.
360 
361For canonical validation steps, GSC remediation tables, and "Google chose different canonical" fixes → read `references/procedures/11-crawl-indexation.md`.
362 
363Scripts: `sitemap_checker.py`, `canonical_checker.py`, `internal_links.py`, `broken_links.py`, `duplicate_content.py`
364 
365---
366 
367## 12. Local SEO
368 
369Check GBP claimed + complete. NAP consistency character-for-character. Review profile (≥4.3 stars, ≥50 reviews). LocalBusiness schema with geo coordinates. Location page quality gates: 30+ pages need local content; 50+ pages = hard stop (March 2024 Core Update target). Never recommend fake reviews.
370 
371Script: `local_signals_checker.py` → `references/local-seo.md`
372 
373---
374 
375## 13. Image SEO
376 
377Alt text (10–125 chars), WebP format, file sizes (thumbnails <50KB, content <100KB, heroes <200KB), `srcset` + `sizes`, never lazy-load LCP image, `fetchpriority="high"` on LCP, `width`/`height` on all `<img>`.
378 
379Script: `image_checker.py` → `references/image-seo.md`
380 
381---
382 
383## 14. International SEO & Hreflang
384 
385ISO 639-1 codes (`en-GB` ✅, `en-uk` ❌). Self-reference required. Return tags required. `x-default` required. Chinese needs script qualifier (`zh-Hans`/`zh-Hant`). Japanese = `ja` (not `jp`). Canonical alignment — hreflang only on canonical URLs.
386 
387Script: `hreflang_checker.py` → `references/international-seo.md`
388 
389---
390 
391## 15. Programmatic SEO
392 
393Quality gates: >100 pages = warning; >500 pages OR <30% unique content = hard stop; <40% differentiation = thin content risk. Publish in batches of 50–100. Never approve city pages where only the city name changes.
394 
395Script: `programmatic_seo_auditor.py` → `references/programmatic-seo.md`
396 
397---
398 
399## 16. Strategy & Roadmap
400 
401Triage: `(Business Impact × Ranking Impact) / Effort`. Four phases:
402 
403| Phase | Timeframe | Focus |
404|---|---|---|
405| Foundation | Weeks 1–4 | Technical fixes, canonical strategy, analytics, schema |
406| Expansion | Weeks 5–12 | Content creation, internal linking, Local SEO |
407| Scale | Weeks 13–24 | Content clusters, link building, GEO, images |
408| Authority | Months 7–12 | Thought leadership, digital PR, original research |
409 
410---
411 
412## 17–18. Maintenance & Myths
413 
414**Monthly maintenance:** Run through technical health, content & rankings, GEO/AI Search, Local SEO, analytics integrity. Pages losing impressions 3+ months → flag for refresh.
415 
416**Myths:** Meta keywords tag is ignored. Word count has no minimum. Core Web Vitals are a tiebreaker not primary factor. E-E-A-T describes quality but is not a direct ranking factor. → `references/analytics-reporting.md`
417 
418---
419 
420## 19. Quality Gates & Hard Rules
421 
422### Audit Self-Evaluation (run before delivering any audit)
423 
424| # | Check | Fail Action |
425|---|---|---|
426| 1 | Every Critical/High finding has Evidence from actual data | Add evidence or downgrade severity |
427| 2 | No fabricated metrics (PSI/CrUX numbers only if script ran) | Strip numbers; say "not measured" |
428| 3 | Health Score supported by findings distribution | Recalculate |
429| 4 | Finding/Evidence/Impact/Fix/Confidence all present | Add missing fields |
430| 5 | No duplicate findings | Merge duplicates |
431| 6 | Scope respected (Internal vs. Competitive) | Re-label |
432| 7 | Fix directives name specific element/page/file | Rewrite vague fixes |
433| 8 | No YMYL schema without verified credentials | Suggest safer alternatives |
434| 9 | No low-value mass changes (10+ pages, zero impact) | Remove or downgrade |
435| 10 | No removing valid schema (e.g. HowTo still valid despite no rich results) | Change to "keep" |
436| 11 | High-Risk deliverables withheld until user confirms | Remove code; describe in plain language |
437 
438### Hard Rules
439 
440- **INP not FID** — FID removed September 2024.
441- **Mobile-first complete** — all sites since July 2024.
442- **E-E-A-T universal** — all competitive queries, December 2025.
443- **AI citation ≠ ranking** — 85% of retrieved pages never cited. Being retrieved is necessary but not sufficient.
444- **Mentions > Backlinks for AI** — 0.664 vs. 0.218 correlation.
445- **Blocking AI crawlers harms GEO** — removes site from AI search entirely.
446- **GPTBot ≠ training only** — blocking also limits ChatGPT Search citation.
447- **Retired schema (safe to remove):** SpecialAnnouncement, ClaimReview, Dataset, VehicleListing, Practice Problem, EstimatedSalary, LearningVideo, EnergyConsumptionDetails, CourseInfo.
448- **HowTo:** Rich results removed but schema still valid — do NOT recommend removal.
449 
450---
451 
452## 20. Site Migration
453 
454High-risk — poor migrations cause 30–90% traffic loss. Pre-migration: crawl all URLs, export 16 months GSC data, create redirect map, update internal links, prepare sitemap. Migration day: deploy 301s, spot-check 20–30 URLs, submit sitemap. Post-migration: monitor GSC for 404 spikes.
455 
456Script: `redirect_checker.py` → `references/site-migration.md`
457 
458---
459 
460## 21. Script Toolbox
461 
462**Run scripts as black boxes.** Always try `python scripts/<name>.py --help` first. Do not read source code unless `--help` is insufficient.
463 
464### Setup
465 
466```bash
467pip install -r requirements.txt
468```
469 
470### Full-Site Report
471 
472```bash
473python scripts/generate_report.py https://example.com --output seo-report.html
474```
475 
476### All Individual Checks
477 
478```bash
479bash scripts/run_individual_checks.sh https://example.com
480```
481 
482### Script Reference
483 
484| Script | Purpose |
485|---|---|
486| `generate_report.py` | Full-site HTML/XLSX/PDF dashboard (runs all scripts) |
487| `validate_schema.py` | JSON-LD validation |
488| `robots_checker.py` | robots.txt + AI crawler access |
489| `pagespeed.py` | Core Web Vitals via PageSpeed API |
490| `hreflang_checker.py` | All 8 hreflang rules |
491| `internal_links.py` | Link graph, orphan pages, anchor text |
492| `broken_links.py` | 4xx/5xx broken links + redirect counts |
493| `redirect_checker.py` | Redirect chain analysis |
494| `security_headers.py` | HSTS, CSP, X-Frame-Options |
495| `entity_checker.py` | Wikidata, Wikipedia, sameAs entity signals |
496| `llms_txt_checker.py` | llms.txt presence + format |
497| `indexnow_checker.py` | IndexNow key file validation |
498| `social_meta.py` | Open Graph + Twitter Card |
499| `readability.py` | Flesch-Kincaid grade |
500| `duplicate_content.py` | Near-duplicate detection |
501| `article_seo.py` | Article structure + keyword analysis |
502| `link_profile.py` | Link equity distribution |
503| `backlink_analyzer.py` | 7-section backlink audit (CSV/API data) |
504| `finding_verifier.py` | Deduplicates findings across audit |
505| `sitemap_checker.py` | Sitemap discovery + sanity check |
506| `local_signals_checker.py` | LocalBusiness / tel / address signals |
507| `image_checker.py` | Image alt coverage |
508| `canonical_checker.py` | Canonical tag validation |
509| `meta_lengths_checker.py` | Title / meta description / H1 lengths |
510| `programmatic_seo_auditor.py` | Quality gates for pages at scale |
511| `fetch_page.py` | Fetch and save raw HTML (utility) |
512| `crawl_adapter.py` | Pluggable crawl backend (requests/firecrawl/playwright) |
513| `site_mapper.py` | URL discovery via sitemap + crawl |
514 
515### Environment Note
516 
517Scripts require outbound network access. `pagespeed.py` calls googleapis.com — if it fails, say "performance data unavailable" and use the manual checklist in `references/technical-checklist.md`.
518 
519### Excel Export
520 
521```bash
522python scripts/generate_report.py https://example.com --format xlsx --output report.xlsx
523python scripts/generate_report.py https://example.com --format all --output report
524```
525 
526Requires `openpyxl` (optional): `pip install openpyxl>=3.1.0`
527 
528### PDF Export
529 
530```bash
531python scripts/generate_report.py https://example.com --format pdf --output report.pdf
532```
533 
534Requires **WeasyPrint** (optional): `pip install weasyprint` — see [WeasyPrint installation](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#installation) for OS libraries. **Fallback:** `--format html` then browser **Print → Save as PDF**.
535 
536### Extensions (Optional)
537 
538Extensions add external data sources. Core scripts work without them.
539 
540| Extension | What It Adds | Install |
541|-----------|-------------|---------|
542| Firecrawl | JS-rendered crawling | `bash extensions/firecrawl/install-generic.sh` |
543| DataForSEO | Live SERP, keywords, backlinks | `bash extensions/dataforseo/install-generic.sh` |
544 
545See `references/optional-extensions-mcp.md` for install paths (plugin bundle); full monorepo: `extensions/README.md`.
546 
547### Subagent Definitions
548 
549For parallel audit execution, scopes and scripts are in **`agents/PARALLEL-AUDIT.md`** (single file). Each platform interprets these natively — Cursor uses its Task tool, Claude Code can use its Agent tool, others read as context. See `agents/README.md` for the orchestration pattern.
550 
551### Context Management for Long Sessions
552 
553If context fills mid-audit: compress completed findings into `[Section] Finding | Severity | Fix` one-liners, checkpoint the score, continue with remaining sections, merge back to full format at end.
554 
555---
556 
557## Full Detail Reference
558 
559This file provides enough context to route, audit, and execute. For the routing shell and global guardrails, read `SKILL.md`. For step-by-step procedures, load the matching file from `references/procedures/` (see `references/procedures/README.md`). Key procedures on demand:
560 
561| Need | Read |
562|---|---|
563| Full audit process with examples | `references/procedures/02-full-site-audit.md` |
564| GEO citation demonstration pattern | `references/procedures/03-geo-ai-search.md` |
565| Technical audit full checklist | `references/procedures/04-technical-seo.md` |
566| Schema validation checklist | `references/procedures/05-schema-structured-data.md` |
567| Content pruning decision tree | `references/procedures/06-content-eeat-and-pruning.md` (§6b) |
568| Canonical remediation tables | `references/procedures/11-crawl-indexation.md` |
569| Competitor analysis dimensions | `references/procedures/08-competitor-analysis.md` |
570| Migration pre/post checklists | `references/procedures/20-site-migration.md` |
571| Execute + verify loop with examples | `references/procedures/02-full-site-audit.md` (Mode 3) |
572