AGENTS.md

1# Seed n8n instance
2 
3`seedInstance.mjs` fills a local n8n instance with a realistic-looking spread of
4projects, workflows, credentials, and data tables via the **public API**. The
5resulting dependency graph is designed to render like a real org's automation
6estate: dense intra-team clusters, sparse cross-team bridges through shared
7utility projects, a few legacy "trenchcoat" projects sitting off to the side,
8and one central data table that everything reaches through a proxy workflow.
9 
10Useful for demos, perf testing, visual QA of the workflow dependency graph, and
11poking at the workflow-index module with non-trivial input.
12 
13## Quick start
14 
15```sh
16N8N_API_KEY="<a public-api JWT for an owner/admin>" \
17  node scripts/instance-seeding/seedInstance.mjs
18```
19 
20Targets `http://localhost:5678` by default. The script is **destructive by
21default**: it deletes its own prior output (anything tagged `[seed]`) and any
22team projects whose names match the current taxonomy plus orphans from older
23runs. Personal-project entities that don't match the seed prefix are left
24alone, as are the n8n-default `My project` team projects.
25 
26### Environment variables
27 
28| Var | Default | Purpose |
29| --- | --- | --- |
30| `N8N_API_KEY` | (required) | Public-API JWT. Must have owner or admin scopes. |
31| `N8N_BASE_URL` | `http://localhost:5678` | n8n instance to seed. |
32| `CLEAR` | `false` | Set to `true` to wipe data instead. |
33| `PERSONAL_WORKFLOWS` | `50` | Amount of workflows to create in the personal project. |
34 
35Runtime is ~30–45 s for a default run (~500 workflows, ~30 projects).
36 
37## What it creates
38 
39**Projects** (30 team projects + your existing personal):
40 
41- **2 utility projects** — `Shared Platform` (technical plumbing: Audit
42  Logger, Slack Alerts Dispatcher, Sentry Error Forwarder, …) and
43  `Org Utilities` (business helpers: Tenant Resolver, Vault Reader, Feature
44  Flag Resolver, …). These are the hub workflows the rest of the org calls
45  into.
46- **25 community projects** organised into 5 themed communities of 5 projects
47  each: Revenue, Customer, Engineering, Operations, Knowledge.
48- **3 trenchcoat projects** — Legacy Migrations, Skunkworks, Founder's
49  Workflows. Smaller, internally split into 2-3 disjoint sub-systems (e.g.
50  `[HR System]`, `[Old Billing]`, `[Acme Acquisition]`), and almost entirely
51  detached from the rest of the org. They model accreted legacy state.
52 
53**Credentials** (~110):
54- Two per project max (random recipes: Notion, Slack, Postgres, GitHub, …).
55- Plus **5 global credentials** living in the utility projects (Production
56  Slack Webhook, Datadog API, GitHub platform bot, OpenAI production,
57  Vault read-only).
58 
59**Data tables** (~15–20):
60- One per ~55% of projects, with 5–20 sample rows.
61- Plus one **central data table** `seed_customers` in `Org Utilities` that
62  the entire org reaches through a single proxy workflow.
63 
64**Workflows** (~500), built in four phases:
65- **Phase 0** — Utility lynchpin workflows in `Shared Platform` and
66  `Org Utilities`, plus the Customers Proxy.
67- **Phase 1** — Leaf workflows in every project, no sub-calls.
68- **Phase 2** — Parent workflows with sub-workflow refs. Community projects
69  pick own/sibling/lynchpin; trenchcoats pick within their internal group.
70- **Phase 3** — Data-table consumer workflows (one per project DT).
71- **Phase 4** — Cross-project data-table proxies for ~6 non-utility DTs.
72 
73## Two non-obvious architectural rules
74 
751. **`DataTable` nodes can only point at tables in their own project.** Cross-
76   project access goes through a proxy workflow. The `customersProxy` in
77   `Org Utilities` is the only workflow with a direct `DataTable` node on
78   the central table; everyone else calls the proxy via `ExecuteWorkflow`.
79   Phase 4 generalises this pattern to ~6 other data tables.
80 
812. **Per-project external-ref budgets.** Each community project is capped at
82   2–5 distinct external workflow refs and 5–10 external credential refs,
83   tracked across all phases. About a third of community projects opt out of
84   utility refs entirely (some of those go fully self-contained — no
85   external refs of any kind). Trenchcoats and utility projects are exempt.
86 
87## Tunable knobs
88 
89All knobs live at the top of `seedInstance.mjs`. The ones that change the
90shape of the graph the most:
91 
92| Constant | Effect |
93| --- | --- |
94| `COMMUNITIES` | Project taxonomy. Add/remove communities or projects. |
95| `TRENCHCOAT_PROJECTS` + `TRENCHCOAT_GROUPS` | Legacy projects and their internal subsystems. |
96| `UTILITY_WORKFLOW_THEMES` | Lynchpin workflow names per utility project. |
97| `LYNCHPIN_CRED_RECIPES` | The 5 global credentials. |
98| `UTILITY_REF_PROB` (0.6) | Per-workflow probability of including a utility ref. |
99| `CENTRAL_DT_REF_PROB` (0.08) | Per-workflow probability of calling the Customers Proxy (indirect central-DT use). |
100| `ORG_UTIL_DIRECT_DT_PROB` (0.5) | Per-workflow probability that an Org Utilities phase-0 workflow gets a direct DataTable node on the central table. |
101| `EXT_WF_REF_BUDGET` / `EXT_CRED_REF_BUDGET` | Per-project distinct-ref caps. |
102| `NON_UTILITY_USING_TARGET` (9) | How many community projects opt out of utility refs. |
103| `SELF_CONTAINED_PROJECT_PROB` | Subset of the opt-out projects that go fully siloed. |
104| `sampleWorkflowCount()` | Power-law-ish size buckets per project kind. |
105 
106`SUBWF_PROB_OWN`/`_SIBLING`/`_LYNCHPIN` (in `pickCommunitySubWf`) govern how
107parent workflows route sub-calls.
108 
109## Verifying a run
110 
111The dependency-graph endpoint is the canonical view of what the seed
112produced. It includes both the community structure and the central-DT proxy
113pattern:
114 
115```sh
116curl -s "$N8N_BASE_URL/api/v1/workflows/dependency-graph?format=dot" \
117  -H "X-N8N-API-KEY: $N8N_API_KEY" | sfdp -Tsvg -Goverlap=prism > graph.svg
118```
119 
120`sfdp`/`fdp` (Graphviz force-directed layouts) reveal the cluster topology
121better than the default hierarchical `dot` layout.
122 
123Counts via API:
124 
125```sh
126for path in workflows projects credentials data-tables; do
127  echo -n "$path: "
128  curl -s "$N8N_BASE_URL/api/v1/$path?limit=1" \
129    -H "X-N8N-API-KEY: $N8N_API_KEY" | jq -r '.data | length'
130done
131```
132 
133Key numbers a default run should land near:
134- ~500 workflows, 30 team projects, ~110 credentials, ~15 data tables
135- ~45% of workflows reference a utility workflow
136- ~5–10% of workflows reach the central data table (2–4 direct refs from
137  workflows inside Org Utilities — Customers Proxy plus a few service
138  workflows like Region Router or Feature Flag Resolver — and the rest via
139  `ExecuteWorkflow` into the proxy)
140- 20 of 29 non-utility projects use utility workflows
141- 0 direct cross-project DataTable references (architectural invariant)
142 
143## Re-running and cleanup
144 
145The clear step is greedy: any workflow / credential / data-table with the
146`[seed]` (or `seed_` for data tables) prefix is deleted, plus any team
147project owning a `[seed]`-prefixed entity, plus any orphan team project from
148an earlier run that doesn't match the current `PROJECT_NAMES` list and is
149empty and not named `My project`.
150 
151To remove all seeded data without reseeding:
152 
153```sh
154N8N_API_KEY=… CLEAR=only node bin/seedInstance/seedInstance.mjs
155```
156 
157 
158## Adding new behaviour
159 
160Adding a new shape of workflow usually means three places:
161 
1621. A constant or recipe at the top of the file (theme name, cred type, …).
1632. A change inside `workflowNodes()` if it needs a new node type.
1643. A phase change (or new phase) inside `main()` that calls `createWf()`.
165 
166`applyOrgUtilityRefs()` is the central hook for "every workflow should
167sometimes touch X". Phase 4 (`/* cross-project data-table proxies */`) is
168the template for "select a few entities, give each its own consumer
169fan-out".
170 
171## Known limitations
172 
173- Trenchcoat phase-3 (data-table consumer) workflows don't carry an internal
174  group label, so they show up as ungrouped within their trenchcoat project.
175  Minor visual artifact only.
176- The `[seed] X: [group] Y N` naming convention is what powers the
177  group-aware visualisation. Renaming a workflow externally severs the
178  link the analyser uses to group it.
179- No protection against running against a non-local instance. **Don't point
180  it at a shared/production n8n** — the clear step will delete everything
181  prefixed `[seed]` regardless of who created it.
182