AGENTS.md

1# AGENTS.md
2 
3## Commands
4 
5```bash
6# Run tests locally
7pnpm --filter=n8n-playwright test:local <file-path>
8pnpm --filter=n8n-playwright test:local tests/e2e/credentials/crud.spec.ts
9 
10# Run with container capabilities (requires pnpm build:docker first)
11pnpm --filter=n8n-playwright test:container:sqlite --grep @capability:email
12 
13# Lint and typecheck
14pnpm --filter=n8n-playwright lint
15pnpm --filter=n8n-playwright typecheck
16```
17 
18Always trim output: `--reporter=list 2>&1 | tail -50`
19 
20## Test Maintenance (Janitor)
21 
22Static analysis for Playwright test architecture. Catches problems before they spread.
23 
24> **CRITICAL: Always use TCR for code changes.**
25> When janitor identifies violations and you fix them, use `pnpm janitor tcr --execute` to safely commit. Never manually commit janitor-related fixes - TCR ensures tests pass before the commit lands.
26 
27### Golden Rules
28 
291. **Analysis only?** Run `pnpm janitor` (no TCR needed)
302. **Making code changes?** Use TCR: `pnpm janitor tcr --execute -m="chore: ..."`
313. **Never** manually `git commit` janitor-related fixes - always go through TCR
324. **Never** modify `.janitor-baseline.json` via TCR - baseline updates must be done manually
33 
34### When to Use
35 
36| User Says | Intent | Approach |
37|-----------|--------|----------|
38| "Clean up the test codebase" | Incremental cleanup | Create baseline first, then use `--max-diff-lines=500` for small PRs. |
39| "Start tracking violations" | Enable incremental cleanup | Run `janitor baseline` to snapshot current state, commit `.janitor-baseline.json`. |
40| "Add a test for X" | New test following patterns | After writing, run janitor to verify architecture compliance. |
41| "Fix architecture drift" | Enforce layered architecture | Run `selector-purity` and `no-page-in-flow` rules. |
42| "Find dead code" | Remove unused methods | Run `dead-code` rule with `--fix --write` for auto-removal. |
43| "Find copy-paste code" | Detect duplicates | Run `duplicate-logic` rule to find structural duplicates. |
44| "This file is messy" | Targeted cleanup | Analyze specific file, fix issues, TCR to safely commit. |
45| "Refactor this page object" | Safe refactoring | Use TCR - changes commit if tests pass, revert if they fail. |
46| "What tests would break?" | Impact analysis | Run `impact` command before changing shared code. |
47| "Prepare for PR" | Pre-commit check | Run janitor on changed files to catch violations early. |
48 
49### Architecture Rules
50 
51The janitor enforces a layered architecture:
52 
53```
54Tests → Flows/Composables → Page Objects → Components → Playwright API
55```
56 
57| Rule | What It Catches |
58|------|-----------------|
59| `selector-purity` | Raw locators in tests/flows: `page.getByTestId()`, `someLocator.locator()` |
60| `no-page-in-flow` | Flows accessing `page` directly (should use page objects) |
61| `boundary-protection` | Pages importing other pages (creates coupling) |
62| `scope-lockdown` | Unscoped locators that escape their container |
63| `dead-code` | Unused public methods in page objects |
64| `deduplication` | Same selector defined in multiple files |
65| `duplicate-logic` | Copy-pasted code across tests/pages (AST fingerprinting) |
66| `no-raw-editor-navigation` | Raw `page.goto()` to a `/workflow/` editor route in tests (use `n8n.start.*` so the canvas loader is awaited) |
67| `valid-owner-annotation` | A spec with no team owner, or an owner not in the canonical list (`CANONICAL_OWNERS` in the rule, mirroring Notion "Ownership v2") |
68 
69### Commands
70 
71```bash
72# Analyze entire codebase
73pnpm janitor
74 
75# Analyze specific file
76pnpm janitor --file=pages/CanvasPage.ts --verbose
77 
78# Run specific rule
79pnpm janitor --rule=dead-code
80 
81# Auto-fix (dead-code only)
82pnpm janitor:fix --rule=dead-code
83 
84# List all rules (short)
85pnpm janitor --list
86 
87# Show detailed rule info (for AI agents)
88pnpm janitor rules --json
89 
90# Discover test specs (for orchestration)
91pnpm janitor discover
92 
93# Distribute specs across shards
94pnpm janitor orchestrate --shards=14
95```
96 
97### Baseline (Incremental Cleanup)
98 
99For codebases with existing violations, create a baseline to enable incremental cleanup:
100 
101```bash
102# Create baseline - snapshots current violations
103pnpm janitor baseline
104 
105# Commit the baseline
106git add .janitor-baseline.json && git commit -m "chore: add janitor baseline"
107```
108 
109Once baseline exists, janitor and TCR **only fail on NEW violations**. Pre-existing violations are tracked but don't block work.
110 
111> **Safeguard:** TCR blocks commits that modify `.janitor-baseline.json`. This prevents accidentally "fixing" violations by updating the baseline instead of the actual code. Baseline updates must be done manually after fixing violations.
112 
113```bash
114# Update baseline after fixing violations (manual commit required)
115pnpm janitor baseline
116git add .janitor-baseline.json && git commit -m "chore: update baseline after cleanup"
117```
118 
119### Incremental Cleanup Strategy
120 
121For large cleanups, keep diffs small and reviewable:
122 
123```bash
124# Show ALL violations (ignoring baseline) for cleanup work
125pnpm janitor --ignore-baseline --json
126 
127# Find easiest files to fix (lowest violation count)
128pnpm janitor --ignore-baseline --json 2>/dev/null | jq '.fileReports | sort_by(.violationCount) | .[:5]'
129 
130# TCR with max diff size (skip if changes are too large)
131pnpm janitor tcr --max-diff-lines=500 --execute -m="chore: cleanup"
132```
133 
134**AI Cleanup Workflow:**
1351. Use `--ignore-baseline` to see all violations (not just new ones)
1362. Pick small fixes from the list
1373. Fix violations, then TCR to safely commit
1384. After fixing, run `pnpm janitor baseline` to update the baseline
139 
140### TCR Workflow (Test && Commit || Revert)
141 
142Safe refactoring loop: make changes, run affected tests, auto-commit or auto-revert.
143 
144```bash
145# Dry run - see what would happen
146pnpm janitor tcr --verbose
147 
148# Execute - actually commit/revert
149pnpm janitor tcr --execute -m="chore: remove dead code"
150 
151# With guardrails - skip if diff too large
152pnpm janitor tcr --execute --max-diff-lines=500 -m="chore: cleanup"
153```
154 
155### After Writing New Tests
156 
157Always run janitor after adding or modifying tests to catch architecture violations early:
158 
159```bash
160pnpm janitor --file=tests/my-new-test.spec.ts --verbose
161```
162 
163See `packages/testing/janitor/README.md` for full documentation.
164 
165## Entry Points
166 
167All tests should start with `n8n.start.*` methods. See `composables/TestEntryComposer.ts`.
168 
169| Method | Use Case |
170|--------|----------|
171| `fromHome()` | Start from home page |
172| `fromBlankCanvas()` | New workflow from scratch |
173| `fromNewProjectBlankCanvas()` | Project-scoped workflow (returns projectId) |
174| `fromNewProject()` | Project-scoped test, no canvas (returns projectId) |
175| `fromImportedWorkflow(file)` | Test pre-built workflow JSON |
176| `withUser(user)` | Isolated browser context per user |
177| `withProjectFeatures()` | Enable sharing/folders/permissions |
178 
179## Test Isolation
180 
181Tests run in parallel. Design tests to be fully isolated so they don't interfere with each other.
182 
183### Unique Identifiers
184 
185Use `nanoid` for unique test data:
186 
187```typescript
188const credentialName = `Test Credential ${nanoid()}`;
189const workflow = await api.workflows.createWorkflow({
190  name: `Test Workflow ${nanoid()}`,
191});
192```
193 
194### Dynamic User Creation
195 
196Create users dynamically via the public API:
197 
198```typescript
199const member = await api.publicApi.createUser({
200  email: `member-${nanoid()}@test.com`,
201  firstName: 'Test',
202  lastName: 'Member',
203});
204```
205 
206### Isolated Browser Contexts
207 
208For UI tests requiring multiple users, create isolated browser contexts:
209 
210```typescript
211// 1. Create users via public API
212const member1 = await api.publicApi.createUser({ role: 'global:member' });
213const member2 = await api.publicApi.createUser({ role: 'global:member' });
214 
215// 2. Get isolated browser contexts
216const member1Page = await n8n.start.withUser(member1);
217const member2Page = await n8n.start.withUser(member2);
218 
219// 3. Each operates independently (no session bleeding)
220await member1Page.navigate.toWorkflows();
221await member2Page.navigate.toCredentials();
222```
223 
224**Reference:** `tests/e2e/building-blocks/user-service.spec.ts`
225 
226| Pattern | Why | Use Instead |
227|---------|-----|-------------|
228| `test.describe.serial` | Creates test dependencies | Parallel tests with isolated setup |
229| Fresh DB per file | Tests need isolated container | `test.use({ capability: { env: { TEST_ISOLATION: 'name' } } })` |
230| Fresh DB per test | Tests modify shared state | `@db:reset` tag on describe (container-only, combined with `test.use()`) |
231| `n8n.api.signin()` | Session bleeding | `n8n.start.withUser()` |
232| `Date.now()` for IDs | Race conditions | `nanoid()` |
233| `waitForTimeout()` | Flaky | `waitForResponse()`, `toBeVisible()` |
234| `.toHaveCount(N)` | Brittle | Named element assertions |
235| Raw `page.goto()` | Bypasses setup | `n8n.navigate.*` methods |
236 
237## Code Style
238 
239- Use specialized locators: `page.getByRole('button')` over `page.locator('[role=button]')`
240- Use `nanoid()` for unique identifiers (parallel-safe)
241- API setup over UI setup when possible (faster, more reliable)
242 
243## Architecture
244 
245```
246Tests (*.spec.ts)
247    ↓ uses
248Composables (*Composer.ts) - Multi-step business workflows
249    ↓ orchestrates
250Page Objects (*Page.ts) - UI interactions
251    ↓ extends
252BasePage - Common utilities
253```
254 
255See `CONTRIBUTING.md` for detailed patterns and conventions.
256 
257## Debugging
258 
259See [README.md#debugging](./README.md#debugging) for detailed instructions on:
260- **Keepalive mode** - Keep containers running after tests with `N8N_CONTAINERS_KEEPALIVE=true`
261- **Victoria exports** - Logs/metrics automatically attached on failure, importable locally via `scripts/import-victoria-data.mjs`
262 
263## Test Migration & Refactoring
264 
265**Test Name = Contract**
266- Name declares intent, assertion proves it, everything else is flow
267- Bad: `should open W1 as U2` (describes action)
268- Good: `should allow sharee to edit shared workflow` (declares rule)
269 
270**Coverage Parity Check**
2711. Read old test name → what was the intent?
2722. Find the explicit assertion that proved it
2733. Verify new test has equivalent proof
2744. No proof found? Document as intentional drop or gap
275 
276**Legacy Tests (unauditable names/assertions)**
277- Prioritize clarity over parity - can't audit what you can't read
278- Document your best interpretation of intent
279- Accept short-term risk, fix regressions forward
280 
281See [Quality Corner: Test Migration Guide](https://www.notion.so/n8n/Best-Practices-Test-Migration-Refactoring) for full rationale and examples.
282 
283## Reference Files
284 
285| Purpose | File |
286|---------|------|
287| Multi-user testing | `tests/e2e/building-blocks/user-service.spec.ts` |
288| Entry points | `composables/TestEntryComposer.ts` |
289| Page object example | `pages/CanvasPage.ts` |
290| Composable example | `composables/WorkflowComposer.ts` |
291| API helpers | `services/api-helper.ts` |
292| Capabilities | `fixtures/capabilities.ts` |
293 
294```typescript
295const member = await api.publicApi.createUser({...});
296const memberN8n = await n8n.start.withUser(member);
297 
298await memberN8n.navigate.toWorkflows();
299await expect(memberN8n.workflows.cards.getWorkflow(workflowName)).toBeVisible();
300```
301### Isolated API Contexts
302 
303For API-only operations as another user, create isolated API contexts (no browser needed):
304 
305```typescript
306const member = await api.publicApi.createUser({...});
307const memberApi = await api.createApiForUser(member);
308 
309const memberProject = await memberApi.projects.getMyPersonalProject();
310await memberApi.credentials.createCredential({...});
311```
312 
313### Identity-Based Assertions
314 
315Assert by identity (name) rather than count for parallel-safe tests:
316 
317```typescript
318await expect(credentialDropdown.getByText(testCredName)).toBeVisible();
319await expect(credentialDropdown.getByText(devCredName)).toBeHidden();
320```
321 
322## Worker Isolation (Fresh Database)
323 
324Use `test.use()` at file top-level with unique capability config:
325 
326```typescript
327// my-isolated-tests.spec.ts
328import { test, expect } from '../fixtures/base';
329 
330// Must be top-level, not inside describe block
331test.use({ capability: { env: { TEST_ISOLATION: 'my-isolated-tests' } } });
332 
333test('test with clean state', async ({ n8n }) => {
334  // Fresh container with reset database
335});
336```
337 
338For per-test database reset (when tests modify shared state like MFA), add `@db:reset` to the describe. **Note:** `@db:reset` is container-only - these tests won't run locally.
339 
340```typescript
341test.use({ capability: { env: { TEST_ISOLATION: 'my-stateful-tests' } } });
342 
343test.describe('My stateful tests @db:reset', () => {
344  // Each test gets a fresh database reset (container-only)
345});
346```
347 
348## Data Setup
349 
350Use API helpers for fast, reliable test data setup. Reserve UI interactions for testing UI behavior:
351 
352```typescript
353// API for data setup
354const credential = await api.credentials.createCredential({
355  name: `Test Credential ${nanoid()}`,
356  type: 'notionApi',
357  data: { apiKey: 'test' },
358});
359 
360const workflow = await api.workflows.createWorkflow({
361  name: `Test Workflow ${nanoid()}`,
362  nodes: [...],
363});
364 
365// UI for verification
366await n8n.navigate.toCredentials();
367await expect(n8n.credentials.cards.getCredential(credential.name)).toBeVisible();
368```
369 
370## Feature Enablement
371 
372The `n8n` fixture automatically enables project features. For API-only tests (no `n8n` fixture), enable features explicitly:
373 
374```typescript
375test('API-only test', async ({ api }) => {
376  await api.enableProjectFeatures();
377  // ...
378});
379```
380 
381### Feature Flag Overrides
382 
383To test features behind feature flags (experiments), use `TestRequirements` with storage overrides:
384 
385```typescript
386import type { TestRequirements } from '../config/TestRequirements';
387 
388const requirements: TestRequirements = {
389  storage: {
390    N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ 'your_experiment': true }),
391  },
392};
393 
394test.use({ requirements });
395 
396test('test with feature flag enabled', async ({ n8n }) => {
397  // Feature flag is now active for this test
398});
399```
400 
401**Common patterns:**
402 
403```typescript
404// Single experiment
405{ storage: { N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ '025_new_canvas': true }) } }
406 
407// Multiple experiments
408{ storage: { N8N_EXPERIMENT_OVERRIDES: JSON.stringify({
409  '025_new_canvas': true,
410  '026_another_feature': 'variant_a'
411}) } }
412 
413// Combined with other requirements
414const requirements: TestRequirements = {
415  storage: {
416    N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ 'your_experiment': true }),
417  },
418  capability: {
419    env: { TEST_ISOLATION: 'my-test-suite' },
420  },
421};
422```
423 
424**Reference:** `config/TestRequirements.ts` for full interface definition.
425 
426## Shard Rebalancing
427 
428When refactoring, adding, or moving significant numbers of tests, consider rebalancing test shards to maintain even CI distribution. See `docs/ORCHESTRATION.md` for details.
429