AGENTS.md

1# @n8n/engine — structure & modularity intent
2 
3## The blueprint we're following
4 
5We structure this package after the **Durable Scheduler modularity blueprint**
6([Notion](https://app.notion.com/p/n8n/The-Durable-Scheduler-a-modularity-blueprint-39f5b6e0c94f8193a5fecca8306c4424),
7worked example: `packages/@n8n/scheduler`, enforced by its
8`src/__tests__/dependency-purity.test.ts`). The core idea: a **pure core that
9decides, with every effect handed in as an injected interface**. Dependency
10arrows point one way — consumers depend on the engine; the engine core reaches
11for nothing.
12 
13## Reality now: loose, but seam-aware
14 
15We are deliberately **loose about composition at this stage**. This one package
16currently holds the engine core *and* its serving/persistence infrastructure.
17We expect things to shuffle, so we're not paying the full ports-and-adapters
18tax yet. What we **do** commit to is keeping the internal boundaries clean, so
19serving infra and persistence can later be lifted into their own packages (e.g.
20a deployable engine worker) without touching core logic.
21 
22## Layers (today, all in this package)
23 
24- **Core (decides)** — `graph/`, `execution/`, `admittance/`. Pure
25  orchestration/policy. Must not open connections, bind an HTTP server, or read
26  the environment. Everything external arrives via constructor args / a deps bag
27  (the `createScheduler(deps)` pattern).
28- **Core interfaces** — interfaces the core depends on, each defined in its own
29  core module beside a default/reference use: `AdmittanceService` (`admittance/`),
30  `WorkQueue` (`queue/`), `ExecutionStore` (`execution/`). Adapters implement
31  them; the core never imports the interface from an adapter. Handed in at
32  construction.
33- **Adapters (do)** — effectful implementations: `database/` (TypeORM entities,
34  migrations, the Postgres `DataSource`, and `TypeOrmExecutionStore`), `queue/`
35  (in-memory default). The Postgres/ORM coupling lives *here only*.
36- **Serving infra** — `server/` (express), `serve.ts` (standalone entrypoint),
37  Dockerfile. First candidate to be extracted later.
38- **Composition roots** — `serve.ts` (standalone) and, in integrated mode,
39  `packages/cli`. These construct the concrete adapters (the `DataSource`, etc.)
40  and hand them in. **Construction lives here, not in the core.**
41 
42## Rules that keep the seams extractable
43 
44- Core modules (`graph`, `execution`, `admittance`) don't import `express`,
45  `pg`, or `@n8n/typeorm`, don't construct a `DataSource`, and never import from
46  `database/`. Persistence, queue, and HTTP are injected. The dependency arrow
47  is one-way: `database/` imports the interface + domain types from the core, not
48  the reverse.
49- `@n8n/typeorm` / `pg` stay confined to `database/`.
50- `@n8n/config` + `@n8n/di` are used only at the serving/composition layer (for
51  `EngineConfig`), never in core logic. (The blueprint flags `@n8n/config` as
52  debatable precisely because it pulls the DI runtime in — keep it out of core.)
53- We go one step stricter than the scheduler's allowlist: **no `n8n-workflow`
54  dependency at all, not even type-only** (per the Engine 2.0 design — the core
55  must stay free of v1 concepts). Shared JSON types are redefined locally in
56  `common/`.
57- Arrows point inward: `cli`/`serve` depend on the engine; the engine never
58  imports `cli`.
59- When serving infra is extracted, add a `dependency-purity.test.ts` (as
60  `@n8n/scheduler` does) to enforce the allowlist. Until then, this doc is the
61  intent.
62 
63## Crash recovery: reconciliation, not transactions
64 
65Handlers advance an execution through several separate writes — claim the
66execution, insert step rows, publish the next message — and we deliberately do
67**not** make that sequence atomic. Crashing partway through can leave partial
68state, such as an execution stuck `running` with no queued step.
69 
70The intended answer is a reconciliation layer that detects crashed or stalled
71executions and drives recovery (CAT-2938), not transactions spanning stores and
72queues. So when you find a partial-write window: make the resulting state
73legible to reconciliation, and don't reach for a cross-store transaction. It's a
74recurring review question — this is the standing answer.
75 
76## Known deviations — the seams to clean up on decomposition
77 
78- `DataSource` construction currently lives in `database/`; it is really a
79  composition-root concern.
80