Copilot instructions

1# Copilot Onboarding Instructions for DeerFlow
2 
3Use this file as the default operating guide for this repository. Follow it first, and only search the codebase when this file is incomplete or incorrect.
4 
5## 1) Repository Summary
6 
7DeerFlow is a full-stack "super agent harness".
8 
9- Backend: Python 3.12, LangGraph + FastAPI gateway, sandbox/tool system, memory, MCP integration.
10- Frontend: Next.js 16 + React 19 + TypeScript + pnpm.
11- Local dev entrypoint: root `Makefile` starts backend + frontend + nginx on `http://localhost:2026`.
12- Docker dev entrypoint: `make docker-*` (mode-aware provisioner startup from `config.yaml`).
13 
14Current repo footprint is medium-large (backend service, frontend app, docker stack, skills library, docs).
15 
16## 2) Runtime and Toolchain Requirements
17 
18Validated in this repo on macOS:
19 
20- Node.js `>=22` (validated with Node `23.11.0`)
21- pnpm (repo expects lockfile generated by pnpm 10; validated with pnpm `10.26.2` and `10.15.0`)
22- Python `>=3.12` (CI uses `3.12`)
23- `uv` (validated with `0.7.20`)
24- `nginx` (required for `make dev` unified local endpoint)
25 
26Always run from repo root unless a command explicitly says otherwise.
27 
28## 3) Build/Test/Lint/Run - Verified Command Sequences
29 
30These were executed and validated in this repository.
31 
32### A. Bootstrap and install
33 
341. Check prerequisites:
35 
36```bash
37make check
38```
39 
40Observed: passes when required tools are installed.
41 
422. Install dependencies (recommended order: backend then frontend, as implemented by `make install`):
43 
44```bash
45make install
46```
47 
48### B. Backend CI-equivalent validation
49 
50Run from `backend/`:
51 
52```bash
53make lint
54make test
55```
56 
57Validated results:
58 
59- `make lint`: pass (`ruff check .`)
60- `make test`: pass (`277 passed, 15 warnings in ~76.6s`)
61 
62CI parity:
63 
64- `.github/workflows/backend-unit-tests.yml` runs on pull requests.
65- CI executes `uv sync --group dev`, then `make lint`, then `make test` in `backend/`.
66 
67### C. Frontend validation
68 
69Run from `frontend/`.
70 
71Recommended reliable sequence:
72 
73```bash
74pnpm lint
75pnpm typecheck
76BETTER_AUTH_SECRET=local-dev-secret pnpm build
77```
78 
79Observed failure modes and workarounds:
80 
81- `pnpm build` fails without `BETTER_AUTH_SECRET` in production-mode env validation.
82- Workaround: set `BETTER_AUTH_SECRET` (best) or set `SKIP_ENV_VALIDATION=1`.
83- Even with `SKIP_ENV_VALIDATION=1`, Better Auth can still warn/error in logs about default secret; prefer setting a real non-default secret.
84- `pnpm check` currently fails (`next lint` invocation is incompatible here and resolves to an invalid directory). Do not rely on `pnpm check`; run `pnpm lint` and `pnpm typecheck` explicitly.
85 
86### D. Run locally (all services)
87 
88From root:
89 
90```bash
91make dev
92```
93 
94Behavior:
95 
96- Stops existing local services first.
97- Starts Gateway (`8001`, with the embedded LangGraph-compatible runtime), Frontend (`3000`), nginx (`2026`). There is no standalone LangGraph service.
98- Unified app endpoint: `http://localhost:2026`.
99- Logs: `logs/gateway.log`, `logs/frontend.log`, `logs/nginx.log`.
100 
101Stop services:
102 
103```bash
104make stop
105```
106 
107If tool sessions/timeouts interrupt `make dev`, run `make stop` again to ensure cleanup.
108 
109### E. Config bootstrap
110 
111From root:
112 
113```bash
114make config
115```
116 
117Important behavior:
118 
119- This intentionally aborts if `config.yaml` (or `config.yml`/`configure.yml`) already exists.
120- Use `make config` only for first-time setup in a clean clone.
121 
122## 4) Command Order That Minimizes Failures
123 
124Use this exact order for local code changes:
125 
1261. `make check`
1272. `make install` (if frontend fails with proxy errors, rerun frontend install with proxy vars unset)
1283. Backend checks: `cd backend && make lint && make test`
1294. Frontend checks: `cd frontend && pnpm lint && pnpm typecheck`
1305. Frontend build (if UI changes or release-sensitive changes): `BETTER_AUTH_SECRET=... pnpm build`
131 
132Always run backend lint/tests before opening PRs because that is what CI enforces.
133 
134## 5) Project Layout and Architecture (High-Value Paths)
135 
136Root-level orchestration and config:
137 
138- `Makefile` - main local/dev/docker command entrypoints
139- `config.example.yaml` - primary app config template
140- `config.yaml` - local active config (gitignored)
141- `docker/docker-compose-dev.yaml` - Docker dev topology
142- `.github/workflows/backend-unit-tests.yml` - PR validation workflow
143 
144Backend core:
145 
146- `backend/packages/harness/deerflow/agents/` - lead agent, middleware chain, memory
147- `backend/app/gateway/` - FastAPI gateway API
148- `backend/packages/harness/deerflow/sandbox/` - sandbox provider + tool wrappers
149- `backend/packages/harness/deerflow/subagents/` - subagent registry/execution
150- `backend/packages/harness/deerflow/mcp/` - MCP integration
151- `backend/langgraph.json` - graph entrypoint (`deerflow.agents:make_lead_agent`)
152- `backend/pyproject.toml` - Python deps and `requires-python`
153- `backend/ruff.toml` - lint/format policy
154- `backend/tests/` - backend unit and integration-like tests
155 
156Frontend core:
157 
158- `frontend/src/app/` - Next.js routes/pages
159- `frontend/src/components/` - UI components
160- `frontend/src/core/` - app logic (threads, tools, API, models)
161- `frontend/src/env.js` - env schema/validation (critical for build behavior)
162- `frontend/package.json` - scripts/deps
163- `frontend/eslint.config.js` - lint rules
164- `frontend/tsconfig.json` - TS config
165 
166Skills and assets:
167 
168- `skills/public/` - built-in skill packs loaded by agent runtime
169 
170## 6) Pre-Checkin / Validation Expectations
171 
172Before submitting changes, run at minimum:
173 
174- Backend: `cd backend && make lint && make test`
175- Frontend (if touched): `cd frontend && pnpm lint && pnpm typecheck`
176- Frontend build when changing env/auth/routing/build-sensitive files: `BETTER_AUTH_SECRET=... pnpm build`
177 
178If touching orchestration/config (`Makefile`, `docker/*`, `config*.yaml`), also run `make dev` and verify the four services start.
179 
180## 7) Non-Obvious Dependencies and Gotchas
181 
182- Proxy env vars can silently break frontend network operations (`pnpm install`/registry access).
183- `BETTER_AUTH_SECRET` is effectively required for reliable frontend production build validation.
184- Next.js may warn about multiple lockfiles and workspace root inference; this is currently a warning, not a build blocker.
185- `make config` is non-idempotent by design when config already exists.
186- `make dev` includes process cleanup and can emit shutdown logs/noise if interrupted; this is expected.
187 
188## 8) Root Inventory (quick reference)
189 
190Important root entries:
191 
192- `.github/`
193- `backend/`
194- `frontend/`
195- `docker/`
196- `skills/`
197- `scripts/`
198- `docs/`
199- `README.md`
200- `CONTRIBUTING.md`
201- `Makefile`
202- `config.example.yaml`
203- `extensions_config.example.json`
204 
205## 9) Instruction Priority
206 
207Trust this onboarding guide first.
208 
209Only do broad repo searches (`grep/find/code search`) when:
210 
211- you need file-level implementation details not listed here,
212- a command here fails and you need updated replacement behavior,
213- or CI/workflow definitions have changed since this file was written.
214