| Dimension | Shared | Only in A | Only in B | Overlap |
|---|---|---|---|---|
| Sections | 0 | 26 | 49 | 0% |
| Commands | 2 | 19 | 16 | 5% |
| Section tags | 7 | 7 | 4 | 39% |
What each file covers
Sections
0 shared · 26 only in A · 49 only in B- − GitHub Copilot Instructions for AutoGPT
- − Repository Overview
- − Build and Validation Instructions
- − Essential Setup Commands
- − Runtime Requirements
- − Development Commands
- − Testing Strategy
- − Critical Validation Steps
- − Project Layout & Architecture
- − Core Architecture
- − Security & Middleware
- − Development Workflow
- − Key Source Files
- − Agent Block System
- − Database & ORM
- − Environment Configuration
- − Configuration Files Priority Order
- − Docker Environment Setup
- − Advanced Development Patterns
- − Adding New Blocks
- − API Development
- − Frontend Development
- − Security Guidelines
- − CI/CD Alignment
- − Collaboration with Other AI Assistants
- − Trust These Instructions
- + CLAUDE.md
- + Project Overview
- + Repository Structure
- + Common Commands
- + Setup & Install
- + Install everything from classic/ directory
- + Running Agents
- + Run forge agent
- + Run original autogpt server
- + Run autogpt CLI
- + Benchmarking
- + Run benchmarks
- + Run specific strategies and models
- + Run a single test
- + List available commands
- + Testing
- + Linting & Formatting
- + Format everything (recommended to run together)
- + Check formatting (CI-style, no changes)
- + Lint
- + Type check
- + Architecture
- + Forge (Core Framework)
- + Original AutoGPT
- + Direct Benchmark
- + Package Structure
- + Code Style
- + Testing Patterns
- + Environment Setup
- + Edit .env with your OPENAI_API_KEY, etc.
- + Workspaces
- + Workspace Structure
- + Key Concepts
- + Specifying a Workspace
- + Default: uses current directory
- + Or specify explicitly via CLI (if supported)
- + Settings Location
- + 1. Environment Variables (Global)
- + Required
- + Optional LLM settings
- + Optional search providers (for web search component)
- + Optional infrastructure
- + 2. Workspace Settings (`{workspace}/.autogpt/autogpt.yaml`)
- + 3. Agent Settings (`{workspace}/.autogpt/agents/{id}/permissions.yaml`)
- + Permissions
- + Permission Check Order
- + Pattern Syntax
- + Interactive Approval Scopes
- + Default Security
Commands
2 shared · 19 only in A · 16 only in B- − git clone <repo> && cd AutoGPT
- − poetry run prisma migrate dev
- − poetry run prisma generate
- − pnpm install
- − poetry run serve
- − poetry run test
- − poetry run format
- − poetry run lint
- − pnpm dev
- − pnpm build
- − pnpm test
- − pnpm test-ui
- − pnpm format
- − pnpm storybook
- − poetry run pytest backend/blocks/test/test_block.py -xvs
- − poetry run pytest 'backend/blocks/test/test_block.py::test_available_blocks[BlockName]' -xvs
- − git diff
- − docker-compose.yml
- − pnpm generate:api
- + poetry run python -m forge
- + poetry run serve --debug
- + poetry run autogpt
- + poetry run direct-benchmark run \
- + poetry run direct-benchmark run --tests ReadFile
- + poetry run direct-benchmark --help
- + poetry run pytest
- + poetry run pytest forge/tests/
- + poetry run pytest original_autogpt/tests/
- + poetry run pytest -k test_name
- + poetry run pytest --cov
- + poetry run black . && poetry run isort .
- + poetry run black --check . && poetry run isort --check-only .
- + poetry run flake8
- + poetry run pyright
- + poetry run autogpt --workspace /path/to/workspace
- poetry install
- poetry run pytest path/to/test.py
Section tags
7 shared · 7 only in A · 4 only in B- − build
- − testing-strategy
- − git-pr
- − database
- − api
- − deployment
- − do-not
- + types
- + ui
- + performance
- + monorepo
- setup
- test
- lint-format
- code-style
- architecture
- security
- agent-behaviour
Line diff
Significant-Gravitas/AutoGPT · .github/copilot-instructions.md
@@ −1 @@
1# GitHub Copilot Instructions for AutoGPT
2
3This file provides comprehensive onboarding information for GitHub Copilot coding agent to work efficiently with the AutoGPT repository.
4
5## Repository Overview
6
7**AutoGPT** is a powerful platform for creating, deploying, and managing continuous AI agents that automate complex workflows. This is a large monorepo (~150MB) containing multiple components:
8
9- **AutoGPT Platform** (`autogpt_platform/`) - Main focus: Modern AI agent platform (Polyform Shield License)
10- **Classic AutoGPT** (`classic/`) - Legacy agent system (MIT License)
11- **Documentation** (`docs/`) - MkDocs-based documentation site
12- **Infrastructure** - Docker configurations, CI/CD, and development tools
13
14**Primary Languages & Frameworks:**
15
16- **Backend**: Python 3.10-3.13, FastAPI, Prisma ORM, PostgreSQL, RabbitMQ
17- **Frontend**: TypeScript, Next.js 15, React, Tailwind CSS, Radix UI
18- **Development**: Docker, Poetry, pnpm, Playwright, Storybook
19
20## Build and Validation Instructions
21
22### Essential Setup Commands
23
24**Always run these commands in the correct directory and in this order:**
25
261. **Initial Setup** (required once):
27
28 ```bash
29 # Clone and enter repository
30 git clone <repo> && cd AutoGPT
31
32 # Start all services (database, redis, rabbitmq, clamav)
33 cd autogpt_platform && docker compose --profile local up deps --build --detach
34 ```
35
362. **Backend Setup** (always run before backend development):
37
38 ```bash
39 cd autogpt_platform/backend
40 poetry install # Install dependencies
41 poetry run prisma migrate dev # Run database migrations
42 poetry run prisma generate # Generate Prisma client
43 ```
44
453. **Frontend Setup** (always run before frontend development):
46 ```bash
47 cd autogpt_platform/frontend
48 pnpm install # Install dependencies
49 ```
50
51### Runtime Requirements
52
53**Critical:** Always ensure Docker services are running before starting development:
54
55```bash
56cd autogpt_platform && docker compose --profile local up deps --build --detach
57```
58
59**Python Version:** Use Python 3.11 (required; managed by Poetry via pyproject.toml)
60**Node.js Version:** Use Node.js 21+ with pnpm package manager
61
62### Development Commands
63
64**Backend Development:**
65
66```bash
67cd autogpt_platform/backend
68poetry run serve # Start development server (port 8000)
69poetry run test # Run all tests (requires ~5 minutes)
70poetry run pytest path/to/test.py # Run specific test
71poetry run format # Format code (Black + isort) - always run first
72poetry run lint # Lint code (ruff) - run after format
73```
74
75**Frontend Development:**
76
77```bash
78cd autogpt_platform/frontend
79pnpm dev # Start development server (port 3000) - use for active development
80pnpm build # Build for production (only needed for E2E tests or deployment)
81pnpm test # Run Playwright E2E tests (requires build first)
82pnpm test-ui # Run tests with UI
83pnpm format # Format and lint code
84pnpm storybook # Start component development server
85```
86
87### Testing Strategy
88
89**Backend Tests:**
90
91- **Block Tests**: `poetry run pytest backend/blocks/test/test_block.py -xvs` (validates all blocks)
92- **Specific Block**: `poetry run pytest 'backend/blocks/test/test_block.py::test_available_blocks[BlockName]' -xvs`
93- **Snapshot Tests**: Use `--snapshot-update` when output changes, always review with `git diff`
94
95**Frontend Tests:**
96
97- **E2E Tests**: Always run `pnpm dev` before `pnpm test` (Playwright requires running instance)
98- **Component Tests**: Use Storybook for isolated component development
99
100### Critical Validation Steps
101
102**Before committing changes:**
103
1041. Run `poetry run format` (backend) and `pnpm format` (frontend)
1052. Ensure all tests pass in modified areas
1063. Verify Docker services are still running
1074. Check that database migrations apply cleanly
108
109**Common Issues & Workarounds:**
110
111- **Prisma issues**: Run `poetry run prisma generate` after schema changes
112- **Permission errors**: Ensure Docker has proper permissions
113- **Port conflicts**: Check the `docker-compose.yml` file for the current list of exposed ports. You can list all mapped ports with:
114- **Test timeouts**: Backend tests can take 5+ minutes, use `-x` flag to stop on first failure
115
116## Project Layout & Architecture
117
118### Core Architecture
119
120**AutoGPT Platform** (`autogpt_platform/`):
121
122- `backend/` - FastAPI server with async support
123 - `backend/backend/` - Core API logic
124 - `backend/blocks/` - Agent execution blocks
125 - `backend/data/` - Database models and schemas
126 - `schema.prisma` - Database schema definition
127- `frontend/` - Next.js application
128 - `src/app/` - App Router pages and layouts
129 - `src/components/` - Reusable React components
130 - `src/lib/` - Utilities and configurations
131- `autogpt_libs/` - Shared Python utilities
132- `docker-compose.yml` - Development stack orchestration
133
134**Key Configuration Files:**
135
136- `pyproject.toml` - Python dependencies and tooling
137- `package.json` - Node.js dependencies and scripts
138- `schema.prisma` - Database schema and migrations
139- `next.config.mjs` - Next.js configuration
140- `tailwind.config.ts` - Styling configuration
141
142### Security & Middleware
143
144**Cache Protection**: Backend includes middleware preventing sensitive data caching in browsers/proxies
145**Authentication**: JWT-based with Supabase integration
146**User ID Validation**: All data access requires user ID checks - verify this for any `data/*.py` changes
147
148### Development Workflow
149
150**GitHub Actions**: Multiple CI/CD workflows in `.github/workflows/`
151
152- `platform-backend-ci.yml` - Backend testing and validation
153- `platform-frontend-ci.yml` - Frontend testing and validation
154- `platform-fullstack-ci.yml` - End-to-end integration tests
155
156**Pre-commit Hooks**: Run linting and formatting checks
157**Conventional Commits**: Use format `type(scope): description` (e.g., `feat(backend): add API`)
158
159### Key Source Files
160
161**Backend Entry Points:**
162
163- `backend/backend/api/rest_api.py` - FastAPI application setup
164- `backend/backend/data/` - Database models and user management
165- `backend/blocks/` - Agent execution blocks and logic
166
167**Frontend Entry Points:**
168
169- `frontend/src/app/layout.tsx` - Root application layout
170- `frontend/src/app/page.tsx` - Home page
171- `frontend/src/lib/supabase/` - Authentication and database client
172
173**Protected Routes**: Update `frontend/lib/supabase/middleware.ts` when adding protected routes
174
175### Agent Block System
176
177Agents are built using a visual block-based system where each block performs a single action. Blocks are defined in `backend/blocks/` and must include:
178
179- Block definition with input/output schemas
180- Execution logic with proper error handling
181- Tests validating functionality
182
183### Database & ORM
184
185**Prisma ORM** with PostgreSQL backend including pgvector for embeddings:
186
187- Schema in `schema.prisma`
188- Migrations in `backend/migrations/`
189- Always run `prisma migrate dev` and `prisma generate` after schema changes
190
191## Environment Configuration
192
193### Configuration Files Priority Order
194
1951. **Backend**: `/backend/.env.default` → `/backend/.env` (user overrides)
1962. **Frontend**: `/frontend/.env.default` → `/frontend/.env` (user overrides)
1973. **Platform**: `/.env.default` (Supabase/shared) → `/.env` (user overrides)
1984. Docker Compose `environment:` sections override file-based config
1995. Shell environment variables have highest precedence
200
201### Docker Environment Setup
202
203- All services use hardcoded defaults (no `${VARIABLE}` substitutions)
204- The `env_file` directive loads variables INTO containers at runtime
205- Backend/Frontend services use YAML anchors for consistent configuration
206- Copy `.env.default` files to `.env` for local development customization
207
208## Advanced Development Patterns
209
210### Adding New Blocks
211
2121. Create file in `/backend/backend/blocks/`
2132. Inherit from `Block` base class with input/output schemas
2143. Implement `run` method with proper error handling
2154. Generate block UUID using `uuid.uuid4()`
2165. Register in block registry
2176. Write tests alongside block implementation
2187. Consider how inputs/outputs connect with other blocks in graph editor
219
220### API Development
221
2221. Update routes in `/backend/backend/api/features/`
2232. Add/update Pydantic models in same directory
2243. Write tests alongside route files
2254. For `data/*.py` changes, validate user ID checks
2265. Run `poetry run test` to verify changes
227
228### Frontend Development
229
230**📖 Complete Frontend Guide**: See `autogpt_platform/frontend/CONTRIBUTING.md` and `autogpt_platform/frontend/.cursorrules` for comprehensive patterns and conventions.
231
232**Quick Reference:**
233
234**Component Structure:**
235
236- Separate render logic from data/behavior
237- Structure: `ComponentName/ComponentName.tsx` + `useComponentName.ts` + `helpers.ts`
238- Exception: Small components (3-4 lines of logic) can be inline
239- Render-only components can be direct files without folders
240
241**Data Fetching:**
242
243- Use generated API hooks from `@/app/api/__generated__/endpoints/`
244- Generated via Orval from backend OpenAPI spec
245- Pattern: `use{Method}{Version}{OperationName}`
246- Example: `useGetV2ListLibraryAgents`
247- Regenerate with: `pnpm generate:api`
248- **Never** use deprecated `BackendAPI` or `src/lib/autogpt-server-api/*`
249
250**Code Conventions:**
251
252- Use function declarations for components and handlers (not arrow functions)
253- Only arrow functions for small inline lambdas (map, filter, etc.)
254- Components: `PascalCase`, Hooks: `camelCase` with `use` prefix
255- No barrel files or `index.ts` re-exports
256- Minimal comments (code should be self-documenting)
257
258**Styling:**
259
260- Use Tailwind CSS utilities only
261- Use design system components from `src/components/` (atoms, molecules, organisms)
262- Never use `src/components/__legacy__/*`
263- Only use Phosphor Icons (`@phosphor-icons/react`)
264- Prefer design tokens over hardcoded values
265
266**Error Handling:**
267
268- Render errors: Use `<ErrorCard />` component
269- Mutation errors: Display with toast notifications
270- Manual exceptions: Use `Sentry.captureException()`
271- Global error boundaries already configured
272
273**Testing:**
274
275- Add/update Storybook stories for UI components (`pnpm storybook`)
276- Run Playwright E2E tests with `pnpm test`
277- Verify in Chromatic after PR
278
279**Architecture:**
280
281- Default to client components ("use client")
282- Server components only for SEO or extreme TTFB needs
283- Use React Query for server state (via generated hooks)
284- Co-locate UI state in components/hooks
285
286### Security Guidelines
287
288**Cache Protection Middleware** (`/backend/backend/api/middleware/security.py`):
289
290- Default: Disables caching for ALL endpoints with `Cache-Control: no-store, no-cache, must-revalidate, private`
291- Uses allow list approach for cacheable paths (static assets, health checks, public pages)
292- Prevents sensitive data caching in browsers/proxies
293- Add new cacheable endpoints to `CACHEABLE_PATHS`
294
295### CI/CD Alignment
296
297The repository has comprehensive CI workflows that test:
298
299- **Backend**: Python 3.11-3.13, services (Redis/RabbitMQ/ClamAV), Prisma migrations, Poetry lock validation
300- **Frontend**: Node.js 21, pnpm, Playwright with Docker Compose stack, API schema validation
301- **Integration**: Full-stack type checking and E2E testing
302
303Match these patterns when developing locally - the copilot setup environment mirrors these CI configurations.
304
305## Collaboration with Other AI Assistants
306
307This repository is actively developed with assistance from Claude (via CLAUDE.md files). When working on this codebase:
308
309- Check for existing CLAUDE.md files that provide additional context
310- Follow established patterns and conventions already in the codebase
311- Maintain consistency with existing code style and architecture
312- Consider that changes may be reviewed and extended by both human developers and AI assistants
313
314## Trust These Instructions
315
316These instructions are comprehensive and tested. Only perform additional searches if:
317
3181. Information here is incomplete for your specific task
3192. You encounter errors not covered by the workarounds
3203. You need to understand implementation details not covered above
321
322For detailed platform development patterns, refer to `autogpt_platform/CLAUDE.md` and `AGENTS.md` in the repository root.
323
Significant-Gravitas/AutoGPT · classic/CLAUDE.md
@@ +1 @@
1# CLAUDE.md
2
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
5## Project Overview
6
7AutoGPT Classic is an experimental, **unsupported** project demonstrating autonomous GPT-4 operation. Dependencies will not be updated, and the codebase contains known vulnerabilities. This is preserved for educational/historical purposes.
8
9## Repository Structure
10
11```
12classic/
13├── pyproject.toml # Single consolidated Poetry project
14├── poetry.lock # Single lock file
15├── forge/
16│ └── forge/ # Core agent framework package
17├── original_autogpt/
18│ └── autogpt/ # AutoGPT agent package
19├── direct_benchmark/
20│ └── direct_benchmark/ # Benchmark harness package
21└── benchmark/ # Challenge definitions (data, not code)
22```
23
24All packages are managed by a single `pyproject.toml` at the classic/ root.
25
26## Common Commands
27
28### Setup & Install
29```bash
30# Install everything from classic/ directory
31cd classic
32poetry install
33```
34
35### Running Agents
36```bash
37# Run forge agent
38poetry run python -m forge
39
40# Run original autogpt server
41poetry run serve --debug
42
43# Run autogpt CLI
44poetry run autogpt
45```
46
47Agents run on `http://localhost:8000` by default.
48
49### Benchmarking
50```bash
51# Run benchmarks
52poetry run direct-benchmark run
53
54# Run specific strategies and models
55poetry run direct-benchmark run \
56 --strategies one_shot,rewoo \
57 --models claude \
58 --parallel 4
59
60# Run a single test
61poetry run direct-benchmark run --tests ReadFile
62
63# List available commands
64poetry run direct-benchmark --help
65```
66
67### Testing
68```bash
69poetry run pytest # All tests
70poetry run pytest forge/tests/ # Forge tests only
71poetry run pytest original_autogpt/tests/ # AutoGPT tests only
72poetry run pytest -k test_name # Single test by name
73poetry run pytest path/to/test.py # Specific test file
74poetry run pytest --cov # With coverage
75```
76
77### Linting & Formatting
78
79Run from the classic/ directory:
80
81```bash
82# Format everything (recommended to run together)
83poetry run black . && poetry run isort .
84
85# Check formatting (CI-style, no changes)
86poetry run black --check . && poetry run isort --check-only .
87
88# Lint
89poetry run flake8 # Style linting
90
91# Type check
92poetry run pyright # Type checking (some errors are expected in infrastructure code)
93```
94
95Note: Always run linters over the entire directory, not specific files, for best results.
96
97## Architecture
98
99### Forge (Core Framework)
100The `forge` package is the foundation that other components depend on:
101- `forge/agent/` - Agent implementation and protocols
102- `forge/llm/` - Multi-provider LLM integrations (OpenAI, Anthropic, Groq, LiteLLM)
103- `forge/components/` - Reusable agent components
104- `forge/file_storage/` - File system abstraction
105- `forge/config/` - Configuration management
106
107### Original AutoGPT
108- `original_autogpt/autogpt/app/` - CLI application entry points
109- `original_autogpt/autogpt/agents/` - Agent implementations
110- `original_autogpt/autogpt/agent_factory/` - Agent creation logic
111
112### Direct Benchmark
113Benchmark harness for testing agent performance:
114- `direct_benchmark/direct_benchmark/` - CLI and harness code
115- `benchmark/agbenchmark/challenges/` - Test cases organized by category (code, retrieval, data, etc.)
116- Reports generated in `direct_benchmark/reports/`
117
118### Package Structure
119All three packages are included in a single Poetry project. Imports are fully qualified:
120- `from forge.agent.base import BaseAgent`
121- `from autogpt.agents.agent import Agent`
122- `from direct_benchmark.harness import BenchmarkHarness`
123
124## Code Style
125
126- Python 3.12 target
127- Line length: 88 characters (Black default)
128- Black for formatting, isort for imports (profile="black")
129- Type hints with Pyright checking
130
131## Testing Patterns
132
133- Async support via pytest-asyncio
134- Fixtures defined in `conftest.py` files provide: `tmp_project_root`, `storage`, `config`, `llm_provider`, `agent`
135- Tests requiring API keys (OPENAI_API_KEY, ANTHROPIC_API_KEY) will skip if not set
136
137## Environment Setup
138
139Copy `.env.example` to `.env` in the relevant directory and add your API keys:
140```bash
141cp .env.example .env
142# Edit .env with your OPENAI_API_KEY, etc.
143```
144
145## Workspaces
146
147Agents operate within a **workspace** - a directory containing all agent data and files. The workspace root defaults to the current working directory.
148
149### Workspace Structure
150
151```
152{workspace}/
153├── .autogpt/
154│ ├── autogpt.yaml # Workspace-level permissions
155│ ├── ap_server.db # Agent Protocol database (server mode)
156│ └── agents/
157│ └── AutoGPT-{agent_id}/
158│ ├── state.json # Agent profile, directives, action history
159│ ├── permissions.yaml # Agent-specific permission overrides
160│ └── workspace/ # Agent's sandboxed working directory
161```
162
163### Key Concepts
164
165- **Multiple agents** can coexist in the same workspace (each gets its own subdirectory)
166- **File access** is sandboxed to the agent's `workspace/` directory by default
167- **State persistence** - agent state saves to `state.json` and survives across sessions
168- **Storage backends** - supports local filesystem, S3, and GCS (via `FILE_STORAGE_BACKEND` env var)
169
170### Specifying a Workspace
171
172```bash
173# Default: uses current directory
174cd /path/to/my/project && poetry run autogpt
175
176# Or specify explicitly via CLI (if supported)
177poetry run autogpt --workspace /path/to/workspace
178```
179
180## Settings Location
181
182Configuration uses a **layered system** with three levels (in order of precedence):
183
184### 1. Environment Variables (Global)
185
186Loaded from `.env` file in the working directory:
187
188```bash
189# Required
190OPENAI_API_KEY=sk-...
191
192# Optional LLM settings
193SMART_LLM=gpt-4o # Model for complex reasoning
194FAST_LLM=gpt-4o-mini # Model for simple tasks
195EMBEDDING_MODEL=text-embedding-3-small
196
197# Optional search providers (for web search component)
198TAVILY_API_KEY=tvly-...
199SERPER_API_KEY=...
200GOOGLE_API_KEY=...
201GOOGLE_CUSTOM_SEARCH_ENGINE_ID=...
202
203# Optional infrastructure
204LOG_LEVEL=DEBUG # DEBUG, INFO, WARNING, ERROR
205DATABASE_STRING=sqlite:///agent.db # Agent Protocol database
206PORT=8000 # Server port
207FILE_STORAGE_BACKEND=local # local, s3, or gcs
208```
209
210### 2. Workspace Settings (`{workspace}/.autogpt/autogpt.yaml`)
211
212Workspace-wide permissions that apply to **all agents** in this workspace:
213
214```yaml
215allow:
216 - read_file({workspace}/**)
217 - write_to_file({workspace}/**)
218 - list_folder({workspace}/**)
219 - web_search(*)
220
221deny:
222 - read_file(**.env)
223 - read_file(**.env.*)
224 - read_file(**.key)
225 - read_file(**.pem)
226 - execute_shell(rm -rf:*)
227 - execute_shell(sudo:*)
228```
229
230Auto-generated with sensible defaults if missing.
231
232### 3. Agent Settings (`{workspace}/.autogpt/agents/{id}/permissions.yaml`)
233
234Agent-specific permission overrides:
235
236```yaml
237allow:
238 - execute_python(*)
239 - web_search(*)
240
241deny:
242 - execute_shell(*)
243```
244
245## Permissions
246
247The permission system uses **pattern matching** with a **first-match-wins** evaluation order.
248
249### Permission Check Order
250
2511. Agent deny list → **Block**
2522. Workspace deny list → **Block**
2533. Agent allow list → **Allow**
2544. Workspace allow list → **Allow**
2555. Session denied list → **Block** (commands denied during this session)
2566. **Prompt user** → Interactive approval (if in interactive mode)
257
258### Pattern Syntax
259
260Format: `command_name(glob_pattern)`
261
262| Pattern | Description |
263|---------|-------------|
264| `read_file({workspace}/**)` | Read any file in workspace (recursive) |
265| `write_to_file({workspace}/*.txt)` | Write only .txt files in workspace root |
266| `execute_shell(python:**)` | Execute Python commands only |
267| `execute_shell(git:*)` | Execute any git command |
268| `web_search(*)` | Allow all web searches |
269
270Special tokens:
271- `{workspace}` - Replaced with actual workspace path
272- `**` - Matches any path including `/`
273- `*` - Matches any characters except `/`
274
275### Interactive Approval Scopes
276
277When prompted for permission, users can choose:
278
279| Scope | Effect |
280|-------|--------|
281| **Once** | Allow this one time only (not saved) |
282| **Agent** | Always allow for this agent (saves to agent `permissions.yaml`) |
283| **Workspace** | Always allow for all agents (saves to `autogpt.yaml`) |
284| **Deny** | Deny this command (saves to appropriate deny list) |
285
286### Default Security
287
288Out of the box, the following are **denied by default**:
289- Reading sensitive files (`.env`, `.key`, `.pem`)
290- Destructive shell commands (`rm -rf`, `sudo`)
291- Operations outside the workspace directory
292
@@ −1 +1 @@
1−# GitHub Copilot Instructions for AutoGPT
1+# CLAUDE.md
22
3−This file provides comprehensive onboarding information for GitHub Copilot coding agent to work efficiently with the AutoGPT repository.
3+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
44
5−## Repository Overview
5+## Project Overview
66
7−**AutoGPT** is a powerful platform for creating, deploying, and managing continuous AI agents that automate complex workflows. This is a large monorepo (~150MB) containing multiple components:
7+AutoGPT Classic is an experimental, **unsupported** project demonstrating autonomous GPT-4 operation. Dependencies will not be updated, and the codebase contains known vulnerabilities. This is preserved for educational/historical purposes.
88
9−- **AutoGPT Platform** (`autogpt_platform/`) - Main focus: Modern AI agent platform (Polyform Shield License)
10−- **Classic AutoGPT** (`classic/`) - Legacy agent system (MIT License)
11−- **Documentation** (`docs/`) - MkDocs-based documentation site
12−- **Infrastructure** - Docker configurations, CI/CD, and development tools
9+## Repository Structure
1310
14−**Primary Languages & Frameworks:**
11+```
12+classic/
13+├── pyproject.toml # Single consolidated Poetry project
14+├── poetry.lock # Single lock file
15+├── forge/
16+│ └── forge/ # Core agent framework package
17+├── original_autogpt/
18+│ └── autogpt/ # AutoGPT agent package
19+├── direct_benchmark/
20+│ └── direct_benchmark/ # Benchmark harness package
21+└── benchmark/ # Challenge definitions (data, not code)
22+```
1523
16−- **Backend**: Python 3.10-3.13, FastAPI, Prisma ORM, PostgreSQL, RabbitMQ
17−- **Frontend**: TypeScript, Next.js 15, React, Tailwind CSS, Radix UI
18−- **Development**: Docker, Poetry, pnpm, Playwright, Storybook
24+All packages are managed by a single `pyproject.toml` at the classic/ root.
1925
20−## Build and Validation Instructions
26+## Common Commands
2127
22−### Essential Setup Commands
28+### Setup & Install
29+```bash
30+# Install everything from classic/ directory
31+cd classic
32+poetry install
33+```
2334
24−**Always run these commands in the correct directory and in this order:**
35+### Running Agents
36+```bash
37+# Run forge agent
38+poetry run python -m forge
2539
26−1. **Initial Setup** (required once):
40+# Run original autogpt server
41+poetry run serve --debug
2742
28− ```bash
29− # Clone and enter repository
30− git clone <repo> && cd AutoGPT
43+# Run autogpt CLI
44+poetry run autogpt
45+```
3146
32− # Start all services (database, redis, rabbitmq, clamav)
33− cd autogpt_platform && docker compose --profile local up deps --build --detach
34− ```
47+Agents run on `http://localhost:8000` by default.
3548
36−2. **Backend Setup** (always run before backend development):
37−
38− ```bash
39− cd autogpt_platform/backend
40− poetry install # Install dependencies
41− poetry run prisma migrate dev # Run database migrations
42− poetry run prisma generate # Generate Prisma client
43− ```
44−
45−3. **Frontend Setup** (always run before frontend development):
46− ```bash
47− cd autogpt_platform/frontend
48− pnpm install # Install dependencies
49− ```
50−
51−### Runtime Requirements
52−
53−**Critical:** Always ensure Docker services are running before starting development:
54−
49+### Benchmarking
5550 ```bash
56−cd autogpt_platform && docker compose --profile local up deps --build --detach
57−```
51+# Run benchmarks
52+poetry run direct-benchmark run
5853
59−**Python Version:** Use Python 3.11 (required; managed by Poetry via pyproject.toml)
60−**Node.js Version:** Use Node.js 21+ with pnpm package manager
54+# Run specific strategies and models
55+poetry run direct-benchmark run \
56+ --strategies one_shot,rewoo \
57+ --models claude \
58+ --parallel 4
6159
62−### Development Commands
60+# Run a single test
61+poetry run direct-benchmark run --tests ReadFile
6362
64−**Backend Development:**
65−
66−```bash
67−cd autogpt_platform/backend
68−poetry run serve # Start development server (port 8000)
69−poetry run test # Run all tests (requires ~5 minutes)
70−poetry run pytest path/to/test.py # Run specific test
71−poetry run format # Format code (Black + isort) - always run first
72−poetry run lint # Lint code (ruff) - run after format
63+# List available commands
64+poetry run direct-benchmark --help
7365 ```
7466
75−**Frontend Development:**
76−
67+### Testing
7768 ```bash
78−cd autogpt_platform/frontend
79−pnpm dev # Start development server (port 3000) - use for active development
80−pnpm build # Build for production (only needed for E2E tests or deployment)
81−pnpm test # Run Playwright E2E tests (requires build first)
82−pnpm test-ui # Run tests with UI
83−pnpm format # Format and lint code
84−pnpm storybook # Start component development server
69+poetry run pytest # All tests
70+poetry run pytest forge/tests/ # Forge tests only
71+poetry run pytest original_autogpt/tests/ # AutoGPT tests only
72+poetry run pytest -k test_name # Single test by name
73+poetry run pytest path/to/test.py # Specific test file
74+poetry run pytest --cov # With coverage
8575 ```
8676
87−### Testing Strategy
77+### Linting & Formatting
8878
89−**Backend Tests:**
79+Run from the classic/ directory:
9080
91−- **Block Tests**: `poetry run pytest backend/blocks/test/test_block.py -xvs` (validates all blocks)
92−- **Specific Block**: `poetry run pytest 'backend/blocks/test/test_block.py::test_available_blocks[BlockName]' -xvs`
93−- **Snapshot Tests**: Use `--snapshot-update` when output changes, always review with `git diff`
81+```bash
82+# Format everything (recommended to run together)
83+poetry run black . && poetry run isort .
9484
95−**Frontend Tests:**
85+# Check formatting (CI-style, no changes)
86+poetry run black --check . && poetry run isort --check-only .
9687
97−- **E2E Tests**: Always run `pnpm dev` before `pnpm test` (Playwright requires running instance)
98−- **Component Tests**: Use Storybook for isolated component development
88+# Lint
89+poetry run flake8 # Style linting
9990
100−### Critical Validation Steps
91+# Type check
92+poetry run pyright # Type checking (some errors are expected in infrastructure code)
93+```
10194
102−**Before committing changes:**
95+Note: Always run linters over the entire directory, not specific files, for best results.
10396
104−1. Run `poetry run format` (backend) and `pnpm format` (frontend)
105−2. Ensure all tests pass in modified areas
106−3. Verify Docker services are still running
107−4. Check that database migrations apply cleanly
97+## Architecture
10898
109−**Common Issues & Workarounds:**
99+### Forge (Core Framework)
100+The `forge` package is the foundation that other components depend on:
101+- `forge/agent/` - Agent implementation and protocols
102+- `forge/llm/` - Multi-provider LLM integrations (OpenAI, Anthropic, Groq, LiteLLM)
103+- `forge/components/` - Reusable agent components
104+- `forge/file_storage/` - File system abstraction
105+- `forge/config/` - Configuration management
110106
111−- **Prisma issues**: Run `poetry run prisma generate` after schema changes
112−- **Permission errors**: Ensure Docker has proper permissions
113−- **Port conflicts**: Check the `docker-compose.yml` file for the current list of exposed ports. You can list all mapped ports with:
114−- **Test timeouts**: Backend tests can take 5+ minutes, use `-x` flag to stop on first failure
107+### Original AutoGPT
108+- `original_autogpt/autogpt/app/` - CLI application entry points
109+- `original_autogpt/autogpt/agents/` - Agent implementations
110+- `original_autogpt/autogpt/agent_factory/` - Agent creation logic
115111
116−## Project Layout & Architecture
112+### Direct Benchmark
113+Benchmark harness for testing agent performance:
114+- `direct_benchmark/direct_benchmark/` - CLI and harness code
115+- `benchmark/agbenchmark/challenges/` - Test cases organized by category (code, retrieval, data, etc.)
116+- Reports generated in `direct_benchmark/reports/`
117117
118−### Core Architecture
118+### Package Structure
119+All three packages are included in a single Poetry project. Imports are fully qualified:
120+- `from forge.agent.base import BaseAgent`
121+- `from autogpt.agents.agent import Agent`
122+- `from direct_benchmark.harness import BenchmarkHarness`
119123
120−**AutoGPT Platform** (`autogpt_platform/`):
124+## Code Style
121125
122−- `backend/` - FastAPI server with async support
123− - `backend/backend/` - Core API logic
124− - `backend/blocks/` - Agent execution blocks
125− - `backend/data/` - Database models and schemas
126− - `schema.prisma` - Database schema definition
127−- `frontend/` - Next.js application
128− - `src/app/` - App Router pages and layouts
129− - `src/components/` - Reusable React components
130− - `src/lib/` - Utilities and configurations
131−- `autogpt_libs/` - Shared Python utilities
132−- `docker-compose.yml` - Development stack orchestration
126+- Python 3.12 target
127+- Line length: 88 characters (Black default)
128+- Black for formatting, isort for imports (profile="black")
129+- Type hints with Pyright checking
133130
134−**Key Configuration Files:**
131+## Testing Patterns
135132
136−- `pyproject.toml` - Python dependencies and tooling
137−- `package.json` - Node.js dependencies and scripts
138−- `schema.prisma` - Database schema and migrations
139−- `next.config.mjs` - Next.js configuration
140−- `tailwind.config.ts` - Styling configuration
133+- Async support via pytest-asyncio
134+- Fixtures defined in `conftest.py` files provide: `tmp_project_root`, `storage`, `config`, `llm_provider`, `agent`
135+- Tests requiring API keys (OPENAI_API_KEY, ANTHROPIC_API_KEY) will skip if not set
141136
142−### Security & Middleware
137+## Environment Setup
143138
144−**Cache Protection**: Backend includes middleware preventing sensitive data caching in browsers/proxies
145−**Authentication**: JWT-based with Supabase integration
146−**User ID Validation**: All data access requires user ID checks - verify this for any `data/*.py` changes
139+Copy `.env.example` to `.env` in the relevant directory and add your API keys:
140+```bash
141+cp .env.example .env
142+# Edit .env with your OPENAI_API_KEY, etc.
143+```
147144
148−### Development Workflow
145+## Workspaces
149146
150−**GitHub Actions**: Multiple CI/CD workflows in `.github/workflows/`
147+Agents operate within a **workspace** - a directory containing all agent data and files. The workspace root defaults to the current working directory.
151148
152−- `platform-backend-ci.yml` - Backend testing and validation
153−- `platform-frontend-ci.yml` - Frontend testing and validation
154−- `platform-fullstack-ci.yml` - End-to-end integration tests
149+### Workspace Structure
155150
156−**Pre-commit Hooks**: Run linting and formatting checks
157−**Conventional Commits**: Use format `type(scope): description` (e.g., `feat(backend): add API`)
151+```
152+{workspace}/
153+├── .autogpt/
154+│ ├── autogpt.yaml # Workspace-level permissions
155+│ ├── ap_server.db # Agent Protocol database (server mode)
156+│ └── agents/
157+│ └── AutoGPT-{agent_id}/
158+│ ├── state.json # Agent profile, directives, action history
159+│ ├── permissions.yaml # Agent-specific permission overrides
160+│ └── workspace/ # Agent's sandboxed working directory
161+```
158162
159−### Key Source Files
163+### Key Concepts
160164
161−**Backend Entry Points:**
165+- **Multiple agents** can coexist in the same workspace (each gets its own subdirectory)
166+- **File access** is sandboxed to the agent's `workspace/` directory by default
167+- **State persistence** - agent state saves to `state.json` and survives across sessions
168+- **Storage backends** - supports local filesystem, S3, and GCS (via `FILE_STORAGE_BACKEND` env var)
162169
163−- `backend/backend/api/rest_api.py` - FastAPI application setup
164−- `backend/backend/data/` - Database models and user management
165−- `backend/blocks/` - Agent execution blocks and logic
170+### Specifying a Workspace
166171
167−**Frontend Entry Points:**
172+```bash
173+# Default: uses current directory
174+cd /path/to/my/project && poetry run autogpt
168175
169−- `frontend/src/app/layout.tsx` - Root application layout
170−- `frontend/src/app/page.tsx` - Home page
171−- `frontend/src/lib/supabase/` - Authentication and database client
176+# Or specify explicitly via CLI (if supported)
177+poetry run autogpt --workspace /path/to/workspace
178+```
172179
173−**Protected Routes**: Update `frontend/lib/supabase/middleware.ts` when adding protected routes
180+## Settings Location
174181
175−### Agent Block System
182+Configuration uses a **layered system** with three levels (in order of precedence):
176183
177−Agents are built using a visual block-based system where each block performs a single action. Blocks are defined in `backend/blocks/` and must include:
184+### 1. Environment Variables (Global)
178185
179−- Block definition with input/output schemas
180−- Execution logic with proper error handling
181−- Tests validating functionality
186+Loaded from `.env` file in the working directory:
182187
183−### Database & ORM
188+```bash
189+# Required
190+OPENAI_API_KEY=sk-...
184191
185−**Prisma ORM** with PostgreSQL backend including pgvector for embeddings:
192+# Optional LLM settings
193+SMART_LLM=gpt-4o # Model for complex reasoning
194+FAST_LLM=gpt-4o-mini # Model for simple tasks
195+EMBEDDING_MODEL=text-embedding-3-small
186196
187−- Schema in `schema.prisma`
188−- Migrations in `backend/migrations/`
189−- Always run `prisma migrate dev` and `prisma generate` after schema changes
197+# Optional search providers (for web search component)
198+TAVILY_API_KEY=tvly-...
199+SERPER_API_KEY=...
200+GOOGLE_API_KEY=...
201+GOOGLE_CUSTOM_SEARCH_ENGINE_ID=...
190202
191−## Environment Configuration
203+# Optional infrastructure
204+LOG_LEVEL=DEBUG # DEBUG, INFO, WARNING, ERROR
205+DATABASE_STRING=sqlite:///agent.db # Agent Protocol database
206+PORT=8000 # Server port
207+FILE_STORAGE_BACKEND=local # local, s3, or gcs
208+```
192209
193−### Configuration Files Priority Order
210+### 2. Workspace Settings (`{workspace}/.autogpt/autogpt.yaml`)
194211
195−1. **Backend**: `/backend/.env.default` → `/backend/.env` (user overrides)
196−2. **Frontend**: `/frontend/.env.default` → `/frontend/.env` (user overrides)
197−3. **Platform**: `/.env.default` (Supabase/shared) → `/.env` (user overrides)
198−4. Docker Compose `environment:` sections override file-based config
199−5. Shell environment variables have highest precedence
212+Workspace-wide permissions that apply to **all agents** in this workspace:
200213
201−### Docker Environment Setup
214+```yaml
215+allow:
216+ - read_file({workspace}/**)
217+ - write_to_file({workspace}/**)
218+ - list_folder({workspace}/**)
219+ - web_search(*)
202220
203−- All services use hardcoded defaults (no `${VARIABLE}` substitutions)
204−- The `env_file` directive loads variables INTO containers at runtime
205−- Backend/Frontend services use YAML anchors for consistent configuration
206−- Copy `.env.default` files to `.env` for local development customization
221+deny:
222+ - read_file(**.env)
223+ - read_file(**.env.*)
224+ - read_file(**.key)
225+ - read_file(**.pem)
226+ - execute_shell(rm -rf:*)
227+ - execute_shell(sudo:*)
228+```
207229
208−## Advanced Development Patterns
230+Auto-generated with sensible defaults if missing.
209231
210−### Adding New Blocks
232+### 3. Agent Settings (`{workspace}/.autogpt/agents/{id}/permissions.yaml`)
211233
212−1. Create file in `/backend/backend/blocks/`
213−2. Inherit from `Block` base class with input/output schemas
214−3. Implement `run` method with proper error handling
215−4. Generate block UUID using `uuid.uuid4()`
216−5. Register in block registry
217−6. Write tests alongside block implementation
218−7. Consider how inputs/outputs connect with other blocks in graph editor
234+Agent-specific permission overrides:
219235
220−### API Development
236+```yaml
237+allow:
238+ - execute_python(*)
239+ - web_search(*)
221240
222−1. Update routes in `/backend/backend/api/features/`
223−2. Add/update Pydantic models in same directory
224−3. Write tests alongside route files
225−4. For `data/*.py` changes, validate user ID checks
226−5. Run `poetry run test` to verify changes
241+deny:
242+ - execute_shell(*)
243+```
227244
228−### Frontend Development
245+## Permissions
229246
230−**📖 Complete Frontend Guide**: See `autogpt_platform/frontend/CONTRIBUTING.md` and `autogpt_platform/frontend/.cursorrules` for comprehensive patterns and conventions.
247+The permission system uses **pattern matching** with a **first-match-wins** evaluation order.
231248
232−**Quick Reference:**
249+### Permission Check Order
233250
234−**Component Structure:**
251+1. Agent deny list → **Block**
252+2. Workspace deny list → **Block**
253+3. Agent allow list → **Allow**
254+4. Workspace allow list → **Allow**
255+5. Session denied list → **Block** (commands denied during this session)
256+6. **Prompt user** → Interactive approval (if in interactive mode)
235257
236−- Separate render logic from data/behavior
237−- Structure: `ComponentName/ComponentName.tsx` + `useComponentName.ts` + `helpers.ts`
238−- Exception: Small components (3-4 lines of logic) can be inline
239−- Render-only components can be direct files without folders
258+### Pattern Syntax
240259
241−**Data Fetching:**
260+Format: `command_name(glob_pattern)`
242261
243−- Use generated API hooks from `@/app/api/__generated__/endpoints/`
244−- Generated via Orval from backend OpenAPI spec
245−- Pattern: `use{Method}{Version}{OperationName}`
246−- Example: `useGetV2ListLibraryAgents`
247−- Regenerate with: `pnpm generate:api`
248−- **Never** use deprecated `BackendAPI` or `src/lib/autogpt-server-api/*`
262+| Pattern | Description |
263+|---------|-------------|
264+| `read_file({workspace}/**)` | Read any file in workspace (recursive) |
265+| `write_to_file({workspace}/*.txt)` | Write only .txt files in workspace root |
266+| `execute_shell(python:**)` | Execute Python commands only |
267+| `execute_shell(git:*)` | Execute any git command |
268+| `web_search(*)` | Allow all web searches |
249269
250−**Code Conventions:**
270+Special tokens:
271+- `{workspace}` - Replaced with actual workspace path
272+- `**` - Matches any path including `/`
273+- `*` - Matches any characters except `/`
251274
252−- Use function declarations for components and handlers (not arrow functions)
253−- Only arrow functions for small inline lambdas (map, filter, etc.)
254−- Components: `PascalCase`, Hooks: `camelCase` with `use` prefix
255−- No barrel files or `index.ts` re-exports
256−- Minimal comments (code should be self-documenting)
275+### Interactive Approval Scopes
257276
258−**Styling:**
277+When prompted for permission, users can choose:
259278
260−- Use Tailwind CSS utilities only
261−- Use design system components from `src/components/` (atoms, molecules, organisms)
262−- Never use `src/components/__legacy__/*`
263−- Only use Phosphor Icons (`@phosphor-icons/react`)
264−- Prefer design tokens over hardcoded values
279+| Scope | Effect |
280+|-------|--------|
281+| **Once** | Allow this one time only (not saved) |
282+| **Agent** | Always allow for this agent (saves to agent `permissions.yaml`) |
283+| **Workspace** | Always allow for all agents (saves to `autogpt.yaml`) |
284+| **Deny** | Deny this command (saves to appropriate deny list) |
265285
266−**Error Handling:**
286+### Default Security
267287
268−- Render errors: Use `<ErrorCard />` component
269−- Mutation errors: Display with toast notifications
270−- Manual exceptions: Use `Sentry.captureException()`
271−- Global error boundaries already configured
272−
273−**Testing:**
274−
275−- Add/update Storybook stories for UI components (`pnpm storybook`)
276−- Run Playwright E2E tests with `pnpm test`
277−- Verify in Chromatic after PR
278−
279−**Architecture:**
280−
281−- Default to client components ("use client")
282−- Server components only for SEO or extreme TTFB needs
283−- Use React Query for server state (via generated hooks)
284−- Co-locate UI state in components/hooks
285−
286−### Security Guidelines
287−
288−**Cache Protection Middleware** (`/backend/backend/api/middleware/security.py`):
289−
290−- Default: Disables caching for ALL endpoints with `Cache-Control: no-store, no-cache, must-revalidate, private`
291−- Uses allow list approach for cacheable paths (static assets, health checks, public pages)
292−- Prevents sensitive data caching in browsers/proxies
293−- Add new cacheable endpoints to `CACHEABLE_PATHS`
294−
295−### CI/CD Alignment
296−
297−The repository has comprehensive CI workflows that test:
298−
299−- **Backend**: Python 3.11-3.13, services (Redis/RabbitMQ/ClamAV), Prisma migrations, Poetry lock validation
300−- **Frontend**: Node.js 21, pnpm, Playwright with Docker Compose stack, API schema validation
301−- **Integration**: Full-stack type checking and E2E testing
302−
303−Match these patterns when developing locally - the copilot setup environment mirrors these CI configurations.
304−
305−## Collaboration with Other AI Assistants
306−
307−This repository is actively developed with assistance from Claude (via CLAUDE.md files). When working on this codebase:
308−
309−- Check for existing CLAUDE.md files that provide additional context
310−- Follow established patterns and conventions already in the codebase
311−- Maintain consistency with existing code style and architecture
312−- Consider that changes may be reviewed and extended by both human developers and AI assistants
313−
314−## Trust These Instructions
315−
316−These instructions are comprehensive and tested. Only perform additional searches if:
317−
318−1. Information here is incomplete for your specific task
319−2. You encounter errors not covered by the workarounds
320−3. You need to understand implementation details not covered above
321−
322−For detailed platform development patterns, refer to `autogpt_platform/CLAUDE.md` and `AGENTS.md` in the repository root.
288+Out of the box, the following are **denied by default**:
289+- Reading sensitive files (`.env`, `.key`, `.pem`)
290+- Destructive shell commands (`rm -rf`, `sudo`)
291+- Operations outside the workspace directory
323292
