Copilot instructions

1# GitHub Copilot Instructions for AutoGPT
2 
3This file provides comprehensive onboarding information for GitHub Copilot coding agent to work efficiently with the AutoGPT repository.
4 
5## Repository Overview
6 
7**AutoGPT** is a powerful platform for creating, deploying, and managing continuous AI agents that automate complex workflows. This is a large monorepo (~150MB) containing multiple components:
8 
9- **AutoGPT Platform** (`autogpt_platform/`) - Main focus: Modern AI agent platform (Polyform Shield License)
10- **Classic AutoGPT** (`classic/`) - Legacy agent system (MIT License)
11- **Documentation** (`docs/`) - MkDocs-based documentation site
12- **Infrastructure** - Docker configurations, CI/CD, and development tools
13 
14**Primary Languages & Frameworks:**
15 
16- **Backend**: Python 3.10-3.13, FastAPI, Prisma ORM, PostgreSQL, RabbitMQ
17- **Frontend**: TypeScript, Next.js 15, React, Tailwind CSS, Radix UI
18- **Development**: Docker, Poetry, pnpm, Playwright, Storybook
19 
20## Build and Validation Instructions
21 
22### Essential Setup Commands
23 
24**Always run these commands in the correct directory and in this order:**
25 
261. **Initial Setup** (required once):
27 
28```bash
29   # Clone and enter repository
30   git clone <repo> && cd AutoGPT
31 
32   # Start all services (database, redis, rabbitmq, clamav)
33   cd autogpt_platform && docker compose --profile local up deps --build --detach
34```
35 
362. **Backend Setup** (always run before backend development):
37 
38```bash
39   cd autogpt_platform/backend
40   poetry install                    # Install dependencies
41   poetry run prisma migrate dev     # Run database migrations
42   poetry run prisma generate        # Generate Prisma client
43```
44 
453. **Frontend Setup** (always run before frontend development):
46```bash
47   cd autogpt_platform/frontend
48   pnpm install                      # Install dependencies
49```
50 
51### Runtime Requirements
52 
53**Critical:** Always ensure Docker services are running before starting development:
54 
55```bash
56cd autogpt_platform && docker compose --profile local up deps --build --detach
57```
58 
59**Python Version:** Use Python 3.11 (required; managed by Poetry via pyproject.toml)
60**Node.js Version:** Use Node.js 21+ with pnpm package manager
61 
62### Development Commands
63 
64**Backend Development:**
65 
66```bash
67cd autogpt_platform/backend
68poetry run serve                     # Start development server (port 8000)
69poetry run test                      # Run all tests (requires ~5 minutes)
70poetry run pytest path/to/test.py    # Run specific test
71poetry run format                    # Format code (Black + isort) - always run first
72poetry run lint                      # Lint code (ruff) - run after format
73```
74 
75**Frontend Development:**
76 
77```bash
78cd autogpt_platform/frontend
79pnpm dev                            # Start development server (port 3000) - use for active development
80pnpm build                          # Build for production (only needed for E2E tests or deployment)
81pnpm test                           # Run Playwright E2E tests (requires build first)
82pnpm test-ui                        # Run tests with UI
83pnpm format                         # Format and lint code
84pnpm storybook                      # Start component development server
85```
86 
87### Testing Strategy
88 
89**Backend Tests:**
90 
91- **Block Tests**: `poetry run pytest backend/blocks/test/test_block.py -xvs` (validates all blocks)
92- **Specific Block**: `poetry run pytest 'backend/blocks/test/test_block.py::test_available_blocks[BlockName]' -xvs`
93- **Snapshot Tests**: Use `--snapshot-update` when output changes, always review with `git diff`
94 
95**Frontend Tests:**
96 
97- **E2E Tests**: Always run `pnpm dev` before `pnpm test` (Playwright requires running instance)
98- **Component Tests**: Use Storybook for isolated component development
99 
100### Critical Validation Steps
101 
102**Before committing changes:**
103 
1041. Run `poetry run format` (backend) and `pnpm format` (frontend)
1052. Ensure all tests pass in modified areas
1063. Verify Docker services are still running
1074. Check that database migrations apply cleanly
108 
109**Common Issues & Workarounds:**
110 
111- **Prisma issues**: Run `poetry run prisma generate` after schema changes
112- **Permission errors**: Ensure Docker has proper permissions
113- **Port conflicts**: Check the `docker-compose.yml` file for the current list of exposed ports. You can list all mapped ports with:
114- **Test timeouts**: Backend tests can take 5+ minutes, use `-x` flag to stop on first failure
115 
116## Project Layout & Architecture
117 
118### Core Architecture
119 
120**AutoGPT Platform** (`autogpt_platform/`):
121 
122- `backend/` - FastAPI server with async support
123  - `backend/backend/` - Core API logic
124  - `backend/blocks/` - Agent execution blocks
125  - `backend/data/` - Database models and schemas
126  - `schema.prisma` - Database schema definition
127- `frontend/` - Next.js application
128  - `src/app/` - App Router pages and layouts
129  - `src/components/` - Reusable React components
130  - `src/lib/` - Utilities and configurations
131- `autogpt_libs/` - Shared Python utilities
132- `docker-compose.yml` - Development stack orchestration
133 
134**Key Configuration Files:**
135 
136- `pyproject.toml` - Python dependencies and tooling
137- `package.json` - Node.js dependencies and scripts
138- `schema.prisma` - Database schema and migrations
139- `next.config.mjs` - Next.js configuration
140- `tailwind.config.ts` - Styling configuration
141 
142### Security & Middleware
143 
144**Cache Protection**: Backend includes middleware preventing sensitive data caching in browsers/proxies
145**Authentication**: JWT-based with Supabase integration
146**User ID Validation**: All data access requires user ID checks - verify this for any `data/*.py` changes
147 
148### Development Workflow
149 
150**GitHub Actions**: Multiple CI/CD workflows in `.github/workflows/`
151 
152- `platform-backend-ci.yml` - Backend testing and validation
153- `platform-frontend-ci.yml` - Frontend testing and validation
154- `platform-fullstack-ci.yml` - End-to-end integration tests
155 
156**Pre-commit Hooks**: Run linting and formatting checks
157**Conventional Commits**: Use format `type(scope): description` (e.g., `feat(backend): add API`)
158 
159### Key Source Files
160 
161**Backend Entry Points:**
162 
163- `backend/backend/api/rest_api.py` - FastAPI application setup
164- `backend/backend/data/` - Database models and user management
165- `backend/blocks/` - Agent execution blocks and logic
166 
167**Frontend Entry Points:**
168 
169- `frontend/src/app/layout.tsx` - Root application layout
170- `frontend/src/app/page.tsx` - Home page
171- `frontend/src/lib/supabase/` - Authentication and database client
172 
173**Protected Routes**: Update `frontend/lib/supabase/middleware.ts` when adding protected routes
174 
175### Agent Block System
176 
177Agents are built using a visual block-based system where each block performs a single action. Blocks are defined in `backend/blocks/` and must include:
178 
179- Block definition with input/output schemas
180- Execution logic with proper error handling
181- Tests validating functionality
182 
183### Database & ORM
184 
185**Prisma ORM** with PostgreSQL backend including pgvector for embeddings:
186 
187- Schema in `schema.prisma`
188- Migrations in `backend/migrations/`
189- Always run `prisma migrate dev` and `prisma generate` after schema changes
190 
191## Environment Configuration
192 
193### Configuration Files Priority Order
194 
1951. **Backend**: `/backend/.env.default` → `/backend/.env` (user overrides)
1962. **Frontend**: `/frontend/.env.default` → `/frontend/.env` (user overrides)
1973. **Platform**: `/.env.default` (Supabase/shared) → `/.env` (user overrides)
1984. Docker Compose `environment:` sections override file-based config
1995. Shell environment variables have highest precedence
200 
201### Docker Environment Setup
202 
203- All services use hardcoded defaults (no `${VARIABLE}` substitutions)
204- The `env_file` directive loads variables INTO containers at runtime
205- Backend/Frontend services use YAML anchors for consistent configuration
206- Copy `.env.default` files to `.env` for local development customization
207 
208## Advanced Development Patterns
209 
210### Adding New Blocks
211 
2121. Create file in `/backend/backend/blocks/`
2132. Inherit from `Block` base class with input/output schemas
2143. Implement `run` method with proper error handling
2154. Generate block UUID using `uuid.uuid4()`
2165. Register in block registry
2176. Write tests alongside block implementation
2187. Consider how inputs/outputs connect with other blocks in graph editor
219 
220### API Development
221 
2221. Update routes in `/backend/backend/api/features/`
2232. Add/update Pydantic models in same directory
2243. Write tests alongside route files
2254. For `data/*.py` changes, validate user ID checks
2265. Run `poetry run test` to verify changes
227 
228### Frontend Development
229 
230**📖 Complete Frontend Guide**: See `autogpt_platform/frontend/CONTRIBUTING.md` and `autogpt_platform/frontend/.cursorrules` for comprehensive patterns and conventions.
231 
232**Quick Reference:**
233 
234**Component Structure:**
235 
236- Separate render logic from data/behavior
237- Structure: `ComponentName/ComponentName.tsx` + `useComponentName.ts` + `helpers.ts`
238- Exception: Small components (3-4 lines of logic) can be inline
239- Render-only components can be direct files without folders
240 
241**Data Fetching:**
242 
243- Use generated API hooks from `@/app/api/__generated__/endpoints/`
244- Generated via Orval from backend OpenAPI spec
245- Pattern: `use{Method}{Version}{OperationName}`
246- Example: `useGetV2ListLibraryAgents`
247- Regenerate with: `pnpm generate:api`
248- **Never** use deprecated `BackendAPI` or `src/lib/autogpt-server-api/*`
249 
250**Code Conventions:**
251 
252- Use function declarations for components and handlers (not arrow functions)
253- Only arrow functions for small inline lambdas (map, filter, etc.)
254- Components: `PascalCase`, Hooks: `camelCase` with `use` prefix
255- No barrel files or `index.ts` re-exports
256- Minimal comments (code should be self-documenting)
257 
258**Styling:**
259 
260- Use Tailwind CSS utilities only
261- Use design system components from `src/components/` (atoms, molecules, organisms)
262- Never use `src/components/__legacy__/*`
263- Only use Phosphor Icons (`@phosphor-icons/react`)
264- Prefer design tokens over hardcoded values
265 
266**Error Handling:**
267 
268- Render errors: Use `<ErrorCard />` component
269- Mutation errors: Display with toast notifications
270- Manual exceptions: Use `Sentry.captureException()`
271- Global error boundaries already configured
272 
273**Testing:**
274 
275- Add/update Storybook stories for UI components (`pnpm storybook`)
276- Run Playwright E2E tests with `pnpm test`
277- Verify in Chromatic after PR
278 
279**Architecture:**
280 
281- Default to client components ("use client")
282- Server components only for SEO or extreme TTFB needs
283- Use React Query for server state (via generated hooks)
284- Co-locate UI state in components/hooks
285 
286### Security Guidelines
287 
288**Cache Protection Middleware** (`/backend/backend/api/middleware/security.py`):
289 
290- Default: Disables caching for ALL endpoints with `Cache-Control: no-store, no-cache, must-revalidate, private`
291- Uses allow list approach for cacheable paths (static assets, health checks, public pages)
292- Prevents sensitive data caching in browsers/proxies
293- Add new cacheable endpoints to `CACHEABLE_PATHS`
294 
295### CI/CD Alignment
296 
297The repository has comprehensive CI workflows that test:
298 
299- **Backend**: Python 3.11-3.13, services (Redis/RabbitMQ/ClamAV), Prisma migrations, Poetry lock validation
300- **Frontend**: Node.js 21, pnpm, Playwright with Docker Compose stack, API schema validation
301- **Integration**: Full-stack type checking and E2E testing
302 
303Match these patterns when developing locally - the copilot setup environment mirrors these CI configurations.
304 
305## Collaboration with Other AI Assistants
306 
307This repository is actively developed with assistance from Claude (via CLAUDE.md files). When working on this codebase:
308 
309- Check for existing CLAUDE.md files that provide additional context
310- Follow established patterns and conventions already in the codebase
311- Maintain consistency with existing code style and architecture
312- Consider that changes may be reviewed and extended by both human developers and AI assistants
313 
314## Trust These Instructions
315 
316These instructions are comprehensive and tested. Only perform additional searches if:
317 
3181. Information here is incomplete for your specific task
3192. You encounter errors not covered by the workarounds
3203. You need to understand implementation details not covered above
321 
322For detailed platform development patterns, refer to `autogpt_platform/CLAUDE.md` and `AGENTS.md` in the repository root.
323