Two files, one repository
dragonflydb/dragonfly ships 2 formats across 3 indexed files. The question worth asking is whether the second one says anything the first does not.
| Dimension | Shared | Only in A | Only in B | Overlap |
|---|---|---|---|---|
| Sections | 0 | 47 | 4 | 0% |
| Commands | 0 | 10 | 0 | 0% |
| Section tags | 4 | 10 | 0 | 29% |
What each file covers
Sections
0 shared · 47 only in A · 4 only in B- − Dragonfly Development Guide
- − Table of Contents
- − Critical Workflow Rules
- − Pull Request Guidelines
- − Quick Command Reference
- − Building (see [Build Instructions](#build-instructions) for details)
- − Debug build (for development)
- − Release build for local benchmarking
- − Testing (see [Testing](#testing) for details)
- − C++ Unit Tests
- − Code Formatting
- − Setup (once)
- − Format code
- − Common Operations
- − Check git status
- − Check current branch
- − View recent commits
- − Architecture Patterns
- − Project Overview
- − Key Characteristics
- − Architectural Highlights
- − Repository Structure
- − Critical Paths to Remember
- − Build Instructions
- − Quick Start
- − Testing
- − Quick Reference
- − Run from the repo root. The binary path defaults to build-dbg/dragonfly.
- − Override with the DRAGONFLY_PATH env var:
- − Run a single test:
- − CI/CD Pipeline
- − Code Style & Pre-commit Hooks
- − Third-Party Dependencies
- − Platform Support
- − CMake Build Options
- − Common Options
- − Minimal build (fast compilation)
- − Full-featured (all options ON by default)
- − Production optimized
- − Key Files Reference
- − Common Pitfalls
- − Debugging Tips
- − Validation Checklist
- − Code Quality
- − Pre-commit & Style
- − Documentation
- − Performance
- + Code Review Instructions
- + Comment Only When
- + Avoid
- + Review Style
Commands
0 shared · 10 only in A · 0 only in B- − git status
- − git branch
- − git log --oneline -10
- − make release
- − make package
- − python3 -m pytest tests/dragonfly/pymemcached_test.py::TestMemcached::test_basic -xvs
- − ninja <unit_test> && ./unit_test
- − git push origin main
- − ninja
- − make
Section tags
4 shared · 10 only in A · 0 only in B- − setup
- − build
- − test
- − lint-format
- − architecture
- − testing-strategy
- − dependencies
- − performance
- − deployment
- − docs
- code-style
- git-pr
- do-not
- agent-behaviour
Line diff
dragonflydb/dragonfly · AGENTS.md
@@ −1 @@
1# Dragonfly Development Guide
2
3> **Essential reference for working with the Dragonfly codebase**
4> Architecture, build system, testing infrastructure, and development workflows.
5
6---
7
8## Table of Contents
9
101. [Critical Workflow Rules](#critical-workflow-rules)
112. [Quick Command Reference](#quick-command-reference)
123. [Project Overview](#project-overview)
134. [Repository Structure](#repository-structure)
145. [Build Instructions](#build-instructions)
156. [Testing](#testing)
167. [CI/CD Pipeline](#cicd-pipeline)
178. [Code Style & Pre-commit Hooks](#code-style--pre-commit-hooks)
189. [Third-Party Dependencies](#third-party-dependencies)
1910. [Platform Support](#platform-support)
2011. [CMake Build Options](#cmake-build-options)
2112. [Key Files Reference](#key-files-reference)
2213. [Common Pitfalls](#common-pitfalls)
2314. [Debugging Tips](#debugging-tips)
2415. [Validation Checklist](#validation-checklist)
25
26---
27
28## Critical Workflow Rules
29
30**MANDATORY - Always Follow This Order:**
31
321. ✅ **Read Before Edit** - Always read files before modifying
332. ✅ **Use Correct Build Commands** - See [Quick Command Reference](#quick-command-reference) below
343. ✅ **Test After Changes** - Build and run a relevant unit test -
35 `ninja <unit_test> && ./unit_test`
364. ✅ **Format Code** - `pre-commit run --files <files>`
375. ✅ **Follow Architecture** - See [Architecture Patterns](#architecture-patterns) below
386. ✅ **Never Push to Main** - Always create a feature branch and open a PR. Never run `git push origin main`.
39
40### Pull Request Guidelines
41
42**Conciseness is Key**: PR descriptions should be short, focused, and easy to scan.
43- **Title**: Imperative, descriptive (e.g., "Fix fiber stack overflow in test_reply_guard_oom")
44- **Summary**: 1-2 sentences explaining *what* changed and *why*
45- **Changes**: Bullet points for key changes
46- **Fixes**: Link issues (e.g., "Fixes #123")
47- **Commit messages**: Keep every line (subject and body) <= 100 characters; wrap long descriptions
48
49---
50
51## Quick Command Reference
52
53**CRITICAL: Read the full sections below for context. These are shortcuts only.**
54
55### Building (see [Build Instructions](#build-instructions) for details)
56
57```bash
58# Debug build (for development)
59./helio/blaze.sh -DWITH_AWS=OFF -DWITH_GCP=OFF
60cd build-dbg && ninja dragonfly # Build main binary
61cd build-dbg && ninja generic_family_test # Build specific test
62
63# Release build for local benchmarking
64./helio/blaze.sh -release -DWITH_AWS=OFF -DWITH_GCP=OFF
65cd build-opt && ninja dragonfly
66```
67
68### Testing (see [Testing](#testing) for details)
69
70```bash
71# C++ Unit Tests
72cd build-dbg
73ctest -V -L DFLY # Run all tests
74./generic_family_test # Run specific test binary
75./generic_family_test --gtest_filter="Set.*" # Run specific test case
76```
77
78### Code Formatting
79
80```bash
81# Setup (once)
82pipx install pre-commit clang-format black
83pre-commit install
84
85# Format code
86pre-commit run --files <files> # Format specific files
87pre-commit run --all-files # Format all files
88```
89
90### Common Operations
91
92```bash
93# Check git status
94git status
95
96# Check current branch
97git branch
98
99# View recent commits
100git log --oneline -10
101```
102
103---
104
105## Architecture Patterns
106
107**Code Style**: [.clang-format](.clang-format) - snake_case vars, PascalCase functions, kPascalCase constants
108
109**DO ✅**:
110- Fiber-aware: `util::fb2::Mutex`, `util::fb2::Fiber` → [helio/util/fibers/](helio/util/fibers/)
111- Per-shard ops (no global state) → [docs/df-share-nothing.md](docs/df-share-nothing.md)
112- Command pattern → [src/server/set_family.cc](src/server/set_family.cc)
113- Error handling: `OpStatus` → [src/server/common.h](src/server/common.h)
114- Test patterns → [tests/dragonfly/conftest.py](tests/dragonfly/conftest.py)
115
116**DON'T ❌**:
117- `std::thread`, `std::mutex` (deadlocks!)
118- Global mutable state
119- Edit without reading
120- Skip tests
121- Use `std::regex` in fiber/server paths (recursive implementation can overflow small fiber stacks)
122- Use `./tools/docker/build.sh` for local development (use `ninja` instead)
123- Use `make` for incremental builds (use `ninja` instead)
124
125---
126
127## Project Overview
128
129**Dragonfly** is a high-performance, Redis and Memcached compatible in-memory data store written in C++20. It delivers significantly higher throughput than traditional single-threaded Redis implementations through innovative architectural choices.
130
131### Key Characteristics
132
133- **Language**: C++20 (Google C++ Style Guide 2020 version)
134- **Architecture**: Shared-nothing multi-threaded design (via `helio` library)
135- **Performance**: Uses io_uring (Linux 5.11+) for high-performance async I/O, with epoll fallback
136- **Threading Model**: Fiber-based cooperative multitasking with lock-free data structures
137- **Build System**: CMake + Ninja via `helio/blaze.sh` wrapper script
138- **Target Platform**: Linux (kernel 5.11+ recommended), FreeBSD support available
139- **Protocols**: Redis RESP2/RESP3, Memcached binary protocol
140- **Compatibility**: Drop-in replacement for Redis API coverage
141
142### Architectural Highlights
143
144**For detailed architecture documentation, see [docs/df-share-nothing.md](docs/df-share-nothing.md)**
145
1461. **Shared-Nothing Design**: Each thread operates independently with its own data structures, minimizing lock contention
1472. **Helio Framework**: Custom I/O and threading library built on io_uring/epoll with fiber support
1483. **DashTable**: Novel hash table implementation optimized for multi-core systems - see [docs/dashtable.md](docs/dashtable.md)
1494. **Transaction Model**: Non-blocking optimistic transactions - see [docs/transaction.md](docs/transaction.md)
1505. **Tiering Support**: Optional disk-backed storage for large datasets
1516. **Search Module**: Full-text search capabilities (when enabled with WITH_SEARCH)
152
153---
154
155## Repository Structure
156
157```
158dragonfly/
159├── src/ # Main C++ source code
160│ ├── server/ # Core server implementation
161│ │ ├── dfly_main.cc # Main entry point
162│ │ ├── main_service.cc # Service lifecycle & command routing
163│ │ ├── db_slice.cc # Per-thread database shard
164│ │ ├── engine_shard_set.cc # Shard management
165│ │ ├── cluster/ # Cluster mode implementation
166│ │ ├── journal/ # Replication journal
167│ │ ├── tiering/ # Tiered storage
168│ │ ├── search/ # Search module
169│ │ └── acl/ # Access control lists
170│ ├── core/ # Core data structures
171│ │ ├── dash.h # DashTable hash table
172│ │ ├── dense_set.h # Compact set implementation
173│ │ ├── string_map.h # Optimized string-keyed maps
174│ │ ├── search/ # Search core algorithms
175│ │ └── json/ # JSON support
176│ ├── facade/ # Network & command handling
177│ │ ├── dragonfly_connection.cc # Connection management
178│ │ ├── redis_parser.cc # RESP protocol parser
179│ │ └── memcache_parser.cc # Memcached protocol
180│ └── redis/ # Redis-specific implementations
181│ └── lua/ # Lua scripting support
182│
183├── helio/ # Git submodule: I/O and threading library
184│ │ # ** DO NOT EDIT unless contributing to helio **
185│ ├── util/ # Utilities: fibers, I/O, synchronization
186│ ├── io/ # io_uring & epoll abstraction
187│ └── blaze.sh # Build configuration wrapper
188│
189├── tests/ # Test suite
190│ ├── dragonfly/ # Python pytest integration/regression tests
191│ │ ├── conftest.py # Pytest fixtures & configuration
192│ │ ├── requirements.txt # Python test dependencies
193│ │ └── *.py # Test files
194│ └── pytest.ini # Pytest configuration & markers
195│
196├── docs/ # Documentation
197│ ├── build-from-source.md # Build instructions
198│ ├── dashtable.md # DashTable internals
199│ ├── transaction.md # Transaction model
200│ ├── df-share-nothing.md # Shared-nothing architecture
201│ └── differences.md # Differences from Redis
202│
203├── contrib/ # Utilities
204│ ├── docker/ # Docker configurations
205│ └── charts/dragonfly/ # Helm chart for Kubernetes
206│
207├── tools/ # Benchmarking & utility tools
208│ └── packaging/ # Packaging scripts
209│
210├── CMakeLists.txt # Root CMake configuration
211├── .clang-format # C++ formatting rules (clang-format v14.0.6)
212├── .pre-commit-config.yaml # Pre-commit hooks configuration
213├── pyproject.toml # Python formatting (Black, 100 chars)
214└── CONTRIBUTING.md # Contribution guidelines
215```
216
217### Critical Paths to Remember
218
219- **Main entry**: `src/server/dfly_main.cc`
220- **Command dispatch**: `src/server/main_service.cc`
221- **Data storage**: `src/server/db_slice.cc`
222- **Networking**: `src/facade/dragonfly_connection.cc`
223- **Helio library**: `helio/` (I/O and threading library)
224
225---
226
227## Build Instructions
228
229**For complete build instructions, see [docs/build-from-source.md](docs/build-from-source.md)**
230
231### Quick Start
232
233**Debug build** (for development):
234```bash
235./helio/blaze.sh
236cd build-dbg && ninja dragonfly
237./dragonfly --alsologtostderr
238```
239
240**Release build** (for production/benchmarking):
241```bash
242./helio/blaze.sh -release
243cd build-opt && ninja dragonfly
244```
245
246**Production release build** (static linking, optimized):
247```bash
248make release # Configure + build
249make package # Create release packages with debug symbols
250```
251
252The [Makefile](Makefile) builds production releases with:
253- Static linking: libstdc++, libgcc, Boost, OpenSSL
254- Architecture optimizations (x86_64: `-march=core2 -msse4.1 -mtune=skylake`)
255- Debug symbols (compressed)
256- Output: `build-release/dragonfly-{arch}.tar.gz`
257
258**Common build options**:
259- See [docs/build-from-source.md](docs/build-from-source.md) for all options
260
261---
262
263## Testing
264
265**For complete testing documentation, see [tests/README.md](tests/README.md)**
266
267### Quick Reference
268
269**C++ Unit Tests**:
270```bash
271cd build-dbg
272ctest -V -L DFLY # Run all tests
273./generic_family_test # Run specific test binary
274./generic_family_test --gtest_filter="Set.*" # Run specific test case
275```
276
277**Python Integration Tests (pytest)**:
278```bash
279# Run from the repo root. The binary path defaults to build-dbg/dragonfly.
280# Override with the DRAGONFLY_PATH env var:
281DRAGONFLY_PATH=build-dbg/dragonfly python3 -m pytest tests/dragonfly/pymemcached_test.py -xvs
282
283# Run a single test:
284python3 -m pytest tests/dragonfly/pymemcached_test.py::TestMemcached::test_basic -xvs
285```
286
287- `DRAGONFLY_PATH` — sets the path to the Dragonfly binary the test harness starts. Defaults to `build-dbg/dragonfly` relative to the `tests/dragonfly/` directory.
288- `--df` — passes **extra flags to the Dragonfly process** (not the binary path). For example: `--df logtostdout --df "vmodule=*=1"`.
289
290---
291
292## CI/CD Pipeline
293
294**For complete CI configuration, see [.github/workflows/ci.yml](.github/workflows/ci.yml)**
295
296The CI workflow runs on all PRs and includes:
297- **Pre-commit checks**: clang-format, black formatters
298- **Build matrix**: Multiple OS/compiler/sanitizer combinations (Ubuntu 20/24, Alpine, GCC/Clang, ASAN/UBSAN)
299- **Test execution**: C++ unit tests, Python integration tests, cluster mode tests
300- **Additional validations**: Helm charts, Docker image builds
301
302---
303
304## Code Style & Pre-commit Hooks
305
306**For complete contribution guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md)**
307
308**Code style configuration files**:
309- **C++**: [.clang-format](.clang-format) - Google C++ Style Guide (2020), clang-format v14.0.6, 100 char limit
310- **Python**: [pyproject.toml](pyproject.toml) - Black formatter, 100 char limit, PEP 8 compliant
311- **Pre-commit hooks**: [.pre-commit-config.yaml](.pre-commit-config.yaml) - Automated formatting checks
312
313**Quick setup**:
314```bash
315pipx install pre-commit clang-format black
316pre-commit install
317pre-commit run --all-files # Run all formatters
318```
319
320---
321
322## Third-Party Dependencies
323
324**Key Libraries**: Abseil (strings/flags), Boost 1.71+ (context/intrusive), mimalloc (allocator), jsoncons (JSON), OpenSSL (TLS), libunwind (traces)
325
326**Build artifacts**: `build-dbg/third_party/` - DO NOT edit
327
328**For complete dependency info, see [docs/build-from-source.md](docs/build-from-source.md)**
329
330---
331
332## Platform Support
333
334**Linux**: Primary platform. Kernel 5.11+ (io_uring), 5.1+ (basic), < 5.1 (epoll fallback)
335- Check: `uname -r`
336- Force epoll: `--proactor_type=epoll`
337- Docker: `--security-opt seccomp=unconfined`
338
339**FreeBSD**: Supported (kqueue backend)
340
341**macOS**: Not supported for production (use Docker/Linux)
342
343**For complete platform info, see [docs/build-from-source.md](docs/build-from-source.md)**
344
345---
346
347## CMake Build Options
348
349**For complete list of build options, see [docs/build-from-source.md](docs/build-from-source.md)**
350
351### Common Options
352
353Pass options to `helio/blaze.sh` with `-D` prefix:
354
355```bash
356./helio/blaze.sh -DWITH_SEARCH=OFF -DWITH_AWS=ON
357```
358
359**Most useful options**:
360- `WITH_ASAN=ON` / `WITH_USAN=ON` - Enable sanitizers for debugging
361- `WITH_SEARCH=OFF` - Disable search module for faster builds
362- `WITH_AWS=OFF` / `WITH_GCP=OFF` - Disable cloud libraries
363- `WITH_TIERING=OFF` - Disable disk storage
364- `USE_MOLD=ON` - Faster linking with LTO (production builds)
365
366**Quick configurations**:
367```bash
368# Minimal build (fast compilation)
369./helio/blaze.sh -DWITH_GPERF=OFF -DWITH_AWS=OFF -DWITH_GCP=OFF -DWITH_TIERING=OFF -DWITH_SEARCH=OFF
370
371# Full-featured (all options ON by default)
372./helio/blaze.sh
373
374# Production optimized
375./helio/blaze.sh -release -DUSE_MOLD=ON
376```
377
378---
379
380## Key Files Reference
381
382Quick reference to the most important files in the codebase.
383
384| Purpose | File Path |
385|---------|-----------|
386| **Entry Points & Core** | |
387| Main entry point | `src/server/dfly_main.cc` |
388| Server lifecycle & command routing | `src/server/main_service.cc` |
389| Per-thread database shard | `src/server/db_slice.cc` |
390| Shard management | `src/server/engine_shard_set.cc` |
391| **Data Structures** | |
392| DashTable hash table | `src/core/dash.h` |
393| Dense set implementation | `src/core/dense_set.h` |
394| String map | `src/core/string_map.h` |
395| **Networking** | |
396| Connection handling | `src/facade/dragonfly_connection.cc` |
397| Redis protocol parser | `src/facade/redis_parser.cc` |
398| Memcached protocol parser | `src/facade/memcache_parser.cc` |
399| **Build System** | |
400| Root CMake config | `CMakeLists.txt` |
401| Build script wrapper | `helio/blaze.sh` |
402| Server CMake config | `src/server/CMakeLists.txt` |
403| **CI/CD** | |
404| Main CI workflow | `.github/workflows/ci.yml` |
405| Pre-commit config | `.pre-commit-config.yaml` |
406| **Code Style** | |
407| C++ formatting | `.clang-format` |
408| Python formatting | `pyproject.toml` |
409| **Testing** | |
410| Pytest configuration | `tests/pytest.ini` |
411| Pytest fixtures | `tests/dragonfly/conftest.py` |
412| Test requirements | `tests/dragonfly/requirements.txt` |
413| **Documentation** | |
414| Build instructions | `docs/build-from-source.md` |
415| Architecture overview | `docs/df-share-nothing.md` |
416| DashTable internals | `docs/dashtable.md` |
417| Transaction model | `docs/transaction.md` |
418| **Configuration** | |
419| Contributing guide | `CONTRIBUTING.md` |
420| CLA agreement | `CLA.txt` |
421
422---
423
424## Common Pitfalls
425
4261. **Pre-commit not installed**: `pipx install pre-commit clang-format black && pre-commit install`
4272. **Wrong binary**: Debug: `build-dbg/dragonfly`, Release: `build-opt/dragonfly`
4283. **Wrong build command**: Use `cd build-dbg && ninja <target>`, NOT `./tools/docker/build.sh`
4294. **Test timeouts**: `timeout 20m ctest -V -L DFLY`
4305. **ASAN leaks**: Check CI, suppress in `helio/util/asan_suppressions.txt`
4316. **Helio modifications**: DON'T edit `helio/` (it's a git submodule - changes go upstream)
4327. **CodeQL checks**: DON'T run codeql_checker when testing changes - it's slow and unnecessary for development
433
434---
435
436## Debugging Tips
437
438**Logging**: `--alsologtostderr --v=1 --vmodule=module=2`
439
440**ASAN**: `ASAN_OPTIONS=detect_leaks=1:symbolize=1`, suppressions: `helio/util/asan_suppressions.txt`
441
442**CI reproduction**: See [.github/workflows/ci.yml](.github/workflows/ci.yml)
443
444**Troubleshooting**: Check fiber deadlocks (use `util::fb2` not `std::mutex`), timeout issues (`--test_timeout`), ASAN reports
445
446---
447
448## Validation Checklist
449
450Before claiming a task is complete, verify:
451
452### Code Quality
453
454- [ ] Code compiles without errors: `cd build-dbg && ninja dragonfly`
455- [ ] Code compiles without warnings (CI uses `-Werror`)
456- [ ] Code follows Google C++ Style Guide (run `clang-format`)
457- [ ] No new ASAN/UBSAN violations
458
459### Testing
460
461- [ ] All existing C++ unit tests pass: `ctest -V -L DFLY`
462- [ ] New feature has corresponding test coverage
463- [ ] Tests pass in both Debug and Release builds
464- [ ] Tests pass with ASAN/UBSAN enabled (if applicable)
465- [ ] **DO NOT run codeql_checker** - it's slow and unnecessary for development testing
466
467### Pre-commit & Style
468
469- [ ] Pre-commit hooks installed: `pre-commit install`
470- [ ] Code formatted with clang-format (C++) and black (Python)
471
472### Documentation
473
474- [ ] Public APIs have comments explaining purpose
475- [ ] Complex algorithms have explanatory comments
476- [ ] README or docs updated if behavior changes
477- [ ] No commented-out code left in final commit
478
479### Performance
480
481- [ ] No obvious performance regressions (run benchmarks if needed)
482- [ ] No unnecessary allocations in hot paths
483- [ ] Lock-free data structures used where appropriate
484
dragonflydb/dragonfly · .github/copilot-instructions.md
@@ +1 @@
1---
2description: 'Code review guidelines for GitHub copilot in this project'
3applyTo: '**'
4excludeAgent: ["coding-agent"]
5---
6
7# Code Review Instructions
8
9Keep reviews high-signal and minimal. Only comment on real bugs with high confidence.
10
11## Comment Only When
12- The issue is a correctness, security, concurrency, or architecture problem.
13- The impact is clear and non-trivial.
14- You can point to concrete evidence in the diff (not speculation).
15
16## Avoid
17- Style, formatting, naming, or minor performance nits.
18- Optional refactors or “nice to have” suggestions.
19- Praise, restating the code, or long explanations.
20- Duplicate comments for the same root cause.
21
22## Review Style
23- Be terse: 1-2 sentences per issue.
24- Include file and line references when possible.
25- If no issues are found, say “No issues found.”
26- Provide concrete suggestions for fixes when possible, or examples to illustrate the problem.
27
@@ −1 +1 @@
1−# Dragonfly Development Guide
2−
3−> **Essential reference for working with the Dragonfly codebase**
4−> Architecture, build system, testing infrastructure, and development workflows.
5−
61 ---
7−
8−## Table of Contents
9−
10−1. [Critical Workflow Rules](#critical-workflow-rules)
11−2. [Quick Command Reference](#quick-command-reference)
12−3. [Project Overview](#project-overview)
13−4. [Repository Structure](#repository-structure)
14−5. [Build Instructions](#build-instructions)
15−6. [Testing](#testing)
16−7. [CI/CD Pipeline](#cicd-pipeline)
17−8. [Code Style & Pre-commit Hooks](#code-style--pre-commit-hooks)
18−9. [Third-Party Dependencies](#third-party-dependencies)
19−10. [Platform Support](#platform-support)
20−11. [CMake Build Options](#cmake-build-options)
21−12. [Key Files Reference](#key-files-reference)
22−13. [Common Pitfalls](#common-pitfalls)
23−14. [Debugging Tips](#debugging-tips)
24−15. [Validation Checklist](#validation-checklist)
25−
2+description: 'Code review guidelines for GitHub copilot in this project'
3+applyTo: '**'
4+excludeAgent: ["coding-agent"]
265 ---
276
28−## Critical Workflow Rules
7+# Code Review Instructions
298
30−**MANDATORY - Always Follow This Order:**
9+Keep reviews high-signal and minimal. Only comment on real bugs with high confidence.
3110
32−1. ✅ **Read Before Edit** - Always read files before modifying
33−2. ✅ **Use Correct Build Commands** - See [Quick Command Reference](#quick-command-reference) below
34−3. ✅ **Test After Changes** - Build and run a relevant unit test -
35− `ninja <unit_test> && ./unit_test`
36−4. ✅ **Format Code** - `pre-commit run --files <files>`
37−5. ✅ **Follow Architecture** - See [Architecture Patterns](#architecture-patterns) below
38−6. ✅ **Never Push to Main** - Always create a feature branch and open a PR. Never run `git push origin main`.
11+## Comment Only When
12+- The issue is a correctness, security, concurrency, or architecture problem.
13+- The impact is clear and non-trivial.
14+- You can point to concrete evidence in the diff (not speculation).
3915
40−### Pull Request Guidelines
16+## Avoid
17+- Style, formatting, naming, or minor performance nits.
18+- Optional refactors or “nice to have” suggestions.
19+- Praise, restating the code, or long explanations.
20+- Duplicate comments for the same root cause.
4121
42−**Conciseness is Key**: PR descriptions should be short, focused, and easy to scan.
43−- **Title**: Imperative, descriptive (e.g., "Fix fiber stack overflow in test_reply_guard_oom")
44−- **Summary**: 1-2 sentences explaining *what* changed and *why*
45−- **Changes**: Bullet points for key changes
46−- **Fixes**: Link issues (e.g., "Fixes #123")
47−- **Commit messages**: Keep every line (subject and body) <= 100 characters; wrap long descriptions
48−
49−---
50−
51−## Quick Command Reference
52−
53−**CRITICAL: Read the full sections below for context. These are shortcuts only.**
54−
55−### Building (see [Build Instructions](#build-instructions) for details)
56−
57−```bash
58−# Debug build (for development)
59−./helio/blaze.sh -DWITH_AWS=OFF -DWITH_GCP=OFF
60−cd build-dbg && ninja dragonfly # Build main binary
61−cd build-dbg && ninja generic_family_test # Build specific test
62−
63−# Release build for local benchmarking
64−./helio/blaze.sh -release -DWITH_AWS=OFF -DWITH_GCP=OFF
65−cd build-opt && ninja dragonfly
66−```
67−
68−### Testing (see [Testing](#testing) for details)
69−
70−```bash
71−# C++ Unit Tests
72−cd build-dbg
73−ctest -V -L DFLY # Run all tests
74−./generic_family_test # Run specific test binary
75−./generic_family_test --gtest_filter="Set.*" # Run specific test case
76−```
77−
78−### Code Formatting
79−
80−```bash
81−# Setup (once)
82−pipx install pre-commit clang-format black
83−pre-commit install
84−
85−# Format code
86−pre-commit run --files <files> # Format specific files
87−pre-commit run --all-files # Format all files
88−```
89−
90−### Common Operations
91−
92−```bash
93−# Check git status
94−git status
95−
96−# Check current branch
97−git branch
98−
99−# View recent commits
100−git log --oneline -10
101−```
102−
103−---
104−
105−## Architecture Patterns
106−
107−**Code Style**: [.clang-format](.clang-format) - snake_case vars, PascalCase functions, kPascalCase constants
108−
109−**DO ✅**:
110−- Fiber-aware: `util::fb2::Mutex`, `util::fb2::Fiber` → [helio/util/fibers/](helio/util/fibers/)
111−- Per-shard ops (no global state) → [docs/df-share-nothing.md](docs/df-share-nothing.md)
112−- Command pattern → [src/server/set_family.cc](src/server/set_family.cc)
113−- Error handling: `OpStatus` → [src/server/common.h](src/server/common.h)
114−- Test patterns → [tests/dragonfly/conftest.py](tests/dragonfly/conftest.py)
115−
116−**DON'T ❌**:
117−- `std::thread`, `std::mutex` (deadlocks!)
118−- Global mutable state
119−- Edit without reading
120−- Skip tests
121−- Use `std::regex` in fiber/server paths (recursive implementation can overflow small fiber stacks)
122−- Use `./tools/docker/build.sh` for local development (use `ninja` instead)
123−- Use `make` for incremental builds (use `ninja` instead)
124−
125−---
126−
127−## Project Overview
128−
129−**Dragonfly** is a high-performance, Redis and Memcached compatible in-memory data store written in C++20. It delivers significantly higher throughput than traditional single-threaded Redis implementations through innovative architectural choices.
130−
131−### Key Characteristics
132−
133−- **Language**: C++20 (Google C++ Style Guide 2020 version)
134−- **Architecture**: Shared-nothing multi-threaded design (via `helio` library)
135−- **Performance**: Uses io_uring (Linux 5.11+) for high-performance async I/O, with epoll fallback
136−- **Threading Model**: Fiber-based cooperative multitasking with lock-free data structures
137−- **Build System**: CMake + Ninja via `helio/blaze.sh` wrapper script
138−- **Target Platform**: Linux (kernel 5.11+ recommended), FreeBSD support available
139−- **Protocols**: Redis RESP2/RESP3, Memcached binary protocol
140−- **Compatibility**: Drop-in replacement for Redis API coverage
141−
142−### Architectural Highlights
143−
144−**For detailed architecture documentation, see [docs/df-share-nothing.md](docs/df-share-nothing.md)**
145−
146−1. **Shared-Nothing Design**: Each thread operates independently with its own data structures, minimizing lock contention
147−2. **Helio Framework**: Custom I/O and threading library built on io_uring/epoll with fiber support
148−3. **DashTable**: Novel hash table implementation optimized for multi-core systems - see [docs/dashtable.md](docs/dashtable.md)
149−4. **Transaction Model**: Non-blocking optimistic transactions - see [docs/transaction.md](docs/transaction.md)
150−5. **Tiering Support**: Optional disk-backed storage for large datasets
151−6. **Search Module**: Full-text search capabilities (when enabled with WITH_SEARCH)
152−
153−---
154−
155−## Repository Structure
156−
157−```
158−dragonfly/
159−├── src/ # Main C++ source code
160−│ ├── server/ # Core server implementation
161−│ │ ├── dfly_main.cc # Main entry point
162−│ │ ├── main_service.cc # Service lifecycle & command routing
163−│ │ ├── db_slice.cc # Per-thread database shard
164−│ │ ├── engine_shard_set.cc # Shard management
165−│ │ ├── cluster/ # Cluster mode implementation
166−│ │ ├── journal/ # Replication journal
167−│ │ ├── tiering/ # Tiered storage
168−│ │ ├── search/ # Search module
169−│ │ └── acl/ # Access control lists
170−│ ├── core/ # Core data structures
171−│ │ ├── dash.h # DashTable hash table
172−│ │ ├── dense_set.h # Compact set implementation
173−│ │ ├── string_map.h # Optimized string-keyed maps
174−│ │ ├── search/ # Search core algorithms
175−│ │ └── json/ # JSON support
176−│ ├── facade/ # Network & command handling
177−│ │ ├── dragonfly_connection.cc # Connection management
178−│ │ ├── redis_parser.cc # RESP protocol parser
179−│ │ └── memcache_parser.cc # Memcached protocol
180−│ └── redis/ # Redis-specific implementations
181−│ └── lua/ # Lua scripting support
182−│
183−├── helio/ # Git submodule: I/O and threading library
184−│ │ # ** DO NOT EDIT unless contributing to helio **
185−│ ├── util/ # Utilities: fibers, I/O, synchronization
186−│ ├── io/ # io_uring & epoll abstraction
187−│ └── blaze.sh # Build configuration wrapper
188−│
189−├── tests/ # Test suite
190−│ ├── dragonfly/ # Python pytest integration/regression tests
191−│ │ ├── conftest.py # Pytest fixtures & configuration
192−│ │ ├── requirements.txt # Python test dependencies
193−│ │ └── *.py # Test files
194−│ └── pytest.ini # Pytest configuration & markers
195−│
196−├── docs/ # Documentation
197−│ ├── build-from-source.md # Build instructions
198−│ ├── dashtable.md # DashTable internals
199−│ ├── transaction.md # Transaction model
200−│ ├── df-share-nothing.md # Shared-nothing architecture
201−│ └── differences.md # Differences from Redis
202−│
203−├── contrib/ # Utilities
204−│ ├── docker/ # Docker configurations
205−│ └── charts/dragonfly/ # Helm chart for Kubernetes
206−│
207−├── tools/ # Benchmarking & utility tools
208−│ └── packaging/ # Packaging scripts
209−│
210−├── CMakeLists.txt # Root CMake configuration
211−├── .clang-format # C++ formatting rules (clang-format v14.0.6)
212−├── .pre-commit-config.yaml # Pre-commit hooks configuration
213−├── pyproject.toml # Python formatting (Black, 100 chars)
214−└── CONTRIBUTING.md # Contribution guidelines
215−```
216−
217−### Critical Paths to Remember
218−
219−- **Main entry**: `src/server/dfly_main.cc`
220−- **Command dispatch**: `src/server/main_service.cc`
221−- **Data storage**: `src/server/db_slice.cc`
222−- **Networking**: `src/facade/dragonfly_connection.cc`
223−- **Helio library**: `helio/` (I/O and threading library)
224−
225−---
226−
227−## Build Instructions
228−
229−**For complete build instructions, see [docs/build-from-source.md](docs/build-from-source.md)**
230−
231−### Quick Start
232−
233−**Debug build** (for development):
234−```bash
235−./helio/blaze.sh
236−cd build-dbg && ninja dragonfly
237−./dragonfly --alsologtostderr
238−```
239−
240−**Release build** (for production/benchmarking):
241−```bash
242−./helio/blaze.sh -release
243−cd build-opt && ninja dragonfly
244−```
245−
246−**Production release build** (static linking, optimized):
247−```bash
248−make release # Configure + build
249−make package # Create release packages with debug symbols
250−```
251−
252−The [Makefile](Makefile) builds production releases with:
253−- Static linking: libstdc++, libgcc, Boost, OpenSSL
254−- Architecture optimizations (x86_64: `-march=core2 -msse4.1 -mtune=skylake`)
255−- Debug symbols (compressed)
256−- Output: `build-release/dragonfly-{arch}.tar.gz`
257−
258−**Common build options**:
259−- See [docs/build-from-source.md](docs/build-from-source.md) for all options
260−
261−---
262−
263−## Testing
264−
265−**For complete testing documentation, see [tests/README.md](tests/README.md)**
266−
267−### Quick Reference
268−
269−**C++ Unit Tests**:
270−```bash
271−cd build-dbg
272−ctest -V -L DFLY # Run all tests
273−./generic_family_test # Run specific test binary
274−./generic_family_test --gtest_filter="Set.*" # Run specific test case
275−```
276−
277−**Python Integration Tests (pytest)**:
278−```bash
279−# Run from the repo root. The binary path defaults to build-dbg/dragonfly.
280−# Override with the DRAGONFLY_PATH env var:
281−DRAGONFLY_PATH=build-dbg/dragonfly python3 -m pytest tests/dragonfly/pymemcached_test.py -xvs
282−
283−# Run a single test:
284−python3 -m pytest tests/dragonfly/pymemcached_test.py::TestMemcached::test_basic -xvs
285−```
286−
287−- `DRAGONFLY_PATH` — sets the path to the Dragonfly binary the test harness starts. Defaults to `build-dbg/dragonfly` relative to the `tests/dragonfly/` directory.
288−- `--df` — passes **extra flags to the Dragonfly process** (not the binary path). For example: `--df logtostdout --df "vmodule=*=1"`.
289−
290−---
291−
292−## CI/CD Pipeline
293−
294−**For complete CI configuration, see [.github/workflows/ci.yml](.github/workflows/ci.yml)**
295−
296−The CI workflow runs on all PRs and includes:
297−- **Pre-commit checks**: clang-format, black formatters
298−- **Build matrix**: Multiple OS/compiler/sanitizer combinations (Ubuntu 20/24, Alpine, GCC/Clang, ASAN/UBSAN)
299−- **Test execution**: C++ unit tests, Python integration tests, cluster mode tests
300−- **Additional validations**: Helm charts, Docker image builds
301−
302−---
303−
304−## Code Style & Pre-commit Hooks
305−
306−**For complete contribution guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md)**
307−
308−**Code style configuration files**:
309−- **C++**: [.clang-format](.clang-format) - Google C++ Style Guide (2020), clang-format v14.0.6, 100 char limit
310−- **Python**: [pyproject.toml](pyproject.toml) - Black formatter, 100 char limit, PEP 8 compliant
311−- **Pre-commit hooks**: [.pre-commit-config.yaml](.pre-commit-config.yaml) - Automated formatting checks
312−
313−**Quick setup**:
314−```bash
315−pipx install pre-commit clang-format black
316−pre-commit install
317−pre-commit run --all-files # Run all formatters
318−```
319−
320−---
321−
322−## Third-Party Dependencies
323−
324−**Key Libraries**: Abseil (strings/flags), Boost 1.71+ (context/intrusive), mimalloc (allocator), jsoncons (JSON), OpenSSL (TLS), libunwind (traces)
325−
326−**Build artifacts**: `build-dbg/third_party/` - DO NOT edit
327−
328−**For complete dependency info, see [docs/build-from-source.md](docs/build-from-source.md)**
329−
330−---
331−
332−## Platform Support
333−
334−**Linux**: Primary platform. Kernel 5.11+ (io_uring), 5.1+ (basic), < 5.1 (epoll fallback)
335−- Check: `uname -r`
336−- Force epoll: `--proactor_type=epoll`
337−- Docker: `--security-opt seccomp=unconfined`
338−
339−**FreeBSD**: Supported (kqueue backend)
340−
341−**macOS**: Not supported for production (use Docker/Linux)
342−
343−**For complete platform info, see [docs/build-from-source.md](docs/build-from-source.md)**
344−
345−---
346−
347−## CMake Build Options
348−
349−**For complete list of build options, see [docs/build-from-source.md](docs/build-from-source.md)**
350−
351−### Common Options
352−
353−Pass options to `helio/blaze.sh` with `-D` prefix:
354−
355−```bash
356−./helio/blaze.sh -DWITH_SEARCH=OFF -DWITH_AWS=ON
357−```
358−
359−**Most useful options**:
360−- `WITH_ASAN=ON` / `WITH_USAN=ON` - Enable sanitizers for debugging
361−- `WITH_SEARCH=OFF` - Disable search module for faster builds
362−- `WITH_AWS=OFF` / `WITH_GCP=OFF` - Disable cloud libraries
363−- `WITH_TIERING=OFF` - Disable disk storage
364−- `USE_MOLD=ON` - Faster linking with LTO (production builds)
365−
366−**Quick configurations**:
367−```bash
368−# Minimal build (fast compilation)
369−./helio/blaze.sh -DWITH_GPERF=OFF -DWITH_AWS=OFF -DWITH_GCP=OFF -DWITH_TIERING=OFF -DWITH_SEARCH=OFF
370−
371−# Full-featured (all options ON by default)
372−./helio/blaze.sh
373−
374−# Production optimized
375−./helio/blaze.sh -release -DUSE_MOLD=ON
376−```
377−
378−---
379−
380−## Key Files Reference
381−
382−Quick reference to the most important files in the codebase.
383−
384−| Purpose | File Path |
385−|---------|-----------|
386−| **Entry Points & Core** | |
387−| Main entry point | `src/server/dfly_main.cc` |
388−| Server lifecycle & command routing | `src/server/main_service.cc` |
389−| Per-thread database shard | `src/server/db_slice.cc` |
390−| Shard management | `src/server/engine_shard_set.cc` |
391−| **Data Structures** | |
392−| DashTable hash table | `src/core/dash.h` |
393−| Dense set implementation | `src/core/dense_set.h` |
394−| String map | `src/core/string_map.h` |
395−| **Networking** | |
396−| Connection handling | `src/facade/dragonfly_connection.cc` |
397−| Redis protocol parser | `src/facade/redis_parser.cc` |
398−| Memcached protocol parser | `src/facade/memcache_parser.cc` |
399−| **Build System** | |
400−| Root CMake config | `CMakeLists.txt` |
401−| Build script wrapper | `helio/blaze.sh` |
402−| Server CMake config | `src/server/CMakeLists.txt` |
403−| **CI/CD** | |
404−| Main CI workflow | `.github/workflows/ci.yml` |
405−| Pre-commit config | `.pre-commit-config.yaml` |
406−| **Code Style** | |
407−| C++ formatting | `.clang-format` |
408−| Python formatting | `pyproject.toml` |
409−| **Testing** | |
410−| Pytest configuration | `tests/pytest.ini` |
411−| Pytest fixtures | `tests/dragonfly/conftest.py` |
412−| Test requirements | `tests/dragonfly/requirements.txt` |
413−| **Documentation** | |
414−| Build instructions | `docs/build-from-source.md` |
415−| Architecture overview | `docs/df-share-nothing.md` |
416−| DashTable internals | `docs/dashtable.md` |
417−| Transaction model | `docs/transaction.md` |
418−| **Configuration** | |
419−| Contributing guide | `CONTRIBUTING.md` |
420−| CLA agreement | `CLA.txt` |
421−
422−---
423−
424−## Common Pitfalls
425−
426−1. **Pre-commit not installed**: `pipx install pre-commit clang-format black && pre-commit install`
427−2. **Wrong binary**: Debug: `build-dbg/dragonfly`, Release: `build-opt/dragonfly`
428−3. **Wrong build command**: Use `cd build-dbg && ninja <target>`, NOT `./tools/docker/build.sh`
429−4. **Test timeouts**: `timeout 20m ctest -V -L DFLY`
430−5. **ASAN leaks**: Check CI, suppress in `helio/util/asan_suppressions.txt`
431−6. **Helio modifications**: DON'T edit `helio/` (it's a git submodule - changes go upstream)
432−7. **CodeQL checks**: DON'T run codeql_checker when testing changes - it's slow and unnecessary for development
433−
434−---
435−
436−## Debugging Tips
437−
438−**Logging**: `--alsologtostderr --v=1 --vmodule=module=2`
439−
440−**ASAN**: `ASAN_OPTIONS=detect_leaks=1:symbolize=1`, suppressions: `helio/util/asan_suppressions.txt`
441−
442−**CI reproduction**: See [.github/workflows/ci.yml](.github/workflows/ci.yml)
443−
444−**Troubleshooting**: Check fiber deadlocks (use `util::fb2` not `std::mutex`), timeout issues (`--test_timeout`), ASAN reports
445−
446−---
447−
448−## Validation Checklist
449−
450−Before claiming a task is complete, verify:
451−
452−### Code Quality
453−
454−- [ ] Code compiles without errors: `cd build-dbg && ninja dragonfly`
455−- [ ] Code compiles without warnings (CI uses `-Werror`)
456−- [ ] Code follows Google C++ Style Guide (run `clang-format`)
457−- [ ] No new ASAN/UBSAN violations
458−
459−### Testing
460−
461−- [ ] All existing C++ unit tests pass: `ctest -V -L DFLY`
462−- [ ] New feature has corresponding test coverage
463−- [ ] Tests pass in both Debug and Release builds
464−- [ ] Tests pass with ASAN/UBSAN enabled (if applicable)
465−- [ ] **DO NOT run codeql_checker** - it's slow and unnecessary for development testing
466−
467−### Pre-commit & Style
468−
469−- [ ] Pre-commit hooks installed: `pre-commit install`
470−- [ ] Code formatted with clang-format (C++) and black (Python)
471−
472−### Documentation
473−
474−- [ ] Public APIs have comments explaining purpose
475−- [ ] Complex algorithms have explanatory comments
476−- [ ] README or docs updated if behavior changes
477−- [ ] No commented-out code left in final commit
478−
479−### Performance
480−
481−- [ ] No obvious performance regressions (run benchmarks if needed)
482−- [ ] No unnecessary allocations in hot paths
483−- [ ] Lock-free data structures used where appropriate
22+## Review Style
23+- Be terse: 1-2 sentences per issue.
24+- Include file and line references when possible.
25+- If no issues are found, say “No issues found.”
26+- Provide concrete suggestions for fixes when possible, or examples to illustrate the problem.
48427
