Two files, one repository
elastic/ml-cpp ships 3 formats across 6 indexed files. The question worth asking is whether the second one says anything the first does not.
| Dimension | Shared | Only in A | Only in B | Overlap |
|---|---|---|---|---|
| Sections | 0 | 23 | 7 | 0% |
| Commands | 1 | 16 | 0 | 6% |
| Section tags | 5 | 3 | 0 | 63% |
What each file covers
Sections
0 shared · 23 only in A · 7 only in B- − Elasticsearch Machine Learning C++
- − Toolchain
- − Build & Run Commands
- − Project Structure
- − CMake Helper Functions
- − Adding a Shared Library
- − Adding an Executable
- − Adding a Test Executable
- − Registering Tests with the Build
- − Platform-Specific Sources
- − Testing
- − Running Tests
- − Precommit (format + test)
- − Writing Tests
- − Formatting & Style
- − Naming Conventions
- − Code Conventions
- − File Layout
- − Documentation
- − License Headers
- − CI
- − Pull Requests
- − Best Practices for Automation Agents
- + ml-cpp AI Context
- + Build System
- + Build Acceleration Options
- + Test Parallelism
- + Coding Conventions
- + CI (Buildkite)
- + CI Analytics
Commands
1 shared · 16 only in A · 0 only in B- − cmake -B cmake-build-relwithdebinfo
- − cmake --build cmake-build-relwithdebinfo -j$(nproc)
- − ./gradlew :compile
- − cmake --build cmake-build-relwithdebinfo -t test
- − cmake --build cmake-build-relwithdebinfo -t test_core
- − cmake --build cmake-build-relwithdebinfo -t test_model
- − cmake --build cmake-build-relwithdebinfo -j8 -t test_individually
- − cmake --build cmake-build-relwithdebinfo -j8 -t test_api_individually
- − cmake --build cmake-build-relwithdebinfo -t test_all_parallel
- − cmake --build cmake-build-relwithdebinfo -j8 -t precommit
- − cmake --build cmake-build-relwithdebinfo -t format
- − ./gradlew
- − cmake/<os>-<arch>.cmake
- − cmake/functions.cmake
- − ./gradlew precommit
- − ./gradlew format
- cmake/
Section tags
5 shared · 3 only in A · 0 only in B- − lint-format
- − architecture
- − docs
- build
- test
- code-style
- git-pr
- do-not
Line diff
elastic/ml-cpp · AGENTS.md
@@ −1 @@
1# Elasticsearch Machine Learning C++
2
3## Toolchain
4- **Language**: C++20 (`CMAKE_CXX_STANDARD 20`).
5- **Build system**: CMake (primary) or Gradle wrapper (`./gradlew`).
6- **Compilers**: GCC 13.3.0 on Linux (built from source, installed to `/usr/local/gcc133/`), Xcode Clang on macOS (Xcode 15.2+ for Ventura/Sonoma), Visual Studio 2022 Professional (MSVC) on Windows.
7- **Key dependencies**: Boost 1.86.0 (dynamic linking, includes Boost.Json for JSON handling), PyTorch 2.7.1 (libtorch), libxml2.
8- **Header-only libraries**: Eigen and valijson are header-only and managed by the `3rd_party/` CMake system (pulled automatically during configuration).
9- **Platforms**: Linux x86_64/aarch64, macOS aarch64, Windows x86_64.
10- **Toolchain files**: Auto-selected from `cmake/<os>-<arch>.cmake` or set via `CMAKE_TOOLCHAIN_FILE`.
11
12## Build & Run Commands
13
14Configure and build (default `RelWithDebInfo`):
15```
16cmake -B cmake-build-relwithdebinfo
17cmake --build cmake-build-relwithdebinfo -j$(nproc)
18```
19
20Or via Gradle:
21```
22./gradlew :compile
23```
24
25Set `ML_DEBUG=1` to switch to a Debug build. Compiler caching (sccache/ccache) is auto-detected.
26
27Refer to `CONTRIBUTING.md` and the `build-setup/` directory for full platform-specific setup instructions.
28
29## Project Structure
30
31```
32bin/ # Application executables
33 autodetect/ # Anomaly detection
34 categorize/ # Log categorization
35 controller/ # Process lifecycle controller
36 data_frame_analyzer/ # Data frame analytics (classification, regression)
37 normalize/ # Anomaly score normalization
38 pytorch_inference/ # PyTorch model inference
39lib/ # Shared libraries
40 api/ # JSON/REST API layer
41 core/ # Platform abstractions, I/O, logging, compression
42 maths/ # Mathematical and statistical algorithms
43 analytics/ # Boosted tree, data frame analytics
44 common/ # Bayesian optimisation, distributions, time series
45 time_series/ # Time series decomposition, forecasting
46 model/ # Anomaly detection models
47 seccomp/ # Seccomp/sandbox filters
48 test/ # Shared test utilities (CBoostTestXmlOutput, etc.)
49 ver/ # Version information
50include/ # Public headers (mirrors lib/ structure)
513rd_party/ # Header-only third-party libraries (Eigen, valijson), licenses
52cmake/ # CMake toolchain files, helper functions, test runners
53build-setup/ # Platform-specific build environment instructions
54.buildkite/ # CI pipeline definitions (Buildkite)
55.ci/ # Packer scripts for building Orka macOS CI VMs
56.github/workflows/ # GitHub Actions (automatic backport)
57dev-tools/ # Developer scripts (clang-format, benchmarks)
58```
59
60Libraries must not have circular dependencies. The dependency order is roughly:
61`core` -> `maths` -> `model` -> `api` -> `bin/*`.
62
63## CMake Helper Functions
64
65The build uses custom CMake functions defined in `cmake/functions.cmake`. Use these instead of raw `add_library`/`add_executable` — they handle platform-specific sources, linking, installation, and Windows resource generation automatically.
66
67### Adding a Shared Library
68
69Set `ML_LINK_LIBRARIES` then call `ml_add_library`:
70
71```cmake
72project("ML MyLib")
73
74set(ML_LINK_LIBRARIES
75 ${Boost_LIBRARIES}
76 MlCore
77 )
78
79ml_add_library(MlMyLib SHARED
80 CMyClass.cc
81 CMyOtherClass.cc
82 )
83```
84
85Libraries are named with the `Ml` prefix (e.g. `MlCore`, `MlModel`). The function handles shared library versioning, RPATH, and installation. Use `SHARED` for distributed libraries or `STATIC` for internal-only ones.
86
87For libraries that should not be installed/distributed (e.g. internal helpers), use `ml_add_non_distributed_library` instead.
88
89### Adding an Executable
90
91Set `ML_LINK_LIBRARIES` then call `ml_add_executable`. A `Main.cc` file is included automatically — do not list it in the sources:
92
93```cmake
94project("ML MyApp")
95
96set(ML_LINK_LIBRARIES
97 ${Boost_LIBRARIES}
98 MlCore
99 MlApi
100 MlVer
101 )
102
103ml_add_executable(myapp
104 CCmdLineParser.cc
105 )
106```
107
108The function creates a companion OBJECT library (`MlMyApp`) from the listed sources, which test executables can link against. The executable itself always builds from `Main.cc` plus those objects.
109
110For executables not intended for distribution (dev tools, benchmarks), use `ml_add_non_distributed_executable`.
111
112### Adding a Test Executable
113
114Test executables live in `unittest/` subdirectories. Set `ML_LINK_LIBRARIES` (including `${Boost_LIBRARIES_WITH_UNIT_TEST}` and `MlTest`), then call `ml_add_test_executable`:
115
116```cmake
117project("ML MyLib unit tests")
118
119set(SRCS
120 CMyClassTest.cc
121 CMyOtherClassTest.cc
122 Main.cc
123 )
124
125set(ML_LINK_LIBRARIES
126 ${Boost_LIBRARIES_WITH_UNIT_TEST}
127 MlCore
128 MlMyLib
129 MlTest
130 )
131
132ml_add_test_executable(mylib ${SRCS})
133```
134
135The `_target` argument (e.g. `mylib`) is used to derive the test executable name (`ml_test_mylib`) and the CMake targets `test_mylib` and `test_mylib_individually`.
136
137### Registering Tests with the Build
138
139After creating the test executable, register it in `test/CMakeLists.txt` by adding an `ml_add_test` call alongside the existing entries:
140
141```cmake
142ml_add_test(lib/core/unittest core)
143ml_add_test(lib/maths/common/unittest maths_common)
144ml_add_test(lib/maths/time_series/unittest maths_time_series)
145ml_add_test(lib/maths/analytics/unittest maths_analytics)
146ml_add_test(lib/model/unittest model)
147ml_add_test(lib/api/unittest api)
148ml_add_test(lib/ver/unittest ver)
149ml_add_test(lib/seccomp/unittest seccomp)
150ml_add_test(bin/controller/unittest controller)
151ml_add_test(bin/pytorch_inference/unittest pytorch_inference)
152ml_add_test(lib/mylib/unittest mylib) # <-- new entry
153```
154
155The first argument is the relative path to the unittest directory; the second is the target name matching `ml_add_test_executable`. Note how nested libraries use underscores in the target name (e.g. `lib/maths/common/unittest` -> `maths_common`).
156
157### Platform-Specific Sources
158
159If a source file has a platform-specific variant (e.g. `CMyClass_Linux.cc`, `CMyClass_Darwin.cc`), the `ml_generate_platform_sources` function (called internally by all `ml_add_*` functions) will automatically substitute the platform-specific file at build time. Just list the base filename (`CMyClass.cc`) in your sources.
160
161## Testing
162
163Tests use the **Boost.Test** framework. Each library and application has a `unittest/` subdirectory containing test files and a `Main.cc` entry point.
164
165### Running Tests
166
167Run all tests:
168```
169cmake --build cmake-build-relwithdebinfo -t test
170```
171
172Run tests for a specific library:
173```
174cmake --build cmake-build-relwithdebinfo -t test_core
175cmake --build cmake-build-relwithdebinfo -t test_model
176```
177
178Run specific test cases (wildcards supported):
179```
180TESTS="*/testPersist" cmake --build cmake-build-relwithdebinfo -t test_model
181```
182
183Run tests individually in separate processes (better isolation, per-suite parallelism):
184```
185cmake --build cmake-build-relwithdebinfo -j8 -t test_individually
186cmake --build cmake-build-relwithdebinfo -j8 -t test_api_individually
187```
188
189Run all test cases from all suites in a single CTest invocation (optimal cross-suite parallelism):
190```
191cmake --build cmake-build-relwithdebinfo -t test_all_parallel
192```
193
194Pass extra flags to the Boost.Test runner:
195```
196TEST_FLAGS="--random" cmake --build cmake-build-relwithdebinfo -t test
197```
198
199### Precommit (format + test)
200
201```
202cmake --build cmake-build-relwithdebinfo -j8 -t precommit
203```
204Or: `./gradlew precommit`
205
206### Writing Tests
207
208- Test files are named `CClassNameTest.cc` and placed in `lib/<module>/unittest/` or `bin/<module>/unittest/`.
209- Each test file uses `BOOST_AUTO_TEST_SUITE(CClassNameTest)` / `BOOST_AUTO_TEST_CASE(testMethodName)`.
210- Use real classes over mocks wherever possible. Tests should reflect real-world usage.
211- Every class should have a corresponding test suite; every public method should have a test.
212- Test cases must be completely independent from one another — they may be run in parallel across separate processes, so they must not depend on execution order or share mutable state.
213
214## Formatting & Style
215
216Code is formatted with `clang-format` (LLVM-based style, 4-space indent). Run before committing:
217```
218cmake --build cmake-build-relwithdebinfo -t format
219```
220Or: `./gradlew format`
221
222The CI pipeline enforces formatting via the `check-style` step; PRs that fail formatting will not pass CI.
223
224The full coding standard is in `STYLEGUIDE.md`. Key points:
225
226### Naming Conventions
227- Classes: `CClassName`, Structs: `SStructName`, Enums: `EEnumName`
228- Member variables: `m_ClassMember`, `s_StructMember`
229- Static members: `ms_ClassStatic`
230- Methods: `methodName` (camelCase)
231- Type aliases: `TTypeName` (e.g. `using TDoubleVec = std::vector<double>`)
232- Constants: `CONSTANT_NAME`
233- Non-boolean accessors: `clientId` (not `getClientId`)
234- Boolean accessors: `isComplete` (not `complete`)
235- Files: `CClassName.cc` / `CClassName.h`
236
237### Code Conventions
238- Use `nullptr`, never `0` or `NULL`.
239- No exceptions — use return codes for error handling. Catch third-party exceptions at the smallest scope.
240- No `assert()`. No C-style casts. No macros unless unavoidable.
241- Prefer smart pointers over raw pointers; prefer references over pointers.
242- Scope member function calls with `this->`.
243- Use `auto` when the type is obvious; avoid it when the type is unclear.
244- Prefer `emplace_back` over `push_back`, range-based for loops, and uniform initializers.
245- `override` must be used consistently; `virtual` must not appear alongside `override`.
246
247### File Layout
248- Implementation files (`.cc`): own header first, then other ML headers, third-party headers, standard library headers.
249- Group includes by library with blank lines between groups (clang-format will sort within groups).
250- Use unnamed namespaces in `.cc` files for file-local helpers, not private class members.
251- Forward-declare classes in headers rather than including their headers.
252
253### Documentation
254- Doxygen comments (exclamation mark style: `//!`) are required for all header files and public/protected methods.
255- Implementation files use regular C++ comments, not Doxygen.
256- Focus comments on the "why", not the "what".
257
258## License Headers
259
260All source files must include the Elastic License 2.0 header. Copy from `copyright_code_header.txt` or any existing source file.
261
262## CI
263
264CI runs on **Buildkite** (`ml-cpp-pr-builds`). The pipeline builds and tests on all platforms (Linux x86_64, Linux aarch64, macOS aarch64, Windows x86_64) in both `RelWithDebInfo` and Debug configurations. It also runs:
265- `clang-format` style validation
266- Snyk security/license scanning
267- Java integration tests against Elasticsearch
268
269Automatic backporting is handled by a GitHub Action (`.github/workflows/backport.yml`) — add version labels (e.g. `v9.3.0`) to a PR and a backport PR is created automatically when it merges.
270
271## Pull Requests
272
273- Title must be prefixed with `[ML]` (e.g. `[ML] Fix anomaly scoring edge case`).
274- Label with `:ml` (mandatory), a type label (`>bug`, `>enhancement`, `>feature`, `>refactoring`, `>test`, `>docs`), and version labels for applicable releases.
275- Squash-and-merge is the standard merge strategy; keep commits clean for review but don't squash manually.
276- Backports start after merging to `main`. Add version labels to trigger automatic backport PRs.
277
278## Best Practices for Automation Agents
279
280- Always read existing code before editing to understand patterns and conventions.
281- Never edit unrelated files; keep diffs tightly scoped.
282- Run `clang-format` before presenting any code changes.
283- Match the naming conventions exactly — the prefixes (`C`, `m_`, `ms_`, `T`, `E`) are strictly followed throughout the codebase.
284- When adding new classes, follow the existing directory and namespace structure. Production code in `lib/foo/` uses namespace `ml::foo`.
285- When adding tests, place them in the corresponding `unittest/` directory and register them in the `CMakeLists.txt`.
286- Do not introduce new third-party dependencies without discussion.
287- Do not add AI attribution trailers (e.g. `Co-Authored-By`) to commit messages.
288- Commit messages should follow the `[ML] Summary of change` format.
289- If unsure about a convention, check a nearby file for the established pattern — consistency with surrounding code is the highest priority.
290
291Stay aligned with `CONTRIBUTING.md`, `STYLEGUIDE.md`, and the `build-setup/` guides; this AGENTS file summarizes but does not replace those authoritative documents.
292
elastic/ml-cpp · CLAUDE.md
@@ +1 @@
1# ml-cpp AI Context
2
3This file provides domain knowledge for AI coding assistants working on the ml-cpp repository. It consolidates the detailed rules in `.cursor/rules/` into a single reference.
4
5For full details, see the individual files in `.cursor/rules/`:
6- `ml-cpp-build-system.mdc` — CMake, Gradle, Docker, build acceleration
7- `ml-cpp-buildkite-ci.mdc` — CI pipelines, API access, known failures
8- `ml-cpp-coding-conventions.mdc` — Naming, cross-platform, testing patterns
9- `ml-cpp-ci-analytics.mdc` — Elasticsearch, anomaly detection, AI analysis
10
11---
12
13## Build System
14
15- **CMake** is the primary build system. Toolchain files in `cmake/` per platform.
16- **Gradle** (`build.gradle`) orchestrates macOS and Windows CI builds, invoking CMake.
17- **Docker** is used for Linux builds (`dev-tools/docker/docker_entrypoint.sh`).
18- `include(CTest)` reserves the `test` target name — our monolithic test target is `ml_test`.
19- `test_individually` runs tests via CTest with parallel execution.
20
21### Build Acceleration Options
22- `-DCMAKE_UNITY_BUILD=ON` — combines source files (not effective on all libraries)
23- `-DML_PCH=ON` — precompiled headers for STL/Boost
24- sccache with GCS backend for persistent compiler caching
25- MSVC uses `/Z7` (not `/Zi`) to avoid PDB serialisation bottleneck
26
27### Test Parallelism
28- Formula: `numCpus <= 4 ? 2 : ceil(numCpus / 2)`
29- Never use wall-clock time for performance assertions — use `std::clock()` (CPU time)
30- Use process ID for unique temp file names in tests, not random numbers
31
32## Coding Conventions
33
34- Classes: `CUpperCamelCase`, Methods: `lowerCamelCase`, Members: `m_Name`
35- Types: `TUpperCamelCase`, Test files: `CClassNameTest.cc`
36- Commit messages: `[ML] Short description`
37- Use `peek() == std::char_traits<char>::eof()` for portable end-of-stream detection
38- Avoid anonymous-namespace constants with common names in libraries using unity builds
39
40## CI (Buildkite)
41
42- PR pipeline: `ml-cpp-pr-builds` (`.buildkite/pipeline.json.py`)
43- Nightly: `ml-cpp-snapshot-builds` (`.buildkite/branch.json.py`)
44- Debug: `ml-cpp-debug-build` (`.buildkite/job-build-test-all-debug.json.py`)
45- Platforms: Linux x86_64 (6 vCPU), Linux aarch64 (8 vCPU), macOS aarch64 (4 core), Windows x86_64 (16 vCPU)
46- Vault secrets via `.buildkite/hooks/post-checkout`
47- Diagnostic steps use: `if: "build.state == 'failed'"` + `soft_fail: true` + `allow_dependency_failure: true`
48
49## CI Analytics
50
51- Build timings indexed into Elasticsearch Serverless (`buildkite-build-timings`)
52- ML anomaly detection job: `build-timing-regressions` (high_mean by step_key)
53- PR regression check: compares against 30-day baseline (mean + 2σ)
54- AI failure analysis: Claude diagnoses failures, posts Buildkite annotations
55- Kibana dashboard: "ML-CPP Build Timing Overview"
56
@@ −1 +1 @@
1−# Elasticsearch Machine Learning C++
1+# ml-cpp AI Context
22
3−## Toolchain
4−- **Language**: C++20 (`CMAKE_CXX_STANDARD 20`).
5−- **Build system**: CMake (primary) or Gradle wrapper (`./gradlew`).
6−- **Compilers**: GCC 13.3.0 on Linux (built from source, installed to `/usr/local/gcc133/`), Xcode Clang on macOS (Xcode 15.2+ for Ventura/Sonoma), Visual Studio 2022 Professional (MSVC) on Windows.
7−- **Key dependencies**: Boost 1.86.0 (dynamic linking, includes Boost.Json for JSON handling), PyTorch 2.7.1 (libtorch), libxml2.
8−- **Header-only libraries**: Eigen and valijson are header-only and managed by the `3rd_party/` CMake system (pulled automatically during configuration).
9−- **Platforms**: Linux x86_64/aarch64, macOS aarch64, Windows x86_64.
10−- **Toolchain files**: Auto-selected from `cmake/<os>-<arch>.cmake` or set via `CMAKE_TOOLCHAIN_FILE`.
3+This file provides domain knowledge for AI coding assistants working on the ml-cpp repository. It consolidates the detailed rules in `.cursor/rules/` into a single reference.
114
12−## Build & Run Commands
5+For full details, see the individual files in `.cursor/rules/`:
6+- `ml-cpp-build-system.mdc` — CMake, Gradle, Docker, build acceleration
7+- `ml-cpp-buildkite-ci.mdc` — CI pipelines, API access, known failures
8+- `ml-cpp-coding-conventions.mdc` — Naming, cross-platform, testing patterns
9+- `ml-cpp-ci-analytics.mdc` — Elasticsearch, anomaly detection, AI analysis
1310
14−Configure and build (default `RelWithDebInfo`):
15−```
16−cmake -B cmake-build-relwithdebinfo
17−cmake --build cmake-build-relwithdebinfo -j$(nproc)
18−```
11+---
1912
20−Or via Gradle:
21−```
22−./gradlew :compile
23−```
13+## Build System
2414
25−Set `ML_DEBUG=1` to switch to a Debug build. Compiler caching (sccache/ccache) is auto-detected.
15+- **CMake** is the primary build system. Toolchain files in `cmake/` per platform.
16+- **Gradle** (`build.gradle`) orchestrates macOS and Windows CI builds, invoking CMake.
17+- **Docker** is used for Linux builds (`dev-tools/docker/docker_entrypoint.sh`).
18+- `include(CTest)` reserves the `test` target name — our monolithic test target is `ml_test`.
19+- `test_individually` runs tests via CTest with parallel execution.
2620
27−Refer to `CONTRIBUTING.md` and the `build-setup/` directory for full platform-specific setup instructions.
21+### Build Acceleration Options
22+- `-DCMAKE_UNITY_BUILD=ON` — combines source files (not effective on all libraries)
23+- `-DML_PCH=ON` — precompiled headers for STL/Boost
24+- sccache with GCS backend for persistent compiler caching
25+- MSVC uses `/Z7` (not `/Zi`) to avoid PDB serialisation bottleneck
2826
29−## Project Structure
27+### Test Parallelism
28+- Formula: `numCpus <= 4 ? 2 : ceil(numCpus / 2)`
29+- Never use wall-clock time for performance assertions — use `std::clock()` (CPU time)
30+- Use process ID for unique temp file names in tests, not random numbers
3031
31−```
32−bin/ # Application executables
33− autodetect/ # Anomaly detection
34− categorize/ # Log categorization
35− controller/ # Process lifecycle controller
36− data_frame_analyzer/ # Data frame analytics (classification, regression)
37− normalize/ # Anomaly score normalization
38− pytorch_inference/ # PyTorch model inference
39−lib/ # Shared libraries
40− api/ # JSON/REST API layer
41− core/ # Platform abstractions, I/O, logging, compression
42− maths/ # Mathematical and statistical algorithms
43− analytics/ # Boosted tree, data frame analytics
44− common/ # Bayesian optimisation, distributions, time series
45− time_series/ # Time series decomposition, forecasting
46− model/ # Anomaly detection models
47− seccomp/ # Seccomp/sandbox filters
48− test/ # Shared test utilities (CBoostTestXmlOutput, etc.)
49− ver/ # Version information
50−include/ # Public headers (mirrors lib/ structure)
51−3rd_party/ # Header-only third-party libraries (Eigen, valijson), licenses
52−cmake/ # CMake toolchain files, helper functions, test runners
53−build-setup/ # Platform-specific build environment instructions
54−.buildkite/ # CI pipeline definitions (Buildkite)
55−.ci/ # Packer scripts for building Orka macOS CI VMs
56−.github/workflows/ # GitHub Actions (automatic backport)
57−dev-tools/ # Developer scripts (clang-format, benchmarks)
58−```
32+## Coding Conventions
5933
60−Libraries must not have circular dependencies. The dependency order is roughly:
61−`core` -> `maths` -> `model` -> `api` -> `bin/*`.
34+- Classes: `CUpperCamelCase`, Methods: `lowerCamelCase`, Members: `m_Name`
35+- Types: `TUpperCamelCase`, Test files: `CClassNameTest.cc`
36+- Commit messages: `[ML] Short description`
37+- Use `peek() == std::char_traits<char>::eof()` for portable end-of-stream detection
38+- Avoid anonymous-namespace constants with common names in libraries using unity builds
6239
63−## CMake Helper Functions
40+## CI (Buildkite)
6441
65−The build uses custom CMake functions defined in `cmake/functions.cmake`. Use these instead of raw `add_library`/`add_executable` — they handle platform-specific sources, linking, installation, and Windows resource generation automatically.
42+- PR pipeline: `ml-cpp-pr-builds` (`.buildkite/pipeline.json.py`)
43+- Nightly: `ml-cpp-snapshot-builds` (`.buildkite/branch.json.py`)
44+- Debug: `ml-cpp-debug-build` (`.buildkite/job-build-test-all-debug.json.py`)
45+- Platforms: Linux x86_64 (6 vCPU), Linux aarch64 (8 vCPU), macOS aarch64 (4 core), Windows x86_64 (16 vCPU)
46+- Vault secrets via `.buildkite/hooks/post-checkout`
47+- Diagnostic steps use: `if: "build.state == 'failed'"` + `soft_fail: true` + `allow_dependency_failure: true`
6648
67−### Adding a Shared Library
49+## CI Analytics
6850
69−Set `ML_LINK_LIBRARIES` then call `ml_add_library`:
70−
71−```cmake
72−project("ML MyLib")
73−
74−set(ML_LINK_LIBRARIES
75− ${Boost_LIBRARIES}
76− MlCore
77− )
78−
79−ml_add_library(MlMyLib SHARED
80− CMyClass.cc
81− CMyOtherClass.cc
82− )
83−```
84−
85−Libraries are named with the `Ml` prefix (e.g. `MlCore`, `MlModel`). The function handles shared library versioning, RPATH, and installation. Use `SHARED` for distributed libraries or `STATIC` for internal-only ones.
86−
87−For libraries that should not be installed/distributed (e.g. internal helpers), use `ml_add_non_distributed_library` instead.
88−
89−### Adding an Executable
90−
91−Set `ML_LINK_LIBRARIES` then call `ml_add_executable`. A `Main.cc` file is included automatically — do not list it in the sources:
92−
93−```cmake
94−project("ML MyApp")
95−
96−set(ML_LINK_LIBRARIES
97− ${Boost_LIBRARIES}
98− MlCore
99− MlApi
100− MlVer
101− )
102−
103−ml_add_executable(myapp
104− CCmdLineParser.cc
105− )
106−```
107−
108−The function creates a companion OBJECT library (`MlMyApp`) from the listed sources, which test executables can link against. The executable itself always builds from `Main.cc` plus those objects.
109−
110−For executables not intended for distribution (dev tools, benchmarks), use `ml_add_non_distributed_executable`.
111−
112−### Adding a Test Executable
113−
114−Test executables live in `unittest/` subdirectories. Set `ML_LINK_LIBRARIES` (including `${Boost_LIBRARIES_WITH_UNIT_TEST}` and `MlTest`), then call `ml_add_test_executable`:
115−
116−```cmake
117−project("ML MyLib unit tests")
118−
119−set(SRCS
120− CMyClassTest.cc
121− CMyOtherClassTest.cc
122− Main.cc
123− )
124−
125−set(ML_LINK_LIBRARIES
126− ${Boost_LIBRARIES_WITH_UNIT_TEST}
127− MlCore
128− MlMyLib
129− MlTest
130− )
131−
132−ml_add_test_executable(mylib ${SRCS})
133−```
134−
135−The `_target` argument (e.g. `mylib`) is used to derive the test executable name (`ml_test_mylib`) and the CMake targets `test_mylib` and `test_mylib_individually`.
136−
137−### Registering Tests with the Build
138−
139−After creating the test executable, register it in `test/CMakeLists.txt` by adding an `ml_add_test` call alongside the existing entries:
140−
141−```cmake
142−ml_add_test(lib/core/unittest core)
143−ml_add_test(lib/maths/common/unittest maths_common)
144−ml_add_test(lib/maths/time_series/unittest maths_time_series)
145−ml_add_test(lib/maths/analytics/unittest maths_analytics)
146−ml_add_test(lib/model/unittest model)
147−ml_add_test(lib/api/unittest api)
148−ml_add_test(lib/ver/unittest ver)
149−ml_add_test(lib/seccomp/unittest seccomp)
150−ml_add_test(bin/controller/unittest controller)
151−ml_add_test(bin/pytorch_inference/unittest pytorch_inference)
152−ml_add_test(lib/mylib/unittest mylib) # <-- new entry
153−```
154−
155−The first argument is the relative path to the unittest directory; the second is the target name matching `ml_add_test_executable`. Note how nested libraries use underscores in the target name (e.g. `lib/maths/common/unittest` -> `maths_common`).
156−
157−### Platform-Specific Sources
158−
159−If a source file has a platform-specific variant (e.g. `CMyClass_Linux.cc`, `CMyClass_Darwin.cc`), the `ml_generate_platform_sources` function (called internally by all `ml_add_*` functions) will automatically substitute the platform-specific file at build time. Just list the base filename (`CMyClass.cc`) in your sources.
160−
161−## Testing
162−
163−Tests use the **Boost.Test** framework. Each library and application has a `unittest/` subdirectory containing test files and a `Main.cc` entry point.
164−
165−### Running Tests
166−
167−Run all tests:
168−```
169−cmake --build cmake-build-relwithdebinfo -t test
170−```
171−
172−Run tests for a specific library:
173−```
174−cmake --build cmake-build-relwithdebinfo -t test_core
175−cmake --build cmake-build-relwithdebinfo -t test_model
176−```
177−
178−Run specific test cases (wildcards supported):
179−```
180−TESTS="*/testPersist" cmake --build cmake-build-relwithdebinfo -t test_model
181−```
182−
183−Run tests individually in separate processes (better isolation, per-suite parallelism):
184−```
185−cmake --build cmake-build-relwithdebinfo -j8 -t test_individually
186−cmake --build cmake-build-relwithdebinfo -j8 -t test_api_individually
187−```
188−
189−Run all test cases from all suites in a single CTest invocation (optimal cross-suite parallelism):
190−```
191−cmake --build cmake-build-relwithdebinfo -t test_all_parallel
192−```
193−
194−Pass extra flags to the Boost.Test runner:
195−```
196−TEST_FLAGS="--random" cmake --build cmake-build-relwithdebinfo -t test
197−```
198−
199−### Precommit (format + test)
200−
201−```
202−cmake --build cmake-build-relwithdebinfo -j8 -t precommit
203−```
204−Or: `./gradlew precommit`
205−
206−### Writing Tests
207−
208−- Test files are named `CClassNameTest.cc` and placed in `lib/<module>/unittest/` or `bin/<module>/unittest/`.
209−- Each test file uses `BOOST_AUTO_TEST_SUITE(CClassNameTest)` / `BOOST_AUTO_TEST_CASE(testMethodName)`.
210−- Use real classes over mocks wherever possible. Tests should reflect real-world usage.
211−- Every class should have a corresponding test suite; every public method should have a test.
212−- Test cases must be completely independent from one another — they may be run in parallel across separate processes, so they must not depend on execution order or share mutable state.
213−
214−## Formatting & Style
215−
216−Code is formatted with `clang-format` (LLVM-based style, 4-space indent). Run before committing:
217−```
218−cmake --build cmake-build-relwithdebinfo -t format
219−```
220−Or: `./gradlew format`
221−
222−The CI pipeline enforces formatting via the `check-style` step; PRs that fail formatting will not pass CI.
223−
224−The full coding standard is in `STYLEGUIDE.md`. Key points:
225−
226−### Naming Conventions
227−- Classes: `CClassName`, Structs: `SStructName`, Enums: `EEnumName`
228−- Member variables: `m_ClassMember`, `s_StructMember`
229−- Static members: `ms_ClassStatic`
230−- Methods: `methodName` (camelCase)
231−- Type aliases: `TTypeName` (e.g. `using TDoubleVec = std::vector<double>`)
232−- Constants: `CONSTANT_NAME`
233−- Non-boolean accessors: `clientId` (not `getClientId`)
234−- Boolean accessors: `isComplete` (not `complete`)
235−- Files: `CClassName.cc` / `CClassName.h`
236−
237−### Code Conventions
238−- Use `nullptr`, never `0` or `NULL`.
239−- No exceptions — use return codes for error handling. Catch third-party exceptions at the smallest scope.
240−- No `assert()`. No C-style casts. No macros unless unavoidable.
241−- Prefer smart pointers over raw pointers; prefer references over pointers.
242−- Scope member function calls with `this->`.
243−- Use `auto` when the type is obvious; avoid it when the type is unclear.
244−- Prefer `emplace_back` over `push_back`, range-based for loops, and uniform initializers.
245−- `override` must be used consistently; `virtual` must not appear alongside `override`.
246−
247−### File Layout
248−- Implementation files (`.cc`): own header first, then other ML headers, third-party headers, standard library headers.
249−- Group includes by library with blank lines between groups (clang-format will sort within groups).
250−- Use unnamed namespaces in `.cc` files for file-local helpers, not private class members.
251−- Forward-declare classes in headers rather than including their headers.
252−
253−### Documentation
254−- Doxygen comments (exclamation mark style: `//!`) are required for all header files and public/protected methods.
255−- Implementation files use regular C++ comments, not Doxygen.
256−- Focus comments on the "why", not the "what".
257−
258−## License Headers
259−
260−All source files must include the Elastic License 2.0 header. Copy from `copyright_code_header.txt` or any existing source file.
261−
262−## CI
263−
264−CI runs on **Buildkite** (`ml-cpp-pr-builds`). The pipeline builds and tests on all platforms (Linux x86_64, Linux aarch64, macOS aarch64, Windows x86_64) in both `RelWithDebInfo` and Debug configurations. It also runs:
265−- `clang-format` style validation
266−- Snyk security/license scanning
267−- Java integration tests against Elasticsearch
268−
269−Automatic backporting is handled by a GitHub Action (`.github/workflows/backport.yml`) — add version labels (e.g. `v9.3.0`) to a PR and a backport PR is created automatically when it merges.
270−
271−## Pull Requests
272−
273−- Title must be prefixed with `[ML]` (e.g. `[ML] Fix anomaly scoring edge case`).
274−- Label with `:ml` (mandatory), a type label (`>bug`, `>enhancement`, `>feature`, `>refactoring`, `>test`, `>docs`), and version labels for applicable releases.
275−- Squash-and-merge is the standard merge strategy; keep commits clean for review but don't squash manually.
276−- Backports start after merging to `main`. Add version labels to trigger automatic backport PRs.
277−
278−## Best Practices for Automation Agents
279−
280−- Always read existing code before editing to understand patterns and conventions.
281−- Never edit unrelated files; keep diffs tightly scoped.
282−- Run `clang-format` before presenting any code changes.
283−- Match the naming conventions exactly — the prefixes (`C`, `m_`, `ms_`, `T`, `E`) are strictly followed throughout the codebase.
284−- When adding new classes, follow the existing directory and namespace structure. Production code in `lib/foo/` uses namespace `ml::foo`.
285−- When adding tests, place them in the corresponding `unittest/` directory and register them in the `CMakeLists.txt`.
286−- Do not introduce new third-party dependencies without discussion.
287−- Do not add AI attribution trailers (e.g. `Co-Authored-By`) to commit messages.
288−- Commit messages should follow the `[ML] Summary of change` format.
289−- If unsure about a convention, check a nearby file for the established pattern — consistency with surrounding code is the highest priority.
290−
291−Stay aligned with `CONTRIBUTING.md`, `STYLEGUIDE.md`, and the `build-setup/` guides; this AGENTS file summarizes but does not replace those authoritative documents.
51+- Build timings indexed into Elasticsearch Serverless (`buildkite-build-timings`)
52+- ML anomaly detection job: `build-timing-regressions` (high_mean by step_key)
53+- PR regression check: compares against 30-day baseline (mean + 2σ)
54+- AI failure analysis: Claude diagnoses failures, posts Buildkite annotations
55+- Kibana dashboard: "ML-CPP Build Timing Overview"
29256
