Two files, one repository

elastic/elasticsearch ships 1 format across 6 indexed files. The question worth asking is whether the second one says anything the first does not.

A · AGENTS.md · 2046 words B · benchmarks/AGENTS.md · 146 words

What each file covers, counted
Dimension	Shared	Only in A	Only in B	Overlap
Sections	0	28	6	0%
Commands	0	9	0	0%
Section tags	1	9	0	10%

What each file covers

Sections

0 shared · 28 only in A · 6 only in B

− Elasticsearch
− Toolchain Snapshot
− Build & Run Commands
− Verification & Lint Tasks
− Project Structure
− Stateless Elasticsearch
− Plugin `deploymentTarget`
− Plugin locations
− Key subsystems
− Testing Cheatsheet
− Test Types
− Distribution selection for external-module tests
− Dependency Hygiene
− Entitlement Policy
− Formatting & Imports
− Types, Generics, and Suppressions
− Naming Conventions
− Logging & Error Handling
− Javadoc & Comments
− License Headers
− Generated Files
− Debugging Missing Tests
− `No tests found for given includes: [**/*$*.class]`
− Best Practices for Automation Agents
− Methods with Required Javadoc Reading
− ES|QL tests
− Backwards compatibility
− Documentation
+ Benchmarks
+ Running benchmarks
+ ColumNAR transform benchmarks
+ Single stage + pattern
+ Quick smoke
+ Self-test

Commands

0 shared · 9 only in A · 0 only in B

− ./gradlew
− ./gradlew spotlessJavaCheck
− ./gradlew test
− ./gradlew :server:test
− ./gradlew :server:test --tests org.elasticsearch.package.ClassName
− ./gradlew :server:test --tests 'org.elasticsearch.package.*'
− ./gradlew :server:test --tests org.elasticsearch.package.ClassName.methodName -Dtests.iters=N
− ./gradlew ":x-pack:plugin:esql:internalClusterTest" --tests "org.elasticsearch.xpack.esql.CsvIT.*<csv-file>*"
− ./gradlew generateTransportVersion

Section tags

1 shared · 9 only in A · 0 only in B

− build
− lint-format
− code-style
− architecture
− types
− testing-strategy
− git-pr
− do-not
− docs
test

Line diff

+22 added−170 removed10 unchanged5.6% identical

elastic/elasticsearch · AGENTS.md

@@ −1 @@

1# Elasticsearch

3## Toolchain Snapshot

4- **Java**: JDK 25 via `JAVA_HOME`; use the bundled Gradle wrapper (`./gradlew`).

5- **Build tooling**: Gradle composite build with `build-conventions`, `build-tools`, and `build-tools-internal`; Docker is required for some packaging/tests.

6- **OS packages**: Packaging and QA jobs expect ephemeral hosts; do not run packaging suites on your workstation.

7- **Security**: Default dev clusters enable security; use `elastic-admin:elastic-password` or disable with `-Dtests.es.xpack.security.enabled=false`.

8- **Cursor/Copilot rules**: None provided in repo; follow this guide plus CONTRIBUTING.md.

10## Build & Run Commands

11- Refer to BUILDING.md, CONTRIBUTING.md & TESTING.asciidoc for comprehensive build/test instructions.

13## Verification & Lint Tasks

14- `./gradlew spotlessJavaCheck` / `spotlessApply` (or `:server:spotlessJavaCheck`): enforce formatter profile in `build-conventions/formatterConfig.xml`.

15- `spotlessApply` also prunes unused imports and reorders imports automatically. Run it instead of manually hunting for unused imports after refactoring.

17## Project Structure

18The repository is organized into several key directories:

19* `server`: The core Elasticsearch server. Few third-party dependencies (Lucene plus a handful of small libraries). Key `org.elasticsearch` sub-packages: `cluster` (cluster state machine), `index` (per-index logic), `search` (query execution), `action` (transport actions), `snapshots` (snapshot/restore), plus `indices`, `repositories`, `rest`, `ingest`, etc.

20* `modules`: Features shipped with Elasticsearch by default, but not considered "core" server code. Many modules provide a specific implementation of a pluggable interface defined in `server`, such as `transport-netty4` (the transport layer) or `repository-s3`/`repository-gcs`/`repository-azure` (snapshot repositories). Others integrate with external systems, such as `apm` (Application Performance Monitoring agent integration).

21* `plugins`: Optional, not bundled by default, but officially supported. Examples: `discovery-ec2`/`discovery-gce`/`discovery-azure-classic` (cloud-aware cluster discovery).

22* `libs`: Internal libraries used by multiple parts of the project. Examples: `logging`, `x-content` (JSON/CBOR/YAML/SMILE parsing abstraction).

23* `client`: The official Java REST client.

24* `test`: Test infrastructure used by the rest of the repo. `framework` holds `ESTestCase`/`ESIntegTestCase`/`ESSingleNodeTestCase`; also `test-clusters` and `yaml-rest-runner` (runner for YAML-based REST API tests).

25* `qa`: Integration and multi-version tests. Examples: `rolling-upgrade`, `mixed-cluster`.

26* `rest-api-spec`: JSON spec definitions for the public REST API endpoints.

27* `docs`: Project documentation.

28* `distribution`: Logic for building distribution packages.

29* `x-pack`: Modules, plugins, and commercial features under the Elastic License 2.0. Example sub-plugins: `security`, `ml` (machine learning), `ccr` (cross-cluster replication), `logsdb` (optimized index mode for log data), and `stateless`.

30* `build-conventions`, `build-tools`, `build-tools-internal`: Gradle build logic. Refer to BUILDING.md for details on how these are structured and used.

32## Stateless Elasticsearch

34Stateless Elasticsearch is a distribution where shard data is stored in an **object store** (e.g., S3, GCS, Azure) rather than local disk. Nodes carry no durable local state. The cluster distinguishes two node roles: **indexing nodes** (`index` role, write path + translog replication to object store) and **search nodes** (`search` role, read-only via shared blob cache). The `DiscoveryNode.STATELESS_ENABLED_SETTING` gates stateless behavior at runtime.

36### Plugin `deploymentTarget`

38Plugins can set `deploymentTarget` in `build.gradle`. That value tells the node **whether to load the plugin**: **`STATEFUL_ONLY`** (stateful clusters only), **`STATELESS_ONLY`** (stateless mode on only), or **`ALL`** (always loaded; this is the default when the property is omitted).

40### Plugin locations

42| Plugin | Gradle path | Purpose |

43|---|---|---|

44| `stateless` | `:x-pack:plugin:stateless` | Core stateless — engines, allocation, cache, object store, recovery |

45| `stateless-sigterm` | `:x-pack:plugin:stateless-sigterm` | Clean SIGTERM shutdown for Kubernetes |

46| `stateless-master-failover` | `:x-pack:plugin:stateless-master-failover` | Master failover behavior |

47| `stateless-no-wait-for-active-shards` | `:x-pack:plugin:stateless-no-wait-for-active-shards` | Suppresses wait-for-active-shards |

48| `stateless-health-shards-availability` | `:x-pack:plugin:stateless-health-shards-availability` | Shard availability health indicators |

50**Package**: `org.elasticsearch.xpack.stateless.*` throughout.

52### Key subsystems

54- **Object store** (`objectstore/`): `ObjectStoreService`, bucket config, GC tasks for stale indices and translogs.

55- **Commits** (`commits/`): `StatelessCommitService` manages shard commits to blob store; `HollowShardsService` manages hollow indexing shards.

56- **Cache & prewarming** (`cache/`): `StatelessSharedBlobCacheService`, online prewarming, `SearchCommitPrefetcher`.

57- **Engines** (`engine/`): `IndexEngine` (write path) and `SearchEngine` (read-only); `TranslogReplicator` replicates translog to object store.

58- **Allocation** (`allocation/`): `StatelessExistingShardsAllocator`, separate balancing weights per tier, heap-usage-aware allocation decisions.

59- **Recovery** (`recovery/`): custom primary relocation and unpromotable shard relocation protocols.

61## Testing Cheatsheet

62- Standard suite: `./gradlew test` (respects cached results; add `-Dtests.timestamp=$(date +%s)` to bypass caches when reusing seeds).

63- Single project: `./gradlew :server:test` (or other subproject path).

64- Single class: `./gradlew :server:test --tests org.elasticsearch.package.ClassName`.

65- Single package: `./gradlew :server:test --tests 'org.elasticsearch.package.*'`.

66- Single method / repeated runs: `./gradlew :server:test --tests org.elasticsearch.package.ClassName.methodName -Dtests.iters=N`.

67- Deterministic seed: append `-Dtests.seed=DEADBEEF` (each method uses derived seeds).

68- JVM tuning knobs: `-Dtests.jvms=8`, `-Dtests.heap.size=4G`, `-Dtests.jvm.argline="-verbose:gc"`, `-Dtests.output=always`, etc.

69- Debugging: append `--debug-jvm` to the Gradle test task and attach a debugger on port 5005.

70- CI reproductions: copy the `REPRODUCE WITH` line from CI logs; it includes project path, seed, and JVM flags.

71- Yaml REST tests: `./gradlew ":rest-api-spec:yamlRestTest" --tests "org.elasticsearch.test.rest.ClientYamlTestSuiteIT.test {yaml=<relative_test_file_path>}"`

72- ES|QL CSV tests: `./gradlew ":x-pack:plugin:esql:internalClusterTest" --tests "org.elasticsearch.xpack.esql.CsvIT.*<csv-file>*"` (e.g. `--tests "...CsvIT.*stats_first_last*"`); append `*<test-name>*` to target a single test within the file.

73- Use the Elasticsearch testing framework where possible for unit and yaml tests and be consistent in style with other elasticsearch tests.

74- Use real classes over mocks or stubs for unit tests, unless the real class is complex then either a simplified subclass should be created within the test or, as a last resort, a mock or stub can be used. Unit tests must be as close to real-world scenarios as possible.

75- Ensure mocks or stubs are well-documented and clearly indicate why they were necessary.

77### Test Types

78- Unit Tests: Preferred. Extend `ESTestCase`.

79- Single Node: Extend `ESSingleNodeTestCase` (lighter than full integ test).

80- Integration: Extend `ESIntegTestCase`.

81- REST API: Extend `ESRestTestCase` or `ESClientYamlSuiteTestCase`. **YAML based REST tests are preferred** for integration/API testing.

83### Distribution selection for external-module tests

84- Prefer the OSS/minimal distribution over `usesDefaultDistribution` whenever possible. `usesDefaultDistribution` packages the full default distribution, which is significantly more expensive to build and run.

85- Only use `usesDefaultDistribution` when the test genuinely requires a feature that is only available in the default distribution and cannot be replicated with a custom cluster configuration that includes just the needed plugins. Always document the reason in the `usesDefaultDistribution(...)` message.

87## Dependency Hygiene

88- Never add a dependency without checking for existing alternatives in the repo.

90## Entitlement Policy

91- Never add an entitlement speculatively. Each entry in `entitlement-policy.yaml` must have a specific justification — ideally a concrete `NotEntitledException` that was observed, or at minimum a clear explanation of why the library requires that capability. Entitlements are a least-privilege mechanism; granting one "just in case" defeats the purpose.

92- Every use of `ESTestCase.WithoutEntitlements` must be accompanied by a comment explaining why the entitlement failure is spurious in the test context and would not occur in production.

94## Formatting & Imports

95- Absolutely no wildcard imports; keep existing import order and avoid reordering untouched lines.

96- In `switch` statements, do not use `default` as a branch for valid or expected options. Enumerate those cases explicitly and reserve `default` for throwing an exception for unexpected values, or an assertion error if this code branch is unreachable.

98## Types, Generics, and Suppressions

99- Prefer type-safe constructs; avoid raw types and unchecked casts.

100- If suppressing warnings, scope `@SuppressWarnings` narrowly (ideally a single statement or method).

101- Document non-obvious casts or type assumptions via Javadoc/comments for reviewers.

102

103## Naming Conventions

104- REST handlers typically use the `Rest*Action` pattern; transport-layer handlers mirror them with `Transport*Action` classes.

105- REST classes expose routes via `RestHandler#routes`; when adding endpoints ensure naming matches existing REST/Transport patterns to aid discoverability.

106- Transport `ActionType` strings encode scope (`indices:data/read/...`, `cluster:admin/...`, etc.); align new names with these conventions to integrate with privilege resolution.

107

108## Logging & Error Handling

109- Elasticsearch should prefer its own logger `org.elasticsearch.logging.LogManager` & `org.elasticsearch.logging.Logger`; declare `private static final Logger logger = LogManager.getLogger(Class.class)`.

110- Always use parameterized logging (`logger.debug("operation [{}]", value)`); never build strings via concatenation.

111- Wrap expensive log-message construction in `() -> Strings.format(...)` suppliers when logging at `TRACE`/`DEBUG` to avoid unnecessary work.

112- Log levels:

113 - `TRACE`: highly verbose developer diagnostics; usually read alongside code.

114 - `DEBUG`: detailed production troubleshooting; ensure volume is bounded.

115 - `INFO`: default-enabled operational milestones; prefer factual language.

116 - `WARN`: actionable problems users must investigate; include context and, if needed, exception stack traces.

117 - `ERROR`: reserve for unrecoverable states (e.g., storage health failures); prefer `WARN` otherwise.

118- Only log client-caused exceptions when the cluster admin can act on them; otherwise rely on API responses.

119- Tests can assert logging via `MockLog` for complex flows.

120

121## Javadoc & Comments

122- New packages/classes/public or abstract methods require Javadoc explaining the "why" rather than the implementation details.

123- Avoid documenting trivial getters/setters; focus on behavior, preconditions, or surprises.

124- For tests, Javadoc can describe scenario setup/expectations to aid future contributors.

125- Do not remove existing comments from code unless the code is also being removed or the comment has become incorrect.

126

127## License Headers

128- Default header (outside `x-pack`): Elastic License 2.0, SSPL v1, or AGPL v3—they are already codified at the top of Java files; copy from existing sources.

129- Files under `x-pack` require the Elastic License 2.0-only header; IDEs configured per CONTRIBUTING.md can insert correct text automatically.

130

131## Generated Files

132- Never hand-edit generated files. Instead, edit the source they are generated from and regenerate.

133- ANTLR-generated files can be regenerated by running the `regen` task on the relevant subproject.

134- Other generated files are regenerated by compiling the project.

135

136## Debugging Missing Tests

137

138When expected test methods are absent from results (not failed, not skipped — simply not present in the XML or binary event stream), check `muted-tests.yml` first. The build translates every entry into a Gradle `TestFilter.excludePattern`, which silently drops matching tests before the randomized runner receives them. A muted test fires no `testStarted` event and leaves no trace in `results-generic.bin`.

139 ```bash

140 grep 'ClassName\|methodName' muted-tests.yml

141 ```

142

143### `No tests found for given includes: [**/*$*.class]`

144

145When a test task fails at execution with `No tests found for given includes: [**/*$*.class](exclude rules)`, it usually does **not** mean Gradle failed to detect the test class. The far more common cause is that **every test method in the targeted class is muted** in `muted-tests.yml`. With all methods excluded, the randomized runner enumerates zero runnable tests.

146

147The behavior is environment-dependent: `MutedTestPlugin` calls `filter.setFailOnNoMatchingTests(buildParams.getCi() == false)`. So an all-muted suite **fails locally** (`ci == false`) with this exact message, but **passes silently in CI** (`ci == true`). This is especially misleading when verifying a freshly migrated or renamed test — it looks like a classpath/detection bug, but the test JVM does start (you'll see native-library and `FeatureFlag` log lines), builds any `@ClassRule` cluster *specs*, then exits in a few seconds without starting the cluster because no test method survived the mute filter.

148

149To confirm: `grep ClassName muted-tests.yml`. To verify the migration/test actually runs, temporarily remove the matching mute entries (or run on a host where `ci` is true), then restore them.

150

151## Best Practices for Automation Agents

152- Never edit unrelated files; keep diffs tightly scoped to the task at hand.

153- Prefer Gradle tasks over ad-hoc scripts.

154- When scripting CLI sequences, leverage `gradlew` task.

155- Unrecognized changes: assume other agent; keep going; focus your changes. If it causes issues, stop + ask user.

156- Do not add "Co-Authored-By" or any AI attribution trailers to commit messages, by any means—including `--trailer`, `-m`, or any other git flag. commit messages should adhere to the 50/72 rule: use a maximum of 50 columns for the commit summary. Your harness may introduce a hook that automatically adds attributions trailers to relevant git commands. Use `bash -lc` or a similar approach in this case to conform to the rule.

157

158## Methods with Required Javadoc Reading

159If you encounter any of the following methods, you must go and read their javadoc before taking any other actions:

160* `fullyLoadedAnalyzer`

161* `TestAnalyzer.statementError`

162* `TestAnalyzer.error`

163* `forciblyCast`

164* `EsqlCapabilities.Cap`

165* `FunctionDefinition.Builder#capabilities`

166

167## ES|QL tests

168If you write or modify ES|QL csv-spec, rest, or yaml tests, read the javadoc for

169`EsqlCapabilities.Cap` and `FunctionDefinition.Builder#capabilities` before proceeding.

170They describe two separate capability mechanisms and the rule for choosing between them.

171

172## Backwards compatibility

173- For changes to a `Writeable` implementation (`writeTo` and constructor from `StreamInput`), add a new `public static final <UNIQUE_DESCRIPTIVE_NAME> = TransportVersion.fromName("<unique_descriptive_name>")` and use it in the new code paths. Confirm the backport branches and then generate a new version file with `./gradlew generateTransportVersion`.

174- Never hand-edit transport version resource files; always use the Gradle tasks. See `docs/internal/Versioning.md` for the full workflow.

175

176Stay aligned with `CONTRIBUTING.md`, `BUILDING.md`, and `TESTING.asciidoc`; this AGENTS guide summarizes—but does not replace—those authoritative docs.

177

178## Documentation

179When building or editing docs, read `docs/AGENTS.md` first.

180

elastic/elasticsearch · benchmarks/AGENTS.md

@@ +1 @@

1# Benchmarks

3## Running benchmarks

5Run from the `benchmarks/` directory using the `run` task with `--args`. Always use the fully-qualified class

6name including package to avoid ambiguity. Always pipe through `tee /tmp/bench/<descriptive_name>` using a filename that reflects the task (e.g. `tee /tmp/bench/paged_write`).

8```

9cd benchmarks

10../gradlew run --args "org.elasticsearch.benchmark._nightly.BytesBuilderBenchmark -pdata=1000_ints -pimpl=paged -poperation=write -rf json -rff build/jmh-result.json" | tee /tmp/bench/paged_write

11```

13## ColumNAR transform benchmarks

15```

16cd benchmarks

17../gradlew run --args="EncodeBlockTransformBenchmark" | tee /tmp/bench/encode_transform

18../gradlew run --args="DecodeBlockTransformBenchmark" | tee /tmp/bench/decode_transform

20# Single stage + pattern

21../gradlew run --args="EncodeBlockTransformBenchmark -p stage=splitDelta -p pattern=TSDB_SPLIT" | tee /tmp/bench/encode_splitdelta_tsdb

23# Quick smoke

24../gradlew run --args="EncodeBlockTransformBenchmark -wi 1 -i 1 -f 1 -w 1 -r 1 -p stage=delta -p pattern=MONOTONIC_TIMESTAMPS"

25```

27## Self-test

29Never skip the self-test. Do not pass `-DskipSelfTest=true` or `--test` to `run.sh`. The

30self-test validates correctness across all impl/operation/data combinations and poisons virtual

31dispatch to behave more like production.

@@ −1 +1 @@

1−# Elasticsearch

1+# Benchmarks

3−## Toolchain Snapshot

4−- **Java**: JDK 25 via `JAVA_HOME`; use the bundled Gradle wrapper (`./gradlew`).

5−- **Build tooling**: Gradle composite build with `build-conventions`, `build-tools`, and `build-tools-internal`; Docker is required for some packaging/tests.

6−- **OS packages**: Packaging and QA jobs expect ephemeral hosts; do not run packaging suites on your workstation.

7−- **Security**: Default dev clusters enable security; use `elastic-admin:elastic-password` or disable with `-Dtests.es.xpack.security.enabled=false`.

8−- **Cursor/Copilot rules**: None provided in repo; follow this guide plus CONTRIBUTING.md.

3+## Running benchmarks

10−## Build & Run Commands

11−- Refer to BUILDING.md, CONTRIBUTING.md & TESTING.asciidoc for comprehensive build/test instructions.

5+Run from the `benchmarks/` directory using the `run` task with `--args`. Always use the fully-qualified class

6+name including package to avoid ambiguity. Always pipe through `tee /tmp/bench/<descriptive_name>` using a filename that reflects the task (e.g. `tee /tmp/bench/paged_write`).

127

13−## Verification & Lint Tasks

14−- `./gradlew spotlessJavaCheck` / `spotlessApply` (or `:server:spotlessJavaCheck`): enforce formatter profile in `build-conventions/formatterConfig.xml`.

15−- `spotlessApply` also prunes unused imports and reorders imports automatically. Run it instead of manually hunting for unused imports after refactoring.

8+```

9+cd benchmarks

10+../gradlew run --args "org.elasticsearch.benchmark._nightly.BytesBuilderBenchmark -pdata=1000_ints -pimpl=paged -poperation=write -rf json -rff build/jmh-result.json" | tee /tmp/bench/paged_write

11+```

1612

17−## Project Structure

18−The repository is organized into several key directories:

19−* `server`: The core Elasticsearch server. Few third-party dependencies (Lucene plus a handful of small libraries). Key `org.elasticsearch` sub-packages: `cluster` (cluster state machine), `index` (per-index logic), `search` (query execution), `action` (transport actions), `snapshots` (snapshot/restore), plus `indices`, `repositories`, `rest`, `ingest`, etc.

20−* `modules`: Features shipped with Elasticsearch by default, but not considered "core" server code. Many modules provide a specific implementation of a pluggable interface defined in `server`, such as `transport-netty4` (the transport layer) or `repository-s3`/`repository-gcs`/`repository-azure` (snapshot repositories). Others integrate with external systems, such as `apm` (Application Performance Monitoring agent integration).

21−* `plugins`: Optional, not bundled by default, but officially supported. Examples: `discovery-ec2`/`discovery-gce`/`discovery-azure-classic` (cloud-aware cluster discovery).

22−* `libs`: Internal libraries used by multiple parts of the project. Examples: `logging`, `x-content` (JSON/CBOR/YAML/SMILE parsing abstraction).

23−* `client`: The official Java REST client.

24−* `test`: Test infrastructure used by the rest of the repo. `framework` holds `ESTestCase`/`ESIntegTestCase`/`ESSingleNodeTestCase`; also `test-clusters` and `yaml-rest-runner` (runner for YAML-based REST API tests).

25−* `qa`: Integration and multi-version tests. Examples: `rolling-upgrade`, `mixed-cluster`.

26−* `rest-api-spec`: JSON spec definitions for the public REST API endpoints.

27−* `docs`: Project documentation.

28−* `distribution`: Logic for building distribution packages.

29−* `x-pack`: Modules, plugins, and commercial features under the Elastic License 2.0. Example sub-plugins: `security`, `ml` (machine learning), `ccr` (cross-cluster replication), `logsdb` (optimized index mode for log data), and `stateless`.

30−* `build-conventions`, `build-tools`, `build-tools-internal`: Gradle build logic. Refer to BUILDING.md for details on how these are structured and used.

13+## ColumNAR transform benchmarks

3114

32−## Stateless Elasticsearch

15+```

16+cd benchmarks

17+../gradlew run --args="EncodeBlockTransformBenchmark" | tee /tmp/bench/encode_transform

18+../gradlew run --args="DecodeBlockTransformBenchmark" | tee /tmp/bench/decode_transform

3319

34−Stateless Elasticsearch is a distribution where shard data is stored in an **object store** (e.g., S3, GCS, Azure) rather than local disk. Nodes carry no durable local state. The cluster distinguishes two node roles: **indexing nodes** (`index` role, write path + translog replication to object store) and **search nodes** (`search` role, read-only via shared blob cache). The `DiscoveryNode.STATELESS_ENABLED_SETTING` gates stateless behavior at runtime.

20+# Single stage + pattern

21+../gradlew run --args="EncodeBlockTransformBenchmark -p stage=splitDelta -p pattern=TSDB_SPLIT" | tee /tmp/bench/encode_splitdelta_tsdb

3522

36−### Plugin `deploymentTarget`

23+# Quick smoke

24+../gradlew run --args="EncodeBlockTransformBenchmark -wi 1 -i 1 -f 1 -w 1 -r 1 -p stage=delta -p pattern=MONOTONIC_TIMESTAMPS"

25+```

3726

38−Plugins can set `deploymentTarget` in `build.gradle`. That value tells the node **whether to load the plugin**: **`STATEFUL_ONLY`** (stateful clusters only), **`STATELESS_ONLY`** (stateless mode on only), or **`ALL`** (always loaded; this is the default when the property is omitted).

27+## Self-test

3928

40−### Plugin locations

41−

42−| Plugin | Gradle path | Purpose |

43−|---|---|---|

44−| `stateless` | `:x-pack:plugin:stateless` | Core stateless — engines, allocation, cache, object store, recovery |

45−| `stateless-sigterm` | `:x-pack:plugin:stateless-sigterm` | Clean SIGTERM shutdown for Kubernetes |

46−| `stateless-master-failover` | `:x-pack:plugin:stateless-master-failover` | Master failover behavior |

47−| `stateless-no-wait-for-active-shards` | `:x-pack:plugin:stateless-no-wait-for-active-shards` | Suppresses wait-for-active-shards |

48−| `stateless-health-shards-availability` | `:x-pack:plugin:stateless-health-shards-availability` | Shard availability health indicators |

49−

50−**Package**: `org.elasticsearch.xpack.stateless.*` throughout.

51−

52−### Key subsystems

53−

54−- **Object store** (`objectstore/`): `ObjectStoreService`, bucket config, GC tasks for stale indices and translogs.

55−- **Commits** (`commits/`): `StatelessCommitService` manages shard commits to blob store; `HollowShardsService` manages hollow indexing shards.

56−- **Cache & prewarming** (`cache/`): `StatelessSharedBlobCacheService`, online prewarming, `SearchCommitPrefetcher`.

57−- **Engines** (`engine/`): `IndexEngine` (write path) and `SearchEngine` (read-only); `TranslogReplicator` replicates translog to object store.

58−- **Allocation** (`allocation/`): `StatelessExistingShardsAllocator`, separate balancing weights per tier, heap-usage-aware allocation decisions.

59−- **Recovery** (`recovery/`): custom primary relocation and unpromotable shard relocation protocols.

60−

61−## Testing Cheatsheet

62−- Standard suite: `./gradlew test` (respects cached results; add `-Dtests.timestamp=$(date +%s)` to bypass caches when reusing seeds).

63−- Single project: `./gradlew :server:test` (or other subproject path).

64−- Single class: `./gradlew :server:test --tests org.elasticsearch.package.ClassName`.

65−- Single package: `./gradlew :server:test --tests 'org.elasticsearch.package.*'`.

66−- Single method / repeated runs: `./gradlew :server:test --tests org.elasticsearch.package.ClassName.methodName -Dtests.iters=N`.

67−- Deterministic seed: append `-Dtests.seed=DEADBEEF` (each method uses derived seeds).

68−- JVM tuning knobs: `-Dtests.jvms=8`, `-Dtests.heap.size=4G`, `-Dtests.jvm.argline="-verbose:gc"`, `-Dtests.output=always`, etc.

69−- Debugging: append `--debug-jvm` to the Gradle test task and attach a debugger on port 5005.

70−- CI reproductions: copy the `REPRODUCE WITH` line from CI logs; it includes project path, seed, and JVM flags.

71−- Yaml REST tests: `./gradlew ":rest-api-spec:yamlRestTest" --tests "org.elasticsearch.test.rest.ClientYamlTestSuiteIT.test {yaml=<relative_test_file_path>}"`

72−- ES|QL CSV tests: `./gradlew ":x-pack:plugin:esql:internalClusterTest" --tests "org.elasticsearch.xpack.esql.CsvIT.*<csv-file>*"` (e.g. `--tests "...CsvIT.*stats_first_last*"`); append `*<test-name>*` to target a single test within the file.

73−- Use the Elasticsearch testing framework where possible for unit and yaml tests and be consistent in style with other elasticsearch tests.

74−- Use real classes over mocks or stubs for unit tests, unless the real class is complex then either a simplified subclass should be created within the test or, as a last resort, a mock or stub can be used. Unit tests must be as close to real-world scenarios as possible.

75−- Ensure mocks or stubs are well-documented and clearly indicate why they were necessary.

76−

77−### Test Types

78−- Unit Tests: Preferred. Extend `ESTestCase`.

79−- Single Node: Extend `ESSingleNodeTestCase` (lighter than full integ test).

80−- Integration: Extend `ESIntegTestCase`.

81−- REST API: Extend `ESRestTestCase` or `ESClientYamlSuiteTestCase`. **YAML based REST tests are preferred** for integration/API testing.

82−

83−### Distribution selection for external-module tests

84−- Prefer the OSS/minimal distribution over `usesDefaultDistribution` whenever possible. `usesDefaultDistribution` packages the full default distribution, which is significantly more expensive to build and run.

85−- Only use `usesDefaultDistribution` when the test genuinely requires a feature that is only available in the default distribution and cannot be replicated with a custom cluster configuration that includes just the needed plugins. Always document the reason in the `usesDefaultDistribution(...)` message.

86−

87−## Dependency Hygiene

88−- Never add a dependency without checking for existing alternatives in the repo.

89−

90−## Entitlement Policy

91−- Never add an entitlement speculatively. Each entry in `entitlement-policy.yaml` must have a specific justification — ideally a concrete `NotEntitledException` that was observed, or at minimum a clear explanation of why the library requires that capability. Entitlements are a least-privilege mechanism; granting one "just in case" defeats the purpose.

92−- Every use of `ESTestCase.WithoutEntitlements` must be accompanied by a comment explaining why the entitlement failure is spurious in the test context and would not occur in production.

93−

94−## Formatting & Imports

95−- Absolutely no wildcard imports; keep existing import order and avoid reordering untouched lines.

96−- In `switch` statements, do not use `default` as a branch for valid or expected options. Enumerate those cases explicitly and reserve `default` for throwing an exception for unexpected values, or an assertion error if this code branch is unreachable.

97−

98−## Types, Generics, and Suppressions

99−- Prefer type-safe constructs; avoid raw types and unchecked casts.

100−- If suppressing warnings, scope `@SuppressWarnings` narrowly (ideally a single statement or method).

101−- Document non-obvious casts or type assumptions via Javadoc/comments for reviewers.

102−

103−## Naming Conventions

104−- REST handlers typically use the `Rest*Action` pattern; transport-layer handlers mirror them with `Transport*Action` classes.

105−- REST classes expose routes via `RestHandler#routes`; when adding endpoints ensure naming matches existing REST/Transport patterns to aid discoverability.

106−- Transport `ActionType` strings encode scope (`indices:data/read/...`, `cluster:admin/...`, etc.); align new names with these conventions to integrate with privilege resolution.

107−

108−## Logging & Error Handling

109−- Elasticsearch should prefer its own logger `org.elasticsearch.logging.LogManager` & `org.elasticsearch.logging.Logger`; declare `private static final Logger logger = LogManager.getLogger(Class.class)`.

110−- Always use parameterized logging (`logger.debug("operation [{}]", value)`); never build strings via concatenation.

111−- Wrap expensive log-message construction in `() -> Strings.format(...)` suppliers when logging at `TRACE`/`DEBUG` to avoid unnecessary work.

112−- Log levels:

113− - `TRACE`: highly verbose developer diagnostics; usually read alongside code.

114− - `DEBUG`: detailed production troubleshooting; ensure volume is bounded.

115− - `INFO`: default-enabled operational milestones; prefer factual language.

116− - `WARN`: actionable problems users must investigate; include context and, if needed, exception stack traces.

117− - `ERROR`: reserve for unrecoverable states (e.g., storage health failures); prefer `WARN` otherwise.

118−- Only log client-caused exceptions when the cluster admin can act on them; otherwise rely on API responses.

119−- Tests can assert logging via `MockLog` for complex flows.

120−

121−## Javadoc & Comments

122−- New packages/classes/public or abstract methods require Javadoc explaining the "why" rather than the implementation details.

123−- Avoid documenting trivial getters/setters; focus on behavior, preconditions, or surprises.

124−- For tests, Javadoc can describe scenario setup/expectations to aid future contributors.

125−- Do not remove existing comments from code unless the code is also being removed or the comment has become incorrect.

126−

127−## License Headers

128−- Default header (outside `x-pack`): Elastic License 2.0, SSPL v1, or AGPL v3—they are already codified at the top of Java files; copy from existing sources.

129−- Files under `x-pack` require the Elastic License 2.0-only header; IDEs configured per CONTRIBUTING.md can insert correct text automatically.

130−

131−## Generated Files

132−- Never hand-edit generated files. Instead, edit the source they are generated from and regenerate.

133−- ANTLR-generated files can be regenerated by running the `regen` task on the relevant subproject.

134−- Other generated files are regenerated by compiling the project.

135−

136−## Debugging Missing Tests

137−

138−When expected test methods are absent from results (not failed, not skipped — simply not present in the XML or binary event stream), check `muted-tests.yml` first. The build translates every entry into a Gradle `TestFilter.excludePattern`, which silently drops matching tests before the randomized runner receives them. A muted test fires no `testStarted` event and leaves no trace in `results-generic.bin`.

139− ```bash

140− grep 'ClassName\|methodName' muted-tests.yml

141− ```

142−

143−### `No tests found for given includes: [**/*$*.class]`

144−

145−When a test task fails at execution with `No tests found for given includes: [**/*$*.class](exclude rules)`, it usually does **not** mean Gradle failed to detect the test class. The far more common cause is that **every test method in the targeted class is muted** in `muted-tests.yml`. With all methods excluded, the randomized runner enumerates zero runnable tests.

146−

147−The behavior is environment-dependent: `MutedTestPlugin` calls `filter.setFailOnNoMatchingTests(buildParams.getCi() == false)`. So an all-muted suite **fails locally** (`ci == false`) with this exact message, but **passes silently in CI** (`ci == true`). This is especially misleading when verifying a freshly migrated or renamed test — it looks like a classpath/detection bug, but the test JVM does start (you'll see native-library and `FeatureFlag` log lines), builds any `@ClassRule` cluster *specs*, then exits in a few seconds without starting the cluster because no test method survived the mute filter.

148−

149−To confirm: `grep ClassName muted-tests.yml`. To verify the migration/test actually runs, temporarily remove the matching mute entries (or run on a host where `ci` is true), then restore them.

150−

151−## Best Practices for Automation Agents

152−- Never edit unrelated files; keep diffs tightly scoped to the task at hand.

153−- Prefer Gradle tasks over ad-hoc scripts.

154−- When scripting CLI sequences, leverage `gradlew` task.

155−- Unrecognized changes: assume other agent; keep going; focus your changes. If it causes issues, stop + ask user.

156−- Do not add "Co-Authored-By" or any AI attribution trailers to commit messages, by any means—including `--trailer`, `-m`, or any other git flag. commit messages should adhere to the 50/72 rule: use a maximum of 50 columns for the commit summary. Your harness may introduce a hook that automatically adds attributions trailers to relevant git commands. Use `bash -lc` or a similar approach in this case to conform to the rule.

157−

158−## Methods with Required Javadoc Reading

159−If you encounter any of the following methods, you must go and read their javadoc before taking any other actions:

160−* `fullyLoadedAnalyzer`

161−* `TestAnalyzer.statementError`

162−* `TestAnalyzer.error`

163−* `forciblyCast`

164−* `EsqlCapabilities.Cap`

165−* `FunctionDefinition.Builder#capabilities`

166−

167−## ES|QL tests

168−If you write or modify ES|QL csv-spec, rest, or yaml tests, read the javadoc for

169−`EsqlCapabilities.Cap` and `FunctionDefinition.Builder#capabilities` before proceeding.

170−They describe two separate capability mechanisms and the rule for choosing between them.

171−

172−## Backwards compatibility

173−- For changes to a `Writeable` implementation (`writeTo` and constructor from `StreamInput`), add a new `public static final <UNIQUE_DESCRIPTIVE_NAME> = TransportVersion.fromName("<unique_descriptive_name>")` and use it in the new code paths. Confirm the backport branches and then generate a new version file with `./gradlew generateTransportVersion`.

174−- Never hand-edit transport version resource files; always use the Gradle tasks. See `docs/internal/Versioning.md` for the full workflow.

175−

176−Stay aligned with `CONTRIBUTING.md`, `BUILDING.md`, and `TESTING.asciidoc`; this AGENTS guide summarizes—but does not replace—those authoritative docs.

177−

178−## Documentation

179−When building or editing docs, read `docs/AGENTS.md` first.

29+Never skip the self-test. Do not pass `-DskipSelfTest=true` or `--test` to `run.sh`. The

30+self-test validates correctness across all impl/operation/data combinations and poisons virtual

31+dispatch to behave more like production.

18032