Comparison

A · AGENTS.md · elastic/elasticsearch B · AGENTS.md · elastic/elasticsearch

What each file covers, counted
Dimension	Shared	Only in A	Only in B	Overlap
Sections	0	6	16	0%
Commands	0	0	7	0%
Section tags	1	0	6	14%

What each file covers

Sections

0 shared · 6 only in A · 16 only in B

− Benchmarks
− Running benchmarks
− ColumNAR transform benchmarks
− Single stage + pattern
− Quick smoke
− Self-test
+ Inference API plugin (x-pack-inference)
+ This plugin vs. the ML plugin — don't confuse them
+ Where the SPI lives (important)
+ Build & Test Commands
+ Unit tests
+ Internal cluster tests (*IT under ...inference.integration)
+ YAML REST tests (specs in src/yamlRestTest/resources/rest-api-spec/test/inference/)
+ Format / forbidden-API / style gate
+ QA suites (`qa/`)
+ Request lifecycle
+ Adding a new external service integration
+ External HTTP layer (`external/`) — traced flow
+ Registry & system indices
+ semantic_text & the search path
+ Gotchas
+ Testing conventions specific to inference

Commands

0 shared · 0 only in A · 7 only in B

+ ./gradlew :x-pack:plugin:inference:test
+ ./gradlew :x-pack:plugin:inference:test --tests "org.elasticsearch.xpack.inference.ModelConfigurationsTests"
+ ./gradlew :x-pack:plugin:inference:test --tests "org.elasticsearch.xpack.inference.ModelConfigurationsTests.testSerialization"
+ ./gradlew :x-pack:plugin:inference:internalClusterTest
+ ./gradlew :x-pack:plugin:inference:yamlRestTest
+ ./gradlew :x-pack:plugin:inference:spotlessApply
+ ./gradlew :x-pack:plugin:inference:precommit

Section tags

1 shared · 0 only in A · 6 only in B

+ build
+ lint-format
+ code-style
+ testing-strategy
+ api
+ do-not
test

Line diff

+98 added−21 removed11 unchanged10.1% identical

elastic/elasticsearch · benchmarks/AGENTS.md

@@ −1 @@

1# Benchmarks

3## Running benchmarks

5Run from the `benchmarks/` directory using the `run` task with `--args`. Always use the fully-qualified class

6name including package to avoid ambiguity. Always pipe through `tee /tmp/bench/<descriptive_name>` using a filename that reflects the task (e.g. `tee /tmp/bench/paged_write`).

8```

9cd benchmarks

10../gradlew run --args "org.elasticsearch.benchmark._nightly.BytesBuilderBenchmark -pdata=1000_ints -pimpl=paged -poperation=write -rf json -rff build/jmh-result.json" | tee /tmp/bench/paged_write

11```

13## ColumNAR transform benchmarks

15```

16cd benchmarks

17../gradlew run --args="EncodeBlockTransformBenchmark" | tee /tmp/bench/encode_transform

18../gradlew run --args="DecodeBlockTransformBenchmark" | tee /tmp/bench/decode_transform

20# Single stage + pattern

21../gradlew run --args="EncodeBlockTransformBenchmark -p stage=splitDelta -p pattern=TSDB_SPLIT" | tee /tmp/bench/encode_splitdelta_tsdb

23# Quick smoke

24../gradlew run --args="EncodeBlockTransformBenchmark -wi 1 -i 1 -f 1 -w 1 -r 1 -p stage=delta -p pattern=MONOTONIC_TIMESTAMPS"

25```

27## Self-test

29Never skip the self-test. Do not pass `-DskipSelfTest=true` or `--test` to `run.sh`. The

30self-test validates correctness across all impl/operation/data combinations and poisons virtual

31dispatch to behave more like production.

elastic/elasticsearch · x-pack/plugin/inference/AGENTS.md

@@ +1 @@

1# Inference API plugin (x-pack-inference)

3Guidance for coding agents working in `x-pack/plugin/inference/` (Gradle `:x-pack:plugin:inference`, esplugin artifact `x-pack-inference`, class `InferencePlugin`). It implements the `_inference` API, integrates many external/internal inference services (see the provider subdirs under `services/`), and provides `semantic_text` and inference-based reranking. The repository-root `AGENTS.md` is authoritative for toolchain, formatting, logging, transport-version, and general testing conventions — this file does not repeat them. Read it first.

5## This plugin vs. the ML plugin — don't confuse them

7These two plugins both say "inference" but do different things:

9| | `x-pack/plugin/inference` (this plugin) | `x-pack/plugin/ml` |

10|---|---|---|

11| Scope | The `_inference` endpoint: register an *inference endpoint*, call out to a service, get embeddings/completions/reranks | Anomaly detection, datafeeds, data-frame analytics, **local trained-model deployment** (PyTorch via `ml-cpp`) |

12| Compute | Mostly **calls external HTTP APIs** (OpenAI, Cohere, Bedrock, …); the `elasticsearch` service runs models locally | Runs native processes on ML nodes |

13| Key feature | `semantic_text` field, semantic query, reranking | jobs, datafeeds, DFA, trained-model assignment |

15**The seam:** the internal `elasticsearch` service (`services/elasticsearch/`, e.g. ELSER and locally-hosted models) delegates to **ML-deployed trained models** — so a request through this plugin can end up running on the ML plugin's deployment infrastructure. The shared SPI lives in **server** (`org.elasticsearch.inference`); trained-model POJOs live in `xpack.core.ml.inference`.

17## Where the SPI lives (important)

19The core inference contracts are **not** in this plugin and **not** in xpack-core — they are in **server** at `server/src/main/java/org/elasticsearch/inference/`: `InferenceService`, `InferenceServiceRegistry`, `Model`, `ModelConfigurations`, `ServiceSettings`/`TaskSettings`/`SecretSettings`, `TaskType`, `InputType`, `InferenceServiceResults`, `ChunkingSettings`, `InferenceServiceExtension`. This plugin provides the *implementations*; other plugins integrate via `InferenceServiceExtension` (loaded through `ExtensiblePlugin`).

21## Build & Test Commands

23Gradle project `:x-pack:plugin:inference`. Run from repo root. `extendedPlugins = ['x-pack-core']`. Heavy third-party deps (AWS SDK v2, Google auth/api-client, Azure identity/msal4j, Nimbus OAuth/JOSE, Jackson, Netty, Reactor) — check `build.gradle` and the thirdPartyAudit allowances before adding more.

25```bash

26# Unit tests

27./gradlew :x-pack:plugin:inference:test

28./gradlew :x-pack:plugin:inference:test --tests "org.elasticsearch.xpack.inference.ModelConfigurationsTests"

29./gradlew :x-pack:plugin:inference:test --tests "org.elasticsearch.xpack.inference.ModelConfigurationsTests.testSerialization"

31# Internal cluster tests (*IT under ...inference.integration)

32./gradlew :x-pack:plugin:inference:internalClusterTest

34# YAML REST tests (specs in src/yamlRestTest/resources/rest-api-spec/test/inference/)

35./gradlew :x-pack:plugin:inference:yamlRestTest

37# Format / forbidden-API / style gate

38./gradlew :x-pack:plugin:inference:spotlessApply

39./gradlew :x-pack:plugin:inference:precommit

40```

42Unit-test base classes: `ESTestCase`, `AbstractWireSerializingTestCase`, `AbstractBWCWireSerializationTestCase`, and `MapperTestCase`/`MapperServiceTestCase` (for `semantic_text`).

44### QA suites (`qa/`)

46| Suite | Command | Purpose |

47|---|---|---|

48| `test-service-plugin` | (cluster plugin, consumed by others) | **Mock inference service** (`TestInferenceServicePlugin`) — the way to write service-level REST tests without real API keys |

49| `inference-service-tests` | `:qa:inference-service-tests:javaRestTest` | provides `InferenceBaseRestTest` (`putModel`/`infer`/`deleteModel` helpers) |

50| `inference-with-security` | `:qa:inference-with-security:yamlRestTest` | security enabled |

51| `multi-node` | `:qa:multi-node:yamlRestTest` | multi-node cluster |

52| `oauth2` | `:qa:oauth2:javaRestTest` | in-process mock OAuth2 server (skipped on FIPS) |

53| `mixed-cluster`, `rolling-upgrade` | `:qa:<name>:vX.Y.Z#javaRestTest` | BWC / rolling-upgrade |

55## Request lifecycle

57REST handler (`rest/`) → Transport action (`action/`, base `BaseTransportInferenceAction`, with license check) → `ModelRegistry` lookup of the endpoint config → `InferenceServiceRegistry` dispatch to the concrete service → service runs inference (external HTTP or local) → `InferenceServiceResults` back to the listener. Streaming uses the chunked/streaming variants.

59## Adding a new external service integration

611. Create `services/<name>/`. Implement the `Model` plus its `<Name>ServiceSettings` (non-secret config), `<Name>TaskSettings` (per-request knobs), and `<Name>SecretSettings` (API keys) — subtypes of the server SPI settings interfaces (base helpers in `services/settings/`).

622. Extend `SenderService` (`services/SenderService.java`, the base `InferenceService` impl) with a concrete `<Name>Service`; implement the `doInfer`/`doChunkedInfer`/`doUnifiedCompletionInfer` abstract methods.

633. Provide a `RequestManager` (builds the outbound HTTP request) and a service-specific `ResponseHandler` (parses the response into `InferenceServiceResults`).

644. Register the service factory in `InferencePlugin` and any `NamedWriteable`s in `InferenceNamedWriteablesProvider`.

655. Respect `ConfigurationParseContext` — `REQUEST` (strict, user API input) vs `PERSISTENT` (lenient, loaded from the index). Use `ServiceUtils`/`ServiceFields` helpers for map extraction and validation.

67## External HTTP layer (`external/`) — traced flow

69For services that call out over HTTP, a request flows:

711. `SenderService` calls `Sender.send(requestManager, inferenceInputs, timeout, listener)`.

722. `HttpRequestSender` lazily starts the async client/executor on first use, then enqueues to `RequestExecutorService`.

733. The executor applies **per-service rate limiting** and dequeues onto the `inference_utility` thread pool.

744. `RequestManager.buildRequest(...)` produces the outbound HTTP request.

755. `RetryingHttpSender` executes it with backoff on 429/5xx/timeouts.

766. `HttpClientManager` (Apache `HttpAsyncClient` connection pool, idle-eviction, XPack SSL) does the actual call.

777. The service's `ResponseHandler` parses the response into `InferenceServiceResults`, returned via the listener (responses delivered on the `inference_response` pool).

79## Registry & system indices

81`registry/ModelRegistry` persists endpoint configurations and syncs them from cluster state for fast `getModel(inferenceId)` lookups. Three system indices:

83- `InferenceIndex` → `.inference` — endpoint configs (service, task_type, service_settings, task_settings, chunking_settings). Root mappings are strict; the settings sub-objects are `dynamic:false` to allow service-specific fields.

84- `InferenceSecretsIndex` → `.inference-secrets` — credentials, kept in a **separate index** from config so secrets can be secured/backed-up independently.

85- An Elastic-Inference-Service cloud-connected-mode index for EIS auth config.

87## semantic_text & the search path

89`semantic_text` makes embedding generation automatic across ingest and search:

91- **Mapping:** `mapper/SemanticTextFieldMapper` defines the field and auto-creates dense- or sparse-vector subfields (plus chunk/text metadata) based on the referenced endpoint's task type.

92- **Ingest:** `action/filter/ShardBulkInferenceActionFilter` intercepts bulk shard requests, batches inference over the configured `inference_fields` (chunking long text, respecting `INDICES_INFERENCE_BATCH_SIZE`), and injects the embeddings into the document source before normal indexing.

93- **Search:** `queries/SemanticQueryBuilder` and the `Semantic{Knn,Match,SparseVector}QueryRewriteInterceptor`s auto-embed the query text at rewrite time and rewrite to native vector/sparse/match queries. `highlight/SemanticTextHighlighter` highlights matched chunks.

94- **Reranking:** `rank/textsimilarity/TextSimilarityRankBuilder` reranks first-pass hits through a rerank endpoint.

96## Gotchas

98- **Register on both sides.** A new service/model/results type must be registered as a service factory in `InferencePlugin` **and** as `NamedWriteable`(s) in `InferenceNamedWriteablesProvider`, or it silently fails to deserialize from cluster state / the index.

99- **Right parse context.** Honor `ConfigurationParseContext`: `REQUEST` (strict, rejects unknown fields — user API input) vs `PERSISTENT` (lenient — loaded from `.inference`). Using the wrong one breaks either client validation or forward-compat reads.

100- **Secrets stay separate.** Credentials are written to `.inference-secrets`, never `.inference`. Don't fold secret settings into the config document.

101- **SPI is in server.** The `InferenceService`/`Model`/settings/`TaskType` contracts live in server's `org.elasticsearch.inference`, not this plugin and not xpack-core — extend those, don't fork them.

102- **Don't block.** External calls go through the async `Sender`/`HttpRequestSender` path on the inference thread pools; don't call services synchronously from a transport or cluster-state thread.

103

104## Testing conventions specific to inference

105

106- Serialization round-trips use the wire/BWC bases; `semantic_text` mapper tests use `MapperTestCase`/`MapperServiceTestCase`.

107- To test service behavior end-to-end without real credentials, depend on `qa/test-service-plugin` (the mock `TestInferenceServicePlugin`) and extend `InferenceBaseRestTest` from `qa/inference-service-tests`.

108- Internal cluster tests are named `*IT` under `...inference.integration`; YAML specs live in `src/yamlRestTest/resources/rest-api-spec/test/inference/`.

109

@@ −1 +1 @@

1−# Benchmarks

1+# Inference API plugin (x-pack-inference)

3−## Running benchmarks

3+Guidance for coding agents working in `x-pack/plugin/inference/` (Gradle `:x-pack:plugin:inference`, esplugin artifact `x-pack-inference`, class `InferencePlugin`). It implements the `_inference` API, integrates many external/internal inference services (see the provider subdirs under `services/`), and provides `semantic_text` and inference-based reranking. The repository-root `AGENTS.md` is authoritative for toolchain, formatting, logging, transport-version, and general testing conventions — this file does not repeat them. Read it first.

5−Run from the `benchmarks/` directory using the `run` task with `--args`. Always use the fully-qualified class

6−name including package to avoid ambiguity. Always pipe through `tee /tmp/bench/<descriptive_name>` using a filename that reflects the task (e.g. `tee /tmp/bench/paged_write`).

5+## This plugin vs. the ML plugin — don't confuse them

8−```

9−cd benchmarks

10−../gradlew run --args "org.elasticsearch.benchmark._nightly.BytesBuilderBenchmark -pdata=1000_ints -pimpl=paged -poperation=write -rf json -rff build/jmh-result.json" | tee /tmp/bench/paged_write

11−```

7+These two plugins both say "inference" but do different things:

128

13−## ColumNAR transform benchmarks

9+| | `x-pack/plugin/inference` (this plugin) | `x-pack/plugin/ml` |

10+|---|---|---|

11+| Scope | The `_inference` endpoint: register an *inference endpoint*, call out to a service, get embeddings/completions/reranks | Anomaly detection, datafeeds, data-frame analytics, **local trained-model deployment** (PyTorch via `ml-cpp`) |

12+| Compute | Mostly **calls external HTTP APIs** (OpenAI, Cohere, Bedrock, …); the `elasticsearch` service runs models locally | Runs native processes on ML nodes |

13+| Key feature | `semantic_text` field, semantic query, reranking | jobs, datafeeds, DFA, trained-model assignment |

1414

15−```

16−cd benchmarks

17−../gradlew run --args="EncodeBlockTransformBenchmark" | tee /tmp/bench/encode_transform

18−../gradlew run --args="DecodeBlockTransformBenchmark" | tee /tmp/bench/decode_transform

15+**The seam:** the internal `elasticsearch` service (`services/elasticsearch/`, e.g. ELSER and locally-hosted models) delegates to **ML-deployed trained models** — so a request through this plugin can end up running on the ML plugin's deployment infrastructure. The shared SPI lives in **server** (`org.elasticsearch.inference`); trained-model POJOs live in `xpack.core.ml.inference`.

1916

20−# Single stage + pattern

21−../gradlew run --args="EncodeBlockTransformBenchmark -p stage=splitDelta -p pattern=TSDB_SPLIT" | tee /tmp/bench/encode_splitdelta_tsdb

17+## Where the SPI lives (important)

2218

23−# Quick smoke

24−../gradlew run --args="EncodeBlockTransformBenchmark -wi 1 -i 1 -f 1 -w 1 -r 1 -p stage=delta -p pattern=MONOTONIC_TIMESTAMPS"

19+The core inference contracts are **not** in this plugin and **not** in xpack-core — they are in **server** at `server/src/main/java/org/elasticsearch/inference/`: `InferenceService`, `InferenceServiceRegistry`, `Model`, `ModelConfigurations`, `ServiceSettings`/`TaskSettings`/`SecretSettings`, `TaskType`, `InputType`, `InferenceServiceResults`, `ChunkingSettings`, `InferenceServiceExtension`. This plugin provides the *implementations*; other plugins integrate via `InferenceServiceExtension` (loaded through `ExtensiblePlugin`).

20+

21+## Build & Test Commands

22+

23+Gradle project `:x-pack:plugin:inference`. Run from repo root. `extendedPlugins = ['x-pack-core']`. Heavy third-party deps (AWS SDK v2, Google auth/api-client, Azure identity/msal4j, Nimbus OAuth/JOSE, Jackson, Netty, Reactor) — check `build.gradle` and the thirdPartyAudit allowances before adding more.

24+

25+```bash

26+# Unit tests

27+./gradlew :x-pack:plugin:inference:test

28+./gradlew :x-pack:plugin:inference:test --tests "org.elasticsearch.xpack.inference.ModelConfigurationsTests"

29+./gradlew :x-pack:plugin:inference:test --tests "org.elasticsearch.xpack.inference.ModelConfigurationsTests.testSerialization"

30+

31+# Internal cluster tests (*IT under ...inference.integration)

32+./gradlew :x-pack:plugin:inference:internalClusterTest

33+

34+# YAML REST tests (specs in src/yamlRestTest/resources/rest-api-spec/test/inference/)

35+./gradlew :x-pack:plugin:inference:yamlRestTest

36+

37+# Format / forbidden-API / style gate

38+./gradlew :x-pack:plugin:inference:spotlessApply

39+./gradlew :x-pack:plugin:inference:precommit

2540 ```

2641

27−## Self-test

42+Unit-test base classes: `ESTestCase`, `AbstractWireSerializingTestCase`, `AbstractBWCWireSerializationTestCase`, and `MapperTestCase`/`MapperServiceTestCase` (for `semantic_text`).

2843

29−Never skip the self-test. Do not pass `-DskipSelfTest=true` or `--test` to `run.sh`. The

30−self-test validates correctness across all impl/operation/data combinations and poisons virtual

31−dispatch to behave more like production.

44+### QA suites (`qa/`)

45+

46+| Suite | Command | Purpose |

47+|---|---|---|

48+| `test-service-plugin` | (cluster plugin, consumed by others) | **Mock inference service** (`TestInferenceServicePlugin`) — the way to write service-level REST tests without real API keys |

49+| `inference-service-tests` | `:qa:inference-service-tests:javaRestTest` | provides `InferenceBaseRestTest` (`putModel`/`infer`/`deleteModel` helpers) |

50+| `inference-with-security` | `:qa:inference-with-security:yamlRestTest` | security enabled |

51+| `multi-node` | `:qa:multi-node:yamlRestTest` | multi-node cluster |

52+| `oauth2` | `:qa:oauth2:javaRestTest` | in-process mock OAuth2 server (skipped on FIPS) |

53+| `mixed-cluster`, `rolling-upgrade` | `:qa:<name>:vX.Y.Z#javaRestTest` | BWC / rolling-upgrade |

54+

55+## Request lifecycle

56+

57+REST handler (`rest/`) → Transport action (`action/`, base `BaseTransportInferenceAction`, with license check) → `ModelRegistry` lookup of the endpoint config → `InferenceServiceRegistry` dispatch to the concrete service → service runs inference (external HTTP or local) → `InferenceServiceResults` back to the listener. Streaming uses the chunked/streaming variants.

58+

59+## Adding a new external service integration

60+

61+1. Create `services/<name>/`. Implement the `Model` plus its `<Name>ServiceSettings` (non-secret config), `<Name>TaskSettings` (per-request knobs), and `<Name>SecretSettings` (API keys) — subtypes of the server SPI settings interfaces (base helpers in `services/settings/`).

62+2. Extend `SenderService` (`services/SenderService.java`, the base `InferenceService` impl) with a concrete `<Name>Service`; implement the `doInfer`/`doChunkedInfer`/`doUnifiedCompletionInfer` abstract methods.

63+3. Provide a `RequestManager` (builds the outbound HTTP request) and a service-specific `ResponseHandler` (parses the response into `InferenceServiceResults`).

64+4. Register the service factory in `InferencePlugin` and any `NamedWriteable`s in `InferenceNamedWriteablesProvider`.

65+5. Respect `ConfigurationParseContext` — `REQUEST` (strict, user API input) vs `PERSISTENT` (lenient, loaded from the index). Use `ServiceUtils`/`ServiceFields` helpers for map extraction and validation.

66+

67+## External HTTP layer (`external/`) — traced flow

68+

69+For services that call out over HTTP, a request flows:

70+

71+1. `SenderService` calls `Sender.send(requestManager, inferenceInputs, timeout, listener)`.

72+2. `HttpRequestSender` lazily starts the async client/executor on first use, then enqueues to `RequestExecutorService`.

73+3. The executor applies **per-service rate limiting** and dequeues onto the `inference_utility` thread pool.

74+4. `RequestManager.buildRequest(...)` produces the outbound HTTP request.

75+5. `RetryingHttpSender` executes it with backoff on 429/5xx/timeouts.

76+6. `HttpClientManager` (Apache `HttpAsyncClient` connection pool, idle-eviction, XPack SSL) does the actual call.

77+7. The service's `ResponseHandler` parses the response into `InferenceServiceResults`, returned via the listener (responses delivered on the `inference_response` pool).

78+

79+## Registry & system indices

80+

81+`registry/ModelRegistry` persists endpoint configurations and syncs them from cluster state for fast `getModel(inferenceId)` lookups. Three system indices:

82+

83+- `InferenceIndex` → `.inference` — endpoint configs (service, task_type, service_settings, task_settings, chunking_settings). Root mappings are strict; the settings sub-objects are `dynamic:false` to allow service-specific fields.

84+- `InferenceSecretsIndex` → `.inference-secrets` — credentials, kept in a **separate index** from config so secrets can be secured/backed-up independently.

85+- An Elastic-Inference-Service cloud-connected-mode index for EIS auth config.

86+

87+## semantic_text & the search path

88+

89+`semantic_text` makes embedding generation automatic across ingest and search:

90+

91+- **Mapping:** `mapper/SemanticTextFieldMapper` defines the field and auto-creates dense- or sparse-vector subfields (plus chunk/text metadata) based on the referenced endpoint's task type.

92+- **Ingest:** `action/filter/ShardBulkInferenceActionFilter` intercepts bulk shard requests, batches inference over the configured `inference_fields` (chunking long text, respecting `INDICES_INFERENCE_BATCH_SIZE`), and injects the embeddings into the document source before normal indexing.

93+- **Search:** `queries/SemanticQueryBuilder` and the `Semantic{Knn,Match,SparseVector}QueryRewriteInterceptor`s auto-embed the query text at rewrite time and rewrite to native vector/sparse/match queries. `highlight/SemanticTextHighlighter` highlights matched chunks.

94+- **Reranking:** `rank/textsimilarity/TextSimilarityRankBuilder` reranks first-pass hits through a rerank endpoint.

95+

96+## Gotchas

97+

98+- **Register on both sides.** A new service/model/results type must be registered as a service factory in `InferencePlugin` **and** as `NamedWriteable`(s) in `InferenceNamedWriteablesProvider`, or it silently fails to deserialize from cluster state / the index.

99+- **Right parse context.** Honor `ConfigurationParseContext`: `REQUEST` (strict, rejects unknown fields — user API input) vs `PERSISTENT` (lenient — loaded from `.inference`). Using the wrong one breaks either client validation or forward-compat reads.

100+- **Secrets stay separate.** Credentials are written to `.inference-secrets`, never `.inference`. Don't fold secret settings into the config document.

101+- **SPI is in server.** The `InferenceService`/`Model`/settings/`TaskType` contracts live in server's `org.elasticsearch.inference`, not this plugin and not xpack-core — extend those, don't fork them.

102+- **Don't block.** External calls go through the async `Sender`/`HttpRequestSender` path on the inference thread pools; don't call services synchronously from a transport or cluster-state thread.

103+

104+## Testing conventions specific to inference

105+

106+- Serialization round-trips use the wire/BWC bases; `semantic_text` mapper tests use `MapperTestCase`/`MapperServiceTestCase`.

107+- To test service behavior end-to-end without real credentials, depend on `qa/test-service-plugin` (the mock `TestInferenceServicePlugin`) and extend `InferenceBaseRestTest` from `qa/inference-service-tests`.

108+- Internal cluster tests are named `*IT` under `...inference.integration`; YAML specs live in `src/yamlRestTest/resources/rest-api-spec/test/inference/`.

32109