AGENTS.md

1# Test Data Manager - Agent Guidelines
2 
3Automated system for managing test data files across multiple test configurations.
4 
5## Module Overview
6 
7This module provides infrastructure for:
8- Comparing test outputs with expected files using variant chains
9- Automatic file management (creation, update, redundancy removal)
10- Test discovery, grouping, and conflict detection
11 
12**Structure:**
13- `testFixtures/` — Runtime API for use by other modules
14- `tests/` — Module's own test suite
15 
16For conceptual details (variant chains, conflicts, convergence), see [README.md](README.md).
17For running test data management tasks (checking/updating test data via Gradle), see [test-data-manager-convention](../../repo/gradle-build-conventions/test-data-manager-convention/README.md).
18 
19## Testing Guidelines (for tests within this module)
20 
21### Core Principles
22 
231. **Readable multi-line string expectations** — Format results as human-readable strings, compare with `assertEquals`
242. **Custom formatters** — Create formatters that produce deterministic, readable output
253. **Domain-specific assertion helpers** — Encapsulate complex assertions in named functions
264. **Descriptive test names** — Use backticks with clear descriptions
27 
28### Testing Patterns
29 
30#### Pattern 1: Readable Output Formatting
31 
32Create formatters that produce deterministic, human-readable output for complex results.
33 
34From `TestDiscoveryAndGroupingIntegrationTest.kt`:
35 
36```kotlin
37private fun formatResult(result: GroupingResult): String = buildString {
38    for (group in result.groups) {
39        val header = if (group.variantDepth == 0) "Group 0 (golden)" else "Group ${group.variantDepth}"
40        appendLine("=== $header ===")
41        for (test in group.tests.sortedBy { it.displayName }) {
42            appendLine("${test.displayName} -> ${test.variantChain}")
43        }
44        appendLine()
45    }
46}.trimEnd()
47 
48@Test
49fun `discovery finds all tests`() {
50    val result = runDiscovery()
51    assertEquals(expected.trimIndent(), formatResult(result))
52}
53```
54 
55#### Pattern 2: Domain-Specific Assertions
56 
57Encapsulate complex assertions in helper functions with clear names.
58 
59From `TestDataManagerGroupingTest.kt`:
60 
61```kotlin
62private fun assertGrouping(tests: List<DiscoveredTest>, expected: String) {
63    val result = groupByVariantDepth(tests)
64    val actual = result.groups.joinToString("\n") { group ->
65        "depth=${group.variantDepth}: ${group.uniqueVariantChains.joinToString(", ")}"
66    }
67    assertEquals(expected.trimIndent(), actual)
68}
69 
70private fun assertConflicts(tests: List<DiscoveredTest>, expected: String) {
71    val conflicts = validateConflicts(tests)
72    val actual = conflicts.joinToString("\n") {
73        "${it.chainA} vs ${it.chainB}: '${it.conflictingVariant}'"
74    }
75    assertEquals(expected.trimIndent(), actual)
76}
77 
78@Test
79fun `tests grouped by variant depth`() {
80    assertGrouping(
81        tests = listOf(
82            DiscoveredTest("1", "golden", emptyList()),
83            DiscoveredTest("2", "js", listOf("js")),
84        ),
85        expected = """
86            depth=0: []
87            depth=1: [js]
88        """
89    )
90}
91```
92 
93#### Pattern 3: State-Based Testing with Setup/Assert Helpers
94 
95For file-based operations, use setup and assertion helpers.
96 
97From `ManagedTestAssertionsTest.kt`:
98 
99```kotlin
100private fun assertFileState(expected: String) {
101    val actual = listOf("test.txt", "test.js.txt").mapNotNull { name ->
102        val file = tempDir.resolve(name)
103        if (file.exists()) "$name: ${file.readText().trim()}" else null
104    }.joinToString("\n")
105    assertEquals(expected.trimIndent(), actual)
106}
107 
108private fun setupFiles(vararg files: Pair<String, String>) {
109    for ((name, content) in files) {
110        tempDir.resolve(name).writeText("$content\n")
111    }
112}
113 
114@Test
115fun `UPDATE mode - mismatch updates file`() {
116    setupFiles("test.txt" to "old")
117    runAssertion(variantChain = emptyList(), actual = "new")
118    assertFileState("test.txt: new")
119}
120```
121 
122#### Pattern 4: Filter Testing with Base Class
123 
124For JUnit filter tests, extend `AbstractPostDiscoveryFilterTest`.
125 
126From `ManagedTestFilterTest.kt`:
127 
128```kotlin
129internal class ManagedTestFilterTest : AbstractPostDiscoveryFilterTest() {
130    @Test
131    fun `ClassSource with ManagedTest is included`() {
132        assertIncluded(
133            filter = ManagedTestFilter,
134            descriptor = descriptorFromClass<FakeGoldenAnalysisApiTestGenerated>(),
135        )
136    }
137 
138    @Test
139    fun `ClassSource without ManagedTest is excluded`() {
140        assertExcluded(
141            filter = ManagedTestFilter,
142            descriptor = descriptorFromClass<NoMetadataClass>(),
143        )
144    }
145}
146```
147 
148Available utilities from `AbstractPostDiscoveryFilterTest`:
149- `assertIncluded(filter, descriptor)` / `assertExcluded(filter, descriptor)`
150- `descriptorFromClass<T>()` — Create descriptor from class
151- `descriptorFromMethod(method)` — Create descriptor from method reference
152- `descriptorWithSource(source)` — Create descriptor with custom source
153 
154#### Pattern 5: Fake Test Classes for Integration Testing
155 
156Create fake test classes in `tests/.../fakes/` to simulate real test configurations.
157 
158```kotlin
159// Base class for all fakes
160abstract class FakeManagedTest : ManagedTest
161 
162// Golden test (no variant)
163@TestMetadata("testData/analysis/api")
164class FakeGoldenAnalysisApiTestGenerated : FakeManagedTest() {
165    override val variantChain = emptyList<String>()
166 
167    @Test
168    @TestMetadata("symbols.kt")
169    fun testSymbols() {}
170}
171 
172// Multi-level variant test
173@TestMetadata("testData/lightClasses")
174class FakeWasmLightClassesTestGenerated : FakeManagedTest() {
175    override val variantChain = listOf("knm", "wasm")
176 
177    @Test
178    @TestMetadata("simple.kt")
179    fun testSimple() {}
180}
181```
182 
183## Usage from Other Modules
184 
185### Implementing ManagedTest
186 
187Implement `ManagedTest` interface and provide variant chain:
188 
189```kotlin
190abstract class MyTestBase : ManagedTest {
191    override val variantChain: List<String>
192        get() = emptyList()
193}
194```
195 
196Variant chain rules:
197- `[]` (empty) — Golden/default configuration, writes to `.txt`
198- `["js"]` — Single variant, writes to `.js.txt`
199- `["knm", "wasm"]` — Multi-level variant, writes to `.wasm.txt` (last element only)
200 
201### Using Assertions
202 
203Use the extension function `ManagedTest.assertEqualsToTestDataFile()`  for comparing test output:
204 
205```kotlin
206class MyTest : ManagedTest {
207    override val variantChain = listOf("js")
208 
209    fun runTest(testDataFile: File) {
210        val actual = computeResult()
211        assertEqualsToTestDataFile(
212            testDataPath = testDataFile.toPath(),
213            actual = actual,
214            extension = ".txt",
215        )
216    }
217}
218```
219 
220Or use `ManagedTestAssertions.assertEqualsToTestDataFile()` directly:
221 
222```kotlin
223ManagedTestAssertions.assertEqualsToTestDataFile(
224    testDataPath = testDataFile.toPath(),
225    actual = actualContent,
226    variantChain = variantChain,
227    extension = ".txt",
228)
229```
230 
231 
232### Behavior Matrix
233 
234| Scenario                  | UPDATE mode | CHECK mode (local) | CHECK mode (CI) |
235|---------------------------|-------------|--------------------|-----------------|
236| actual=null, file missing | Pass        | Pass               | Pass            |
237| actual=null, file exists  | Delete      | Delete + throw     | Throw           |
238| File missing (golden)     | Create      | Create + throw     | Throw           |
239| File missing (secondary)  | Create      | Throw              | Throw           |
240| Content matches           | Pass        | Pass               | Pass            |
241| Write-target redundant    | Delete      | Delete + throw     | Throw           |
242| Content mismatch          | Update      | Throw              | Throw           |
243 
244## Key Classes Reference
245 
246| Class                    | Location             | Purpose                                          |
247|--------------------------|----------------------|--------------------------------------------------|
248| `ManagedTest`            | testFixtures         | Interface for tests managed by the system        |
249| `ManagedTestAssertions`  | testFixtures         | Assertion functions for test data comparison     |
250| `TestDataManagerRunner`  | testFixtures         | Main runner (discovery, grouping, execution)     |
251| `TestDataContext`        | testFixtures         | File path resolution and mode for variant chains |
252| `ManagedTestFilter`      | testFixtures/filters | JUnit filter for ManagedTest implementations     |
253| `TestMetadataFilter`     | testFixtures/filters | JUnit filter by @TestMetadata paths              |
254| `VariantChainComparator` | testFixtures         | Orders variant chains by depth                   |
255