CLAUDE.md

1# ICSharpCode.Decompiler.Tests guide
2 
3How the decompiler test suite is structured, what each test kind does, and how to add tests.
4 
5## The matrix-testing model
6 
7Most fixtures run one logical test against a whole matrix of compilers and options: an NUnit
8`[Test]` method takes a `CompilerOptions` parameter fed by `[ValueSource]` from static config
9arrays declared per runner (`defaultOptions` including mcs, `roslynOnlyOptions`,
10`roslyn2OrNewerOptions`, `roslyn3OrNewerOptions`, `roslyn4OrNewerOptions`, each also in a
11`...WithNet40Options` variant). One test method therefore becomes 4-26 test cases.
12 
13`CompilerOptions` flags (see `Helpers/Tester.cs`) select:
14- the compiler: legacy csc (`None`), `UseRoslyn1_3_2`, `UseRoslyn2_10_0`, `UseRoslyn3_11_0`,
15  `UseRoslyn4_14_0`, `UseRoslynLatest` (version comes from `RoslynVersion` in
16  `Directory.Packages.props`), or `UseMcs2_6_4`/`UseMcs5_23`
17- the target: `TargetNet40` (compiles against .NET Framework reference assemblies from the
18  `ILSpy-tests` submodule) vs. .NET Core reference packs (net5.0 for Roslyn 3, current preview
19  for Roslyn 4/latest)
20- codegen options: `Optimize` (`-o+`, defines `OPT`), `UseDebug`, `Force32Bit`, `Library`,
21  `GeneratePdb`, `NullableEnable`, `CheckForOverflowUnderflow`, ...
22 
23Compiled artifacts are named `<TestName><suffix>.exe|dll` where the suffix encodes the config
24(`Tester.GetSuffix`, e.g. `.opt.roslyn3.net40`). By default they land next to the test case
25sources; set `TestsAssemblyTempPath` in `DecompilerTests.config.json` to redirect them
26(`Helpers/TestsAssemblyOutput.cs`).
27 
28First test run on a machine: the `[SetUpFixture]` in `TestTraceListener.cs` calls
29`Tester.Initialize()`, which downloads the Roslyn toolsets, vswhere, and the reference-assembly
30packs from NuGet (network required; cached under the test output directory afterwards) and
31builds the self-contained `ICSharpCode.Decompiler.TestRunner`. Package downloads check the
32`ILSpy-tests/nuget` folder first, so the `ILSpy-tests` submodule must be initialized (see the
33root `CLAUDE.md` section on the submodule).
34 
35## Test kinds
36 
37| Kind | Runner / fixture dir (`TestCases/...`) | Pipeline | Compared against |
38|---|---|---|---|
39| Pretty | `PrettyTestRunner` / `Pretty/*.cs` | compile -> decompile | the test source itself |
40| Correctness | `CorrectnessTestRunner` / `Correctness/*.{cs,vb,il}` | compile -> decompile -> recompile -> execute both | runtime output (stdout/stderr/exit code) of original vs. re-compiled |
41| ILPretty | `ILPrettyTestRunner` / `ILPretty/*.il` | ilasm -> decompile | sibling `.cs` file |
42| Ugly | `UglyTestRunner` / `Ugly/*.cs` | compile -> decompile with sugar settings disabled | sibling `.Expected.cs` file |
43| Disassembler | `DisassemblerPrettyTestRunner` / `Disassembler/Pretty/*.il` | ilasm -> disassemble with our `ReflectionDisassembler` | the `.il` source (or `.expected.il`, e.g. `SortedOutput`) |
44| VBPretty | `VBPrettyTestRunner` / `VBPretty/*.vb` | vbc -> decompile to C# | sibling `.cs` file |
45| PdbGen | `PdbGenerationTestRunner` / `PdbGen/*.cs` | in-proc Roslyn compile (real PDB = oracle), decompile, generate portable PDB with `PortablePdbWriter`, parse both PDBs' sequence-point blobs | the compiler's PDB, projected to the visible breakpoint map (see below) |
46| Roundtrip | `RoundtripAssembly` (inputs from `ILSpy-tests/`) | whole-project decompile -> MSBuild rebuild -> run original NUnit tests against the rebuilt assembly | test-run success |
47| Unit tests | `TypeSystem/`, `Semantics/`, `Output/`, `Util/`, `DataFlowTest`, `Metadata/`, `ProjectDecompiler/` | plain in-process NUnit | assertions |
48 
49## How to add a test
50 
51Common to the file-based kinds: every file in the fixture directory must have a matching test
52method - each runner has an `AllFilesHaveTests` test that fails otherwise. The method name must
53equal the file name (minus extension); it usually just calls the runner's `Run`/`RunForLibrary`
54helper, which picks the file via `[CallerMemberName]`.
55 
56- **Pretty** (decompiler produces nice code): add `TestCases/Pretty/MyTest.cs` plus a test
57  method choosing the narrowest sensible config group (e.g. C# 8 features need
58  `roslyn3OrNewerOptions`). The file is simultaneously input and expected output, so write it
59  exactly as ILSpy pretty-prints (tabs, `switch {` on the same line, trailing commas, ...).
60  Iterate by running the fixture and adjusting the file to the diff.
61- **Correctness** (decompiled code behaves identically): add
62  `TestCases/Correctness/MyTest.cs` with a `Main` that prints observable state; the harness
63  compiles it, decompiles, re-compiles the decompiled output, executes both, and diffs the
64  output streams. Roslyn-non-net40 configs execute through
65  `ICSharpCode.Decompiler.TestRunner` (an `AssemblyLoadContext` host); other configs run the
66  exe directly.
67- **Ugly** (output with decompiler features switched off still compiles/behaves): add
68  `MyTest.cs` plus the expected decompilation as `MyTest.Expected.cs`.
69- **ILPretty / Disassembler**: add a `.il` file (assembled with the NuGet ilasm) and the
70  expected `.cs` (`ILPretty`) or rely on round-tripping the `.il` itself (`Disassembler`).
71- **VBPretty**: add `MyTest.vb` and the expected C# decompilation `MyTest.cs`.
72- **PdbGen** (the reconstructed PDB's breakpoints match the C# compiler's): add `MyTest.cs`,
73  written exactly as ILSpy pretty-prints a *single type* (no assembly-attribute header - the
74  same discipline as Pretty). The runner compiles it with Roslyn (whose PDB is the oracle),
75  decompiles, reconstructs a PDB, and compares the **breakpoint map**: per method, the
76  ordered source locations of visible sequence points and the placement of hidden sequence
77  points anchored to neighboring source locations. IL offsets, local scopes and the embedded
78  source are all dropped, because the decompiler reconstructs them differently and they never
79  match byte-for-byte. The comparer also runs an oracle-free
80  well-formedness check (strictly increasing IL offsets = no duplicate/overlapping points).
81  `TestSequencePoints()` asserts the map matches the compiler exactly; for the handful of methods
82  where the decompiler legitimately diverges (e.g. it breakpoints a method's opening brace where
83  the compiler keeps it hidden), call `TestSequencePoints(knownResidual: true)`; the residual is
84  auto-derived and committed as `MyTest.residual.txt`, so improvements and regressions both flip
85  the test like a pretty diff. On a mismatch the test writes `MyTest.residual.txt.generated` and
86  fails; accept a deliberate change by re-running with `ILSPY_ACCEPT_PDB_RESIDUAL=1` set (which
87  overwrites the snapshot in place) or by copying the `.generated` file over it. `Tolerance.Lines` drops column comparison
88  for statements whose column placement differs. Nothing is hand-maintained except the `.cs`
89  source and, rarely, the residual snapshot. No `.expected.*` is committed (all regenerated).
90 
91## Conditional expectations (#if) and comparison rules
92 
93Different configs legitimately produce different decompilations. Pretty-style comparisons parse
94both sides with Roslyn using the config's preprocessor symbols and delete inactive `#if`
95regions (`Helpers/CodeAssert.cs`), so test sources can branch on:
96- `OPT` (optimized build), `EXPECTED_OUTPUT` (defined only while comparing, never while
97  compiling - use it for "what ILSpy prints" vs. "equivalent compilable input" differences)
98- compiler family/version: `LEGACY_CSC`, `LEGACY_VBC`, `MCS`, `MCS2`, `MCS5`, `ROSLYN`,
99  `ROSLYN2`, `ROSLYN3`, `ROSLYN4`
100- language version: `CS60` ... `CS130`, `VB11` ... `VB16`
101- target framework: `NET40`, `NETCORE`, `NET50` ... `NET100`
102 
103Normalization before diffing: lines are trimmed, `//` comments stripped, lines starting with
104`#` ignored (so `#pragma`/`#region` in test sources are harmless), blank lines ignored.
105Everything else must match exactly.
106 
107## Probing compiler codegen
108 
109To learn how every supported compiler/option combination lowers a construct, add a test case
110exercising it and run the fixture: the harness automatically compiles it with all configured
111compiler versions and settings, and each failing config's diff shows you what the decompiler
112produced for that compiler's IL. This beats hand-running csc versions.
113 
114## Running
115 
116```
117dotnet test --solution ILSpy.sln --report-trx --filter FullyQualifiedName~PrettyTestRunner.SwitchExpressions
118```
119 
120(Microsoft.Testing.Platform syntax; see root `CLAUDE.md` "Test discipline".) A failing
121comparison prints an aligned diff with ` + `/` - ` markers. On failure the decompiled output
122file is left on disk for inspection (Correctness failures print its path; output diffs are
123also written to `%TEMP%/<test>.original.out` / `.decompiled.out`).
124