Copilot instructions

1---
2description: "Guidance for GitHub Copilot when working with .NET MAUI integration tests"
3applyTo: "src/TestUtils/src/Microsoft.Maui.IntegrationTests/**"
4---
5 
6# .NET MAUI Integration Tests Guidelines
7 
8Integration tests validate end-to-end functionality by creating, building, and running .NET MAUI projects using templates and the local workload.
9 
10## Test Framework
11 
12These integration tests use **NUnit** as the test framework.
13 
14## CI Infrastructure
15 
16All integration tests run on **Azure DevOps agents** via the stage template `eng/pipelines/arcade/stage-integration-tests.yml`.
17 
18- **Windows tests**: Run on Windows 1ES pools
19- **macOS tests**: Run on Azure Pipelines hosted images
20  - General macOS tests: `macOS-15` (ARM64/Apple Silicon)
21  - `RunOnAndroid`: `macOS-15` (via MacOSPool)
22  - `RunOniOS_*`: Each iOS test runs in **two separate lanes**:
23    - **ARM64 lane** (`MacOSPoolArm64`): `macOS-15` with Apple Silicon
24    - **MacOSPool lane** (`MacOSPool`): For comparison testing
25 
26### Test Retry
27 
28Integration tests are configured with **automatic retry on failure** (`retryCountOnTaskFailure: 1`). If a test fails, Azure DevOps will automatically retry it once before reporting failure.
29 
30**Log Preservation**: Logs from each attempt are saved to separate folders (`attempt-1/`, `attempt-2/`, etc.) so that both the original run and retry logs are preserved for debugging. This uses the `SYSTEM_JOBATTEMPT` environment variable provided by Azure DevOps.
31 
32### Individual iOS Test Lanes
33 
34iOS tests are split into individual jobs for faster debugging and parallel execution. Each test runs on both ARM64 and MacOSPool for comparison:
35 
36| Test | Description | Timeout |
37|------|-------------|---------|
38| `RunOniOS_MauiDebug` | MAUI app, Debug config | 45 min |
39| `RunOniOS_MauiRelease` | MAUI app, Release config | 45 min |
40| `RunOniOS_MauiReleaseTrimFull` | MAUI app, Release, full trim | 45 min |
41| `RunOniOS_BlazorDebug` | Blazor app, Debug config | 45 min |
42| `RunOniOS_BlazorRelease` | Blazor app, Release config | 45 min |
43| `RunOniOS_MauiNativeAOT` | MAUI app, NativeAOT | 45 min |
44 
45**Note**: `RunOniOS_BlazorReleaseTrimFull` is temporarily disabled due to [ASP.NET Core issue #63951](https://github.com/dotnet/aspnetcore/issues/63951).
46 
47### MacOSPool Lane Conditions
48 
49The **MacOSPool lanes** (non-ARM64 comparison tests) only run under specific conditions:
50 
511. **Non-PR builds** on branches: `main`, `net*.0`, `release/*`, or `inflight/*`
522. **PR builds** where the target branch is: `net*.0`, `release/*`, or `inflight/*`
53 
54This ensures MacOSPool comparison tests don't run on regular PR builds targeting `main`, saving CI resources while still running them for release-related branches.
55 
56## Test Categories
57 
58Tests are organized by categories (defined in `Utilities/Categories.cs`) that map to CI jobs:
59 
60| Category | Purpose | Platform |
61|----------|---------|----------|
62| `Build` | Basic template build tests | All |
63| `WindowsTemplates` | Windows-specific scenarios | Windows |
64| `macOSTemplates` | macOS-specific scenarios | macOS |
65| `Blazor` | Blazor hybrid templates | All |
66| `MultiProject` | Multi-project templates | All |
67| `AOT` | Native AOT compilation | macOS |
68| `RunOnAndroid` | Build, install, run on Android emulator | macOS |
69| `RunOniOS` | iOS simulator tests (class-level category on `AppleTemplateTests`) | macOS |
70| `Samples` | Sample project builds | All |
71 
72**Note**: The `RunOniOS` category is applied at the **class level** on `AppleTemplateTests`. Individual test methods don't need their own category attribute.
73 
74## Writing Integration Tests
75 
76### Basic Pattern
77 
78```csharp
79[Test]
80[Category(Categories.Build)]
81[TestCase("maui", DotNetCurrent, "Debug")]
82[TestCase("maui", DotNetCurrent, "Release")]
83public void Build(string id, string framework, string config)
84{
85    var projectDir = TestDirectory;
86    var projectFile = Path.Combine(projectDir, $"{Path.GetFileName(projectDir)}.csproj");
87 
88    // Create from template
89    Assert.IsTrue(DotnetInternal.New(id, projectDir, framework),
90        $"Unable to create template {id}.");
91 
92    // Build
93    Assert.IsTrue(DotnetInternal.Build(projectFile, config, properties: BuildProps),
94        $"Project failed to build.");
95}
96```
97 
98### Device Test Pattern (iOS)
99 
100```csharp
101[Test]
102[Category(Categories.RunOniOS)]
103[TestCase("maui", "Release", DotNetCurrent, RuntimeVariant.Mono, null)]
104public void RunOniOS(string id, string config, string framework, RuntimeVariant rv, string? trimMode)
105{
106    // Create and build
107    Assert.IsTrue(DotnetInternal.New(id, projectDir, framework));
108    Assert.IsTrue(DotnetInternal.Build(projectFile, config, 
109        framework: $"{framework}-ios",
110        properties: buildProps,
111        runtimeIdentifier: TestEnvironment.IOSSimulatorRuntimeIdentifier));
112 
113    // Run with XHarness (omit UDID to let XHarness control simulator lifecycle)
114    Assert.IsTrue(XHarness.RunAppleForTimeout(appPath, resultDir, TestSimulator.XHarnessID));
115}
116```
117 
118### Platform Guards
119 
120```csharp
121if (!TestEnvironment.IsMacOS)
122    Assert.Ignore("Running Apple templates is only supported on macOS.");
123 
124if (!TestEnvironment.IsWindows)
125    Assert.Ignore("Running Windows templates is only supported on Windows.");
126```
127 
128## Key Classes
129 
130| Class | Purpose |
131|-------|---------|
132| `BaseBuildTest` | Base class with `TestDirectory`, `BuildProps`, lifecycle methods |
133| `BaseTemplateTests` | Extends BaseBuildTest with template-specific setup |
134| `DotnetInternal` | Wraps `dotnet` CLI using local SDK (`.dotnet/dotnet`) |
135| `XHarness` | Runs apps on devices/simulators |
136| `Simulator` | iOS simulator management (boot, shutdown, UDID) |
137| `TestEnvironment` | Platform detection, paths, `IOSSimulatorRuntimeIdentifier` |
138| `FileUtilities` | File manipulation helpers |
139 
140## Template IDs
141 
142- `maui` - .NET MAUI App
143- `maui-blazor` - .NET MAUI Blazor Hybrid App
144- `maui-blazor-web` - .NET MAUI Blazor Web Solution
145- `mauilib` - .NET MAUI Class Library
146- `maui-multiproject` - .NET MAUI Multi-Project App
147 
148## Running Tests Locally
149 
150### 🚨 ALWAYS Use the Skill
151 
152**When asked to run integration tests, ALWAYS use the `run-integration-tests` skill:**
153 
154```powershell
155# macOS examples
156pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "macOSTemplates" -SkipBuild -SkipInstall -SkipXcodeVersionCheck
157pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "RunOniOS" -SkipBuild -SkipInstall -SkipXcodeVersionCheck
158pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "RunOnAndroid" -SkipBuild -SkipInstall
159 
160# Windows examples
161pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "WindowsTemplates" -SkipBuild -SkipInstall
162pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -SkipBuild -SkipInstall
163```
164 
165The skill handles:
166- ✅ Environment variable setup (`MAUI_PACKAGE_VERSION`, `SKIP_XCODE_VERSION_CHECK`)
167- ✅ Cross-platform support (Windows and macOS)
168- ✅ Test results in TRX format
169- ✅ Proper error reporting
170 
171See `.github/skills/run-integration-tests/SKILL.md` for full documentation.
172 
173---
174 
175### Prerequisites (Manual Setup)
176 
177If the skill reports missing prerequisites, provision the local SDK:
178 
1791. **Provision the local SDK and workloads** - The `.dotnet/` folder must contain a fully provisioned .NET SDK with MAUI workloads. Run:
180 
181```bash
182   # Step 0: Restore repo-local tools (Cake, etc.) from .config/dotnet-tools.json
183   dotnet tool restore
184 
185   # Step 1: Download the .NET SDK (creates .dotnet/dotnet binary)
186   dotnet cake --target=dotnet
187 
188   # Step 2: Install MAUI workloads into the local SDK (takes ~5 minutes)
189   dotnet cake --target=dotnet-local-workloads
190```
191 
192   **Verification**: After provisioning, verify the setup:
193```bash
194   # Check dotnet binary exists
195   ls .dotnet/dotnet
196   
197   # Check MAUI workloads are installed
198   ls .dotnet/packs/Microsoft.Maui.Sdk
199```
200 
201### Environment Variables (Reference)
202 
203The skill sets these automatically, but for manual runs:
204 
205| Variable | Required | Purpose |
206|----------|----------|---------|
207| `MAUI_PACKAGE_VERSION` | Yes | Version of MAUI packages being tested |
208| `IOS_TEST_DEVICE` | No | iOS simulator target (e.g., `ios-simulator-64_18.5`) |
209| `SKIP_XCODE_VERSION_CHECK` | No | Set to `true` to bypass Xcode version validation |
210 
211### Manual Run Commands (Fallback Only)
212 
213**⚠️ Only use these if the skill is unavailable:**
214 
215```bash
216# Set environment first
217export MAUI_PACKAGE_VERSION=$(ls .dotnet/packs/Microsoft.Maui.Sdk | head -1)
218export SKIP_XCODE_VERSION_CHECK=true
219 
220# Run specific category
221dotnet test src/TestUtils/src/Microsoft.Maui.IntegrationTests \
222  --filter "Category=Build"
223 
224# Run specific test
225dotnet test src/TestUtils/src/Microsoft.Maui.IntegrationTests \
226  --filter "FullyQualifiedName~AppleTemplateTests.RunOniOS"
227```
228 
229## Common Pitfalls
230 
231| Issue | Solution |
232|-------|----------|
233| Template not found | Verify workloads installed, `MAUI_PACKAGE_VERSION` set |
234| Xcode version mismatch | Set `SKIP_XCODE_VERSION_CHECK=true` |
235| Device/simulator not found | Verify `IOS_TEST_DEVICE` or emulator is running |
236| XHarness timeout on iOS | Expected behavior - app runs until timeout |
237| Architecture mismatch | Use `TestEnvironment.IOSSimulatorRuntimeIdentifier` |
238 
239## Best Practices
240 
241### DO
242- Use `BuildProps` from base class for isolation
243- Apply `[Category]` at the **class level** when all tests share the same category
244- Check platform with `TestEnvironment.Is*` before platform-specific tests
245- Use `TestEnvironment.IOSSimulatorRuntimeIdentifier` for iOS builds
246- Include meaningful assertion messages
247 
248### DON'T
249- Hardcode paths - use `TestDirectory`, `TestEnvironment` helpers
250- Add `[Category]` to each test method when the class has a category
251- Skip platform guards for platform-specific tests
252- Hardcode iOS runtime identifiers (arm64 vs x64)
253