Copilot instructions

1---
2applyTo: ".github/**/*.{yml,yaml}"
3---
4 
5# Publishing Pester Test Results Instructions
6 
7This document describes how the PowerShell repository uses GitHub Actions to publish Pester test results.
8 
9## Overview
10 
11The PowerShell repository uses a custom composite GitHub Action located at `.github/actions/test/process-pester-results` to process and publish Pester test results in CI/CD workflows.
12This action aggregates test results from NUnitXml formatted files, creates a summary in the GitHub Actions job summary, and uploads the results as artifacts.
13 
14## How It Works
15 
16### Action Location and Structure
17 
18**Path**: `.github/actions/test/process-pester-results/`
19 
20The action consists of two main files:
21 
221. **action.yml** - The composite action definition
231. **process-pester-results.ps1** - PowerShell script that processes test results
24 
25### Action Inputs
26 
27The action accepts the following inputs:
28 
29- **name** (required): A descriptive name for the test run (e.g., "UnelevatedPesterTests-CI")
30    - Used for naming the uploaded artifact and in the summary
31    - Format: `junit-pester-{name}`
32 
33- **testResultsFolder** (optional): Path to the folder containing test result XML files
34    - Default: `${{ runner.workspace }}/testResults`
35    - The script searches for all `*.xml` files in this folder recursively
36 
37### Action Workflow
38 
39The action performs the following steps:
40 
411. **Process Test Results**
42    - Runs `process-pester-results.ps1` with the provided name and test results folder
43    - Parses all NUnitXml formatted test result files (`*.xml`)
44    - Aggregates test statistics across all files:
45        - Total test cases
46        - Errors
47        - Failures
48        - Not run tests
49        - Inconclusive tests
50        - Ignored tests
51        - Skipped tests
52        - Invalid tests
53 
541. **Generate Summary**
55    - Creates a markdown summary using the `$GITHUB_STEP_SUMMARY` environment variable
56    - Uses `Write-Log` and `Write-LogGroupStart`/`Write-LogGroupEnd` functions from `build.psm1`
57    - Outputs a formatted summary with all test statistics
58    - Example format:
59 
60```markdown
61    # Summary of {Name}
62 
63    - Total Tests: X
64    - Total Errors: X
65    - Total Failures: X
66    - Total Not Run: X
67    - Total Inconclusive: X
68    - Total Ignored: X
69    - Total Skipped: X
70    - Total Invalid: X
71```
72 
731. **Upload Artifacts**
74    - Uses `actions/upload-artifact@v4` to upload test results
75    - Artifact name: `junit-pester-{name}`
76    - Always runs (even if previous steps fail) via `if: always()`
77    - Uploads the entire test results folder
78 
791. **Exit Status**
80    - Fails the job (exit 1) if:
81        - Any test errors occurred (`$testErrorCount -gt 0`)
82        - Any test failures occurred (`$testFailureCount -gt 0`)
83        - No test cases were run (`$testCaseCount -eq 0`)
84 
85## Usage in Test Actions
86 
87The `process-pester-results` action is called by two platform-specific composite test actions:
88 
89### Linux/macOS Tests: `.github/actions/test/nix`
90 
91Used in:
92 
93- `.github/workflows/linux-ci.yml`
94- `.github/workflows/macos-ci.yml`
95 
96Example usage (lines 99-104 in `nix/action.yml`):
97 
98```yaml
99- name: Convert, Publish, and Upload Pester Test Results
100  uses: "./.github/actions/test/process-pester-results"
101  with:
102    name: "${{ inputs.purpose }}-${{ inputs.tagSet }}"
103    testResultsFolder: "${{ runner.workspace }}/testResults"
104```
105 
106### Windows Tests: `.github/actions/test/windows`
107 
108Used in:
109 
110- `.github/workflows/windows-ci.yml`
111 
112Example usage (line 78-83 in `windows/action.yml`):
113 
114```yaml
115- name: Convert, Publish, and Upload Pester Test Results
116  uses: "./.github/actions/test/process-pester-results"
117  with:
118    name: "${{ inputs.purpose }}-${{ inputs.tagSet }}"
119    testResultsFolder: ${{ runner.workspace }}\testResults
120```
121 
122## Workflow Integration
123 
124The process-pester-results action is integrated into the CI workflows through a multi-level hierarchy:
125 
126### Level 1: Main CI Workflows
127 
128- `linux-ci.yml`
129- `macos-ci.yml`
130- `windows-ci.yml`
131 
132### Level 2: Test Jobs
133 
134Each workflow contains multiple test jobs with different purposes and tag sets:
135 
136- `UnelevatedPesterTests` with tagSet `CI`
137- `ElevatedPesterTests` with tagSet `CI`
138- `UnelevatedPesterTests` with tagSet `Others`
139- `ElevatedPesterTests` with tagSet `Others`
140 
141### Level 3: Platform Test Actions
142 
143Test jobs use platform-specific actions:
144 
145- `nix` for Linux and macOS
146- `windows` for Windows
147 
148### Level 4: Process Results Action
149 
150Platform actions call `process-pester-results` to publish results
151 
152## Test Execution Flow
153 
1541. **Build Phase**: Source code is built (e.g., in `ci_build` job)
1551. **Test Preparation**:
156    - Build artifacts are downloaded
157    - PowerShell is bootstrapped
158    - Test binaries are extracted
1591. **Test Execution**:
160    - `Invoke-CITest` is called with:
161        - `-Purpose`: Test purpose (e.g., "UnelevatedPesterTests")
162        - `-TagSet`: Test category (e.g., "CI", "Others")
163        - `-OutputFormat NUnitXml`: Results format
164    - Results are written to `${{ runner.workspace }}/testResults`
1651. **Results Processing**:
166    - `process-pester-results` action runs
167    - Results are aggregated and summarized
168    - Artifacts are uploaded
169    - Job fails if any tests failed or errored
170 
171## Key Dependencies
172 
173### PowerShell Modules
174 
175- **build.psm1**: Provides utility functions
176    - `Write-Log`: Logging function with GitHub Actions support
177    - `Write-LogGroupStart`: Creates collapsible log groups
178    - `Write-LogGroupEnd`: Closes collapsible log groups
179 
180### GitHub Actions Features
181 
182- **GITHUB_STEP_SUMMARY**: Environment variable for job summary
183- **actions/upload-artifact@v4**: For uploading test results
184- **Composite Actions**: For reusable workflow steps
185 
186### Test Result Format
187 
188- **NUnitXml**: XML format for test results
189- Expected XML structure with `test-results` root element containing:
190    - `total`: Total number of tests
191    - `errors`: Number of errors
192    - `failures`: Number of failures
193    - `not-run`: Number of tests not run
194    - `inconclusive`: Number of inconclusive tests
195    - `ignored`: Number of ignored tests
196    - `skipped`: Number of skipped tests
197    - `invalid`: Number of invalid tests
198 
199## Best Practices
200 
2011. **Naming Convention**: Use descriptive names that include both purpose and tagSet:
202    - Format: `{purpose}-{tagSet}`
203    - Example: `UnelevatedPesterTests-CI`
204 
2051. **Test Results Location**:
206    - Default location: `${{ runner.workspace }}/testResults`
207    - Use platform-appropriate path separators (Windows: `\`, Unix: `/`)
208 
2091. **Always Upload**: The artifact upload step uses `if: always()` to ensure results are uploaded even when tests fail
210 
2111. **Error Handling**: The action will fail the job if:
212    - Tests have errors or failures (intentional fail-fast behavior)
213    - No tests were executed (potential configuration issue)
214    - `GITHUB_STEP_SUMMARY` is not set (environment issue)
215 
216## Customizing for Your Repository
217 
218To use this pattern in another repository:
219 
2201. **Copy the Action Files**:
221    - Copy `.github/actions/test/process-pester-results/` directory
222    - Ensure the PowerShell script has proper permissions
223 
2241. **Adjust Dependencies**:
225    - Modify or remove the `Import-Module "$PSScriptRoot/../../../../build.psm1"` line
226    - Implement equivalent `Write-Log` and `Write-LogGroup*` functions if needed
227 
2281. **Customize Summary Format**:
229    - Modify the here-string in `process-pester-results.ps1` to change summary format
230    - Add additional metrics or formatting as needed
231 
2321. **Call from Your Workflows**:
233 
234```yaml
235    - name: Process Test Results
236      uses: "./.github/actions/test/process-pester-results"
237      with:
238        name: "my-test-run"
239        testResultsFolder: "path/to/results"
240```
241 
242## Related Documentation
243 
244- [GitHub Actions: Creating composite actions](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action)
245- [GitHub Actions: Job summaries](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#adding-a-job-summary)
246- [GitHub Actions: Uploading artifacts](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts)
247- [Pester: PowerShell testing framework](https://pester.dev/)
248- [NUnit XML Format](https://docs.nunit.org/articles/nunit/technical-notes/usage/Test-Result-XML-Format.html)
249 
250## Troubleshooting
251 
252### No Test Results Found
253 
254- Verify `testResultsFolder` path is correct
255- Ensure tests are generating NUnitXml formatted output
256- Check that `*.xml` files exist in the specified folder
257 
258### Action Fails with "GITHUB_STEP_SUMMARY is not set"
259 
260- Ensure the action runs within a GitHub Actions environment
261- Cannot be run locally without mocking this environment variable
262 
263### All Tests Pass but Job Fails
264 
265- Check if any tests are marked as errors (different from failures)
266- Verify that at least some tests executed (`$testCaseCount -eq 0`)
267 
268### Artifact Upload Fails
269 
270- Check artifact name for invalid characters
271- Ensure the test results folder exists
272- Verify actions/upload-artifact version compatibility
273