Copilot instructions

1---
2applyTo: "**/*.Tests.ps1"
3---
4 
5# Pester Test Status Meanings and Working Tests
6 
7## Purpose
8 
9This guide clarifies Pester test outcomes and what it means for a test to be "working" - which requires both **passing** AND **actually validating functionality**.
10 
11## Test Statuses in Pester
12 
13### Passed ✓
14**Status Code**: `Passed` 
15**Exit Result**: Test ran successfully, all assertions passed
16 
17**What it means**:
18- Test executed without errors
19- All `Should` statements evaluated to true
20- Test setup and teardown completed without issues
21- Test is **validating** the intended functionality
22 
23**What it does NOT mean**:
24- The feature is working (assertions could be wrong)
25- The test is meaningful (could be testing wrong thing)
26- The test exercises all code paths
27 
28### Failed ✗
29**Status Code**: `Failed`
30**Exit Result**: Test ran but assertions failed
31 
32**What it means**:
33- Test executed but an assertion returned false
34- Expected value did not match actual value
35- Test detected a problem with the functionality
36 
37**Examples**:
38```
39Expected $true but got $false
40Expected 5 items but got 3
41Expected no error but got: Cannot find parameter
42```
43 
44### Error ⚠
45**Status Code**: `Error`
46**Exit Result**: Test crashed with an exception
47 
48**What it means**:
49- Test failed to complete
50- An exception was thrown during test execution
51- Could be in test setup, test body, or test cleanup
52- Often indicates environmental issue, not code functional issue
53 
54**Examples**:
55```
56Cannot bind argument to parameter 'Path' because it is null
57File not found: C:\expected\config.json
58Access denied writing to registry
59```
60 
61### Pending ⏳
62**Status Code**: `Pending`
63**Exit Result**: Test ran but never completed assertions
64 
65**What it means**:
66- Test was explicitly marked as not ready to run
67- `Set-ItResult -Pending` was called
68- Used to indicate: known bugs, missing features, environmental issues
69 
70**When to use Pending**:
71- Test for feature in development
72- Test disabled due to known bug (issue #1234)
73- Test disabled due to intermittent failures being fixed
74- Platform-specific issues being resolved
75 
76**⚠️ WARNING**: Pending tests are NOT validating functionality. They hide problems.
77 
78### Skipped ⊘
79**Status Code**: `Skipped`  
80**Exit Result**: Test did not run (detected at start)
81 
82**What it means**:
83- Test was intentionally not executed
84- `-Skip` parameter or `It -Skip:$condition` was used
85- Environment doesn't support this test
86 
87**When to use Skip**:
88- Test not applicable to current platform (Windows-only test on Linux)
89- Test requires feature that's not available (admin privileges)
90- Test requires specific configuration not present
91 
92**Difference from Pending**:
93- **Skip**: "This test shouldn't run here" (known upfront)
94- **Pending**: "This test should eventually run but can't now"
95 
96### Ignored ✛
97**Status Code**: `Ignored`
98**Exit Result**: Test marked as not applicable
99 
100**What it means**:
101- Test has `[Ignore("reason")]` attribute
102- Test is permanently disabled in this location
103- Not the same as Skipped (which is conditional)
104 
105**When to use Ignore**:
106- Test for deprecated feature
107- Test for bug that won't be fixed
108- Test moved to different test file
109 
110---
111 
112## What Does "Working" Actually Mean?
113 
114A test is **working** when it meets BOTH criteria:
115 
116### 1. **Test Status is PASSED** ✓
117```powershell
118It "Test name" {
119    # Test executes
120    # All assertions pass
121    # Returns Passed status
122}
123```
124 
125### 2. **Test Actually Validates Functionality**
126```powershell
127# ✓ GOOD: Tests actual functionality
128It "Get-Item returns files from directory" -Tags @('Unit') {
129    $testDir = New-Item -ItemType Directory -Force
130    New-Item -Path $testDir -Name "file.txt" -ItemType File | Out-Null
131    
132    $result = Get-Item -Path "$testDir\file.txt"
133    
134    $result.Name | Should -Be "file.txt"
135    $result | Should -Exist
136    
137    Remove-Item $testDir -Recurse -Force
138}
139 
140# ✗ BAD: Returns Passed but doesn't validate functionality
141It "Get-Item returns files from directory" -Tags @('Unit') {
142    $result = Get-Item -Path somepath  # May not exist, may not actually test
143    $result | Should -Not -BeNullOrEmpty  # Too vague
144}
145 
146# ✗ BAD: Test marked Pending - validation is hidden
147It "Get-Item returns files from directory" -Tags @('Unit') {
148    Set-ItResult -Pending -Because "File system not working"
149    return
150    # No validation happens at all
151}
152```
153 
154---
155 
156## The Problem with Pending Tests
157 
158### Why Pending Tests Hide Problems
159 
160```powershell
161# BAD: Test marked Pending - looks like "working" status but validation is skipped
162It "Download help from web" {
163    Set-ItResult -Pending -Because "Web connectivity issues"
164    return
165    
166    # This code never runs:
167    Update-Help -Module PackageManagement -Force -ErrorAction Stop
168    Get-Help Get-Package | Should -Not -BeNullOrEmpty
169}
170```
171 
172**Result**:
173- ✗ Feature is broken (Update-Help fails)
174- ✓ Test shows "Pending" (looks acceptable)
175- ✗ Problem is hidden and never fixed
176 
177### The Right Approach
178 
179**Option A: Fix the root cause**
180```powershell
181It "Download help from web" {
182    # Use local assets that are guaranteed to work
183    Update-Help -Module PackageManagement -SourcePath ./assets -Force -ErrorAction Stop
184    
185    Get-Help Get-Package | Should -Not -BeNullOrEmpty
186}
187```
188 
189**Option B: Gracefully skip when unavailable**
190```powershell
191It "Download help from web" -Skip:$(-not $hasInternet) {
192    Update-Help -Module PackageManagement -Force -ErrorAction Stop
193    Get-Help Get-Package | Should -Not -BeNullOrEmpty
194}
195```
196 
197**Option C: Add retry logic for intermittent issues**
198```powershell
199It "Download help from web" {
200    $maxRetries = 3
201    $attempt = 0
202    
203    while ($attempt -lt $maxRetries) {
204        try {
205            Update-Help -Module PackageManagement -Force -ErrorAction Stop
206            break
207        }
208        catch {
209            $attempt++
210            if ($attempt -ge $maxRetries) { throw }
211            Start-Sleep -Seconds 2
212        }
213    }
214    
215    Get-Help Get-Package | Should -Not -BeNullOrEmpty
216}
217```
218 
219---
220 
221## Test Status Summary Table
222 
223| Status | Passed? | Validates? | Counts as "Working"? | Use When |
224|--------|---------|------------|----------------------|----------|
225| **Passed** | ✓ | ✓ | **YES** | Feature is working and test proves it |
226| **Failed** | ✗ | ✓ | NO | Feature is broken or test has wrong expectation |
227| **Error** | ✗ | ✗ | NO | Test infrastructure broken, can't validate |
228| **Pending** | - | ✗ | **NO** ⚠️ | Temporary - test should eventually pass |
229| **Skipped** | - | ✗ | NO | Test not applicable to this environment |
230| **Ignored** | - | ✗ | NO | Test permanently disabled |
231 
232---
233 
234## Recommended Patterns
235 
236### Pattern 1: Resilient Test with Fallback
237```powershell
238It "Feature works with web or local source" {
239    $useLocal = $false
240    
241    try {
242        Update-Help -Module Package -Force -ErrorAction Stop
243    }
244    catch {
245        $useLocal = $true
246        Update-Help -Module Package -SourcePath ./assets -Force -ErrorAction Stop
247    }
248    
249    # Validate functionality regardless of source
250    Get-Help Get-Package | Should -Not -BeNullOrEmpty
251}
252```
253 
254### Pattern 2: Conditional Skip with Clear Reason
255```powershell
256Describe "Update-Help from Web" -Skip $(-not (Test-InternetConnectivity)) {
257    It "Downloads help successfully" {
258        Update-Help -Module PackageManagement -Force -ErrorAction Stop
259        Get-Help Get-Package | Should -Not -BeNullOrEmpty
260    }
261}
262```
263 
264### Pattern 3: Separate Suites by Dependency
265```powershell
266Describe "Help Content Tests - Web" {
267    # Tests that require internet - can be skipped if unavailable
268    It "Downloads from web" { ... }
269}
270 
271Describe "Help Content Tests - Local" {
272    # Tests with local assets - should always pass
273    It "Loads from local assets" {
274        Update-Help -Module Package -SourcePath ./assets -Force
275        Get-Help Get-Package | Should -Not -BeNullOrEmpty
276    }
277}
278```
279 
280---
281 
282## Checklist: Is Your Test "Working"?
283 
284- [ ] Test status is **Passed** (not Pending, not Skipped, not Failed)
285- [ ] Test actually **executes** the feature being tested
286- [ ] Test has **specific assertions** (not just `Should -Not -BeNullOrEmpty`)
287- [ ] Test includes **cleanup** (removes temp files, restores state)
288- [ ] Test can run **multiple times** without side effects
289- [ ] Test failure **indicates a real problem** (not flaky assertions)
290- [ ] Test success **proves the feature works** (not just "didn't crash")
291 
292If any of these is false, your test may be passing but not "working" properly.
293 
294---
295 
296## See Also
297 
298- [Pester Documentation](https://pester.dev/)
299- [Set-ItResult Documentation](https://pester.dev/docs/commands/Set-ItResult)
300