Cursor rule

1---
2description:
3globs:
4alwaysApply: true
5---
6# Testing Guide for Script Kit App
7 
8## Avoid Running the App
9 
10Never, ever run the app. Ask me to start it, then you can check the logs.
11 
12## Test Configuration
13 
14### Performance Optimizations
15- Use `bail: 1` for fast failure (stop on first test failure)
16- Enable parallel execution with `singleThread: false` and `isolate: true` for speed
17- Set reasonable thread limits: `maxThreads: 4`, `minThreads: 2`
18- Use `vitest run` instead of `vitest` for single test runs
19 
20### Environment Setup
21- **Main process tests**: Use `node` environment
22- **Renderer tests**: Use `jsdom` environment
23- Configure separate vitest workspace configs for different environments
24 
25### Test Script Configuration
26```json
27{
28  "scripts": {
29    "test": "vitest run"  // Single run, not watch mode
30  }
31}
32```
33 
34## Test Execution Strategy
35 
36### Concurrent vs Sequential Testing
37Choose the right execution strategy based on test complexity:
38 
39#### ✅ Parallel Tests (Use `describe.concurrent()`)
40- Simple file operations (create, read, delete)
41- Basic validation tests
42- Mock-heavy tests with no external dependencies
43- Tests that don't require complex timing
44 
45#### ✅ Sequential Tests (Use regular `describe()`)
46- File system operations that require exclusive access
47- Directory renames and complex file operations
48- Tests with intricate timing dependencies
49- Resource-intensive operations
50 
51### Load-Resilient Test Design
52Tests must work under both isolated and concurrent execution:
53 
54```typescript
55// ❌ Fragile - only works in isolation
56const events = await collectEvents(500, async () => {
57  await writeFile(path, content);
58});
59 
60// ✅ Robust - works under system load
61const events = await collectEventsIsolated(1500, async (events, dirs) => {
62  await writeFile(path, content);
63  // Wait for file system under load
64  await new Promise(resolve => setTimeout(resolve, 300));
65});
66```
67 
68### Test Environment Differences
69Be aware that `pnpm test` creates different conditions than individual file execution:
70 
71- **Individual files**: Lower resource contention, faster execution
72- **Full test suite**: Higher system load, requires longer timeouts
73- **Solution**: Design tests to be resilient to both conditions
74 
75## Mocking Strategies
76 
77### ✅ What Works
78 
79#### Inline Mocks in Test Files
80Prefer inline mocks over shared mock utilities to avoid circular dependencies:
81 
82```typescript
83vi.mock('electron', () => ({
84  app: {
85    getPath: vi.fn((name: string) => {
86      switch (name) {
87        case 'userData': return '/Users/test/Library/Application Support/ScriptKit';
88        default: return '/Users/test';
89      }
90    }),
91    // ... other app methods
92  },
93  powerMonitor: {
94    on: vi.fn(),
95    addListener: vi.fn(),  // Important: Include both on AND addListener
96    listeners: vi.fn(() => []),
97  }
98}));
99```
100 
101#### Essential Electron APIs to Mock
102- `app.getPath()` - Critical for path resolution
103- `powerMonitor.addListener()` - Not just `on()`, include both
104- `nativeTheme` - Often required by components
105- `BrowserWindow` - With full webContents mock
106- `crashReporter.start()` - Often called during initialization
107 
108#### Electron-log Mock Structure
109```typescript
110vi.mock('electron-log', () => ({
111  default: {
112    transports: {
113      file: { level: 'info' },
114      console: { level: false },
115      ipc: { level: false },  // Essential - prevents "Cannot set properties of undefined"
116    },
117    info: vi.fn(),
118    error: vi.fn(),
119    // ... other log methods
120  }
121}));
122```
123 
124#### Node.js Module Mocking
125For `node:os` and other Node modules, provide complete API surface:
126```typescript
127vi.mock('node:os', () => ({
128  default: {
129    homedir: vi.fn(() => '/Users/test'),
130    platform: vi.fn(() => 'darwin'),
131    // ... all other os methods
132    constants: {
133      signals: { /* full signal definitions */ }
134    }
135  }
136}));
137```
138 
139### ❌ What Doesn't Work
140 
141#### Shared Mock Utilities
142Avoid creating shared mock files that are imported across tests:
143```typescript
144// ❌ Don't do this - causes circular dependencies
145import { setupCommonMocks } from './src/test-utils/mocks';
146```
147 
148#### Incomplete Mock Objects
149Missing required properties cause runtime errors:
150```typescript
151// ❌ Incomplete - missing required properties
152const testScript = {
153  filePath: '/test/path/script.ts',
154  system: 'resume'
155  // Missing: command, id, name
156};
157```
158 
159#### Complex Debounce/Timing Tests
160Tests that rely on complex timing with fake timers and lodash debounce are fragile:
161```typescript
162// ❌ Fragile - timing-dependent
163vi.advanceTimersByTime(250);
164expect(debouncedFunction).toHaveBeenCalledTimes(2);
165```
166 
167## Test Object Requirements
168 
169### Script Objects
170When creating test script objects, include all required properties:
171```typescript
172const testScript = {
173  filePath: '/test/path/script.ts',
174  kenv: '',
175  system: 'resume' as const,
176  type: ProcessType.System,
177  command: 'node',      // Required
178  id: 'test-script',    // Required
179  name: 'test-script'   // Required for Choice interface
180};
181```
182 
183### System Event Strings
184Use proper system event syntax:
185```typescript
186system: 'suspend lock-screen' as const  // Multiple events
187system: 'resume' as const               // Single event
188```
189 
190## Mock State Management
191 
192### Between Tests
193```typescript
194beforeEach(() => {
195  vi.useFakeTimers();
196  // Don't clear mocks if they're used by debounced functions
197});
198 
199afterEach(() => {
200  vi.useRealTimers();
201  vi.clearAllMocks(); // Clear after tests complete
202});
203```
204 
205### PowerMonitor Event Handling
206For tests that need to simulate events:
207```typescript
208const mockElectronBase = vi.hoisted(() => {
209  const handlers = new Map();
210  return {
211    powerMonitor: {
212      addListener: vi.fn((event: string, handler: Function) => {
213        if (!handlers.has(event)) handlers.set(event, []);
214        handlers.get(event).push(handler);
215      }),
216      listeners: vi.fn((event: string) => handlers.get(event) || [])
217    }
218  };
219});
220```
221 
222## Integration vs Unit Tests
223 
224### Skip Integration Tests
225Integration tests that require external dependencies should be skipped in the main test suite:
226```typescript
227it.skip('should run external command', async () => {
228  // Test requires pnpm, specific file paths, etc.
229});
230```
231 
232### Focus on Unit Logic
233Test the core business logic rather than external integrations:
234- File watching logic (mocked file operations)
235- Event registration/deregistration
236- State management
237- Component rendering
238 
239## Performance Testing
240 
241### File System Tests
242Use temporary directories and proper cleanup:
243```typescript
244const testDir = vi.hoisted(() =>
245  import('tmp-promise').then(({ dir }) => dir({ unsafeCleanup: true }))
246);
247```
248 
249### Isolated Directory Pattern
250For file system tests that need complete isolation from each other:
251 
252```typescript
253/**
254 * Create isolated test directories for parallel-safe testing
255 */
256async function createIsolatedTestDirs(testName: string) {
257  const { dir } = await import('tmp-promise');
258  const tmpDir = await dir({
259    unsafeCleanup: true,
260    prefix: `test-${testName}-`,
261  });
262 
263  const isolatedDirs = {
264    root: tmpDir.path,
265    kit: path.join(tmpDir.path, '.kit'),
266    kenv: path.join(tmpDir.path, '.kenv'),
267    scripts: path.join(tmpDir.path, '.kenv', 'scripts'),
268    // ... other directories
269    cleanup: tmpDir.cleanup,
270  };
271 
272  // Create directory structure
273  await Promise.all([
274    ensureDir(isolatedDirs.kit),
275    ensureDir(isolatedDirs.kenv),
276    ensureDir(isolatedDirs.scripts),
277  ]);
278 
279  return isolatedDirs;
280}
281 
282/**
283 * Isolated test execution with environment variable override
284 */
285async function collectEventsIsolated(
286  duration: number,
287  action: (events: TestEvent[], dirs: any) => Promise<void>,
288  testName: string,
289): Promise<TestEvent[]> {
290  const isolatedDirs = await createIsolatedTestDirs(testName);
291 
292  // Override environment variables for this test
293  const originalKIT = process.env.KIT;
294  const originalKENV = process.env.KENV;
295  process.env.KIT = isolatedDirs.kit;
296  process.env.KENV = isolatedDirs.kenv;
297 
298  try {
299    // Execute test logic...
300    return events;
301  } finally {
302    // Restore environment variables
303    process.env.KIT = originalKIT;
304    process.env.KENV = originalKENV;
305    await isolatedDirs.cleanup();
306  }
307}
308```
309 
310### Test Isolation Strategy
311Choose isolation level based on test requirements:
312 
313- **Shared test directory**: Fast, but potential cross-test interference
314- **Isolated directories**: Slower setup, but complete isolation
315- **Environment variable override**: Essential for file system tests
316 
317### Timing Guidelines
318Adjust timeouts based on execution context:
319 
320```typescript
321// ❌ Fixed timing - breaks under load
322it('should detect file changes', async () => {
323  // Always fails when system is busy
324  const events = await collectEvents(500, ...);
325}, 3000);
326 
327// ✅ Load-aware timing
328it('should detect file changes', async () => {
329  const events = await collectEventsIsolated(
330    1500, // Longer collection time for concurrent environment
331    async (events, dirs) => {
332      await writeFile(filePath, content);
333      // Extra wait for file system under load
334      await new Promise(resolve => setTimeout(resolve, 300));
335    },
336    'test-name'
337  );
338}, 8000); // Longer overall timeout
339```
340 
341## Common Pitfalls
342 
3431. **Missing `addListener` in powerMonitor** - Electron uses both `on` and `addListener`
3442. **Incomplete electron-log transports** - Must include `ipc` transport
3453. **Type mismatches in test objects** - Include all required Script properties
3464. **Mock state bleeding between tests** - Timing of `vi.clearAllMocks()`
3475. **Integration test failures** - Skip tests requiring external setup
3486. **Resource contention in parallel tests** - Move complex operations to sequential execution
3497. **Fixed timing assumptions** - Tests fail under system load, use load-aware timeouts
3508. **Ignoring test environment differences** - `pnpm test` vs individual files require different strategies
3519. **Over-engineering solutions** - Sometimes moving tests is better than complex timing fixes
35210. **Insufficient isolation** - File system tests interfere without proper directory isolation
353 
354## Problem-Solving Strategy
355 
356When tests fail intermittently:
357 
3581. **First, try the obvious solution** - Move timing-sensitive tests to sequential execution
3592. **Increase timeouts progressively** - 500ms → 1500ms → 2000ms until stable
3603. **Add load-aware waits** - Extra delays between file operations under concurrent load
3614. **Use isolated directories** - Prevent cross-test contamination
3625. **Check execution context** - Individual vs full test suite may need different approaches
363 
364## Performance Optimization Results
365 
366### Actual Optimization Case Study (Chokidar Tests)
367- **Original**: 30.9 seconds, multiple failures
368- **After timing optimization**: 17.0 seconds, 100% passing
369- **After strategic test placement**: 18.1 seconds, 100% reliable
370- **Total improvement**: 42% faster + 100% reliability
371 
372### Key Strategies That Worked
3731. **Strategic test categorization** - Parallel vs sequential based on complexity
3742. **Load-resilient timing** - Timeouts that work under system load
3753. **Isolated directory pattern** - Complete test isolation
3764. **Pragmatic problem solving** - Move problematic tests rather than fix timing
377 
378## Success Metrics
379 
380A well-optimized test suite should achieve:
381- ✅ 75+ tests passing (expanded coverage)
382- ✅ ~21 second execution time (full suite)
383- ✅ 100% reliability under both individual and concurrent execution
384- ✅ Clean exit code (0) in all environments
385- ✅ Strategic test placement (parallel vs sequential)
386- ✅ Load-resilient timing patterns
387- ✅ Comprehensive file system coverage with proper isolation
388 
389Focus on **reliability over speed** - better to have slightly slower tests that always pass than fast tests that fail intermittently.
390 
391## Benchmark Testing and Naming Conventions
392 
393### File Naming Standards
394 
395#### ✅ `.bench.ts` - Pure Performance Benchmarks
396- Industry standard for dedicated benchmark files
397- Uses `bench()` functions for performance measurement
398- Separate from regular test suite
399- Run with: `pnpm vitest bench file.bench.ts --run`
400- Requires vitest config to include `bench` in file patterns
401 
402#### ✅ `.test.ts` - Performance Tests with Assertions
403- Performance validation using `it()` + `expect()`
404- Part of regular test suite with performance assertions
405- Includes detailed console output + validation
406- Run with: `pnpm vitest run file.test.ts`
407 
408### Vitest Configuration for Benchmarks
409Update [vitest.config.ts](mdc:vitest.config.ts) to include benchmark files:
410```typescript
411test: {
412  include: ['**/*.{test,spec,bench}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
413  benchmark: {
414    // Add tinybench options here if needed
415  }
416}
417```
418 
419### Benchmark Execution Commands
420```bash
421# Pure benchmarks (single-run, not watch mode)
422pnpm vitest bench src/main/file.bench.ts --run
423 
424# Performance tests with assertions
425pnpm vitest run src/main/file.test.ts
426 
427# Both together via script
428node scripts/bench-search.js
429```
430 
431## Search Performance Testing
432 
433### Large Dataset Testing (10,000+ Choices)
434For testing search performance with realistic datasets:
435 
436```typescript
437// Generate realistic mock choices
438function generateMockChoices(count: number): Choice[] {
439  const categories = ['File Operations', 'Git Tools', 'Development'];
440  const prefixes = ['Quick', 'Advanced', 'Simple', 'Super'];
441  const actions = ['Manager', 'Tool', 'Helper', 'Runner'];
442 
443  return Array.from({ length: count }, (_, i) => ({
444    id: `choice-${i}`,
445    name: `${prefixes[i % prefixes.length]} ${actions[i % actions.length]} ${i}`,
446    keyword: `keyword${i}`,
447    group: categories[i % categories.length],
448    // Add variety: 10% shortcodes, 1% info choices, 2% hidden
449    info: i % 100 === 0,
450    hideWithoutInput: i % 50 === 0,
451  }));
452}
453 
454// Performance measurement utility
455class PerformanceTracker {
456  private measurements: Map<string, number[]> = new Map();
457 
458  startTimer(operation: string): () => number {
459    const start = performance.now();
460    return () => {
461      const duration = performance.now() - start;
462      if (!this.measurements.has(operation)) {
463        this.measurements.set(operation, []);
464      }
465      this.measurements.get(operation)!.push(duration);
466      return duration;
467    };
468  }
469 
470  getStats(operation: string) {
471    const times = this.measurements.get(operation) || [];
472    if (times.length === 0) return null;
473 
474    const sorted = times.slice().sort((a, b) => a - b);
475    return {
476      avg: times.reduce((a, b) => a + b, 0) / times.length,
477      min: sorted[0],
478      max: sorted[sorted.length - 1],
479      p95: sorted[Math.floor(sorted.length * 0.95)],
480    };
481  }
482}
483```
484 
485### Search Performance Benchmarks
486Essential benchmarks to include:
487 
488```typescript
489// Progressive typing simulation
490bench('Progressive typing simulation', () => {
491  const progressiveTerms = ['f', 'fi', 'fil', 'file', 'file m'];
492  progressiveTerms.forEach(term => measureSearch(term));
493});
494 
495// Real-world usage patterns
496bench('Real-world usage pattern', () => {
497  const searchTerms = ['quick', 'file', 'git', '', ' ', 'nonexistent'];
498  searchTerms.forEach(term => measureSearch(term));
499});
500 
501// Scaling tests
502it('should benchmark different choice set sizes', () => {
503  const sizes = [1000, 2500, 5000, 7500, 10000];
504  sizes.forEach(size => {
505    const subset = choices.slice(0, size);
506    setChoices(mockPrompt, subset, { preload: false });
507    const result = measureSearch('file manager');
508    expect(result.duration).toBeLessThan(size * 0.02); // Max 0.02ms per choice
509  });
510});
511```
512 
513### Performance Assertions and Targets
514Set realistic performance targets:
515 
516```typescript
517// Individual search performance
518expect(result.duration).toBeLessThan(100); // Should be under 100ms
519 
520// Average performance across many searches
521expect(avgDuration).toBeLessThan(50); // Average under 50ms
522 
523// Memory usage validation
524expect(memoryDiff.heapUsed).toBeLessThan(200 * 1024 * 1024); // Under 200MB growth
525 
526// Scaling requirements
527expect(slowSearches.length).toBeLessThan(results.length * 0.1); // <10% slow searches
528```
529 
530## Integration Testing Patterns
531 
532### UI Input → IPC → Search Flow Testing
533Test the complete user interaction flow:
534 
535```typescript
536describe('UI Input Integration', () => {
537  // Test Channel.INPUT message handling
538  it('should handle user typing via Channel.INPUT', () => {
539    const inputMessage: InputMessage = {
540      input: 'file manager',
541      from: 'user-typing'
542    };
543 
544    // Simulate IPC message
545    handleInputMessage(mockPrompt, inputMessage);
546 
547    // Verify search was invoked
548    expect(mockInvokeSearch).toHaveBeenCalledWith(
549      mockPrompt,
550      'file manager',
551      'user-typing'
552    );
553 
554    // Verify results sent back
555    expect(mockSendToPrompt).toHaveBeenCalledWith(
556      Channel.SET_SCORED_CHOICES,
557      expect.any(Array)
558    );
559  });
560 
561  // Test search state management
562  it('should update search state correctly', () => {
563    const choices = generateMockChoices(1000);
564    setChoices(mockPrompt, choices, { preload: false });
565 
566    // Test hasGroup setting for info choices
567    mockPrompt.kitSearch.hasGroup = true;
568 
569    invokeSearch(mockPrompt, 'test', 'integration');
570 
571    expect(mockPrompt.kitSearch.input).toBe('test');
572    expect(mockPrompt.kitSearch.choices).toHaveLength(1000);
573  });
574});
575```
576 
577### End-to-End Search Integration
578Simulate complete user workflows:
579 
580```typescript
581// Helper to simulate complete UI → IPC → Search flow
582const simulateUserTyping = (
583  input: string,
584  choices: Choice[],
585  options: { mode?: Mode; ui?: UI; expectSearch?: boolean } = {}
586) => {
587  // Setup choices
588  mockPrompt.kitSearch.choices = choices;
589  mockPrompt.kitSearch.hasGroup = choices.some(c => !!c.group);
590 
591  // Mock QuickScore for realistic search
592  const mockQs = {
593    search: vi.fn((searchInput: string) => {
594      return choices
595        .filter(choice => {
596          if (choice.hideWithoutInput && (!searchInput || searchInput.trim() === '')) {
597            return false;
598          }
599          return choice.name?.toLowerCase().includes(searchInput.toLowerCase()) ||
600                 choice.keyword?.toLowerCase().includes(searchInput.toLowerCase()) ||
601                 choice.info === true;
602        })
603        .map(choice => ({
604          item: choice,
605          score: 0.8,
606          matches: { name: [[0, searchInput.length]] },
607          _: ''
608        }));
609    })
610  };
611  mockPrompt.kitSearch.qs = mockQs as any;
612 
613  // Execute search
614  invokeSearch(mockPrompt, input, 'test');
615 
616  // Return results for analysis
617  const scoredChoicesMessage = sentMessages.find(m => m.channel === Channel.SET_SCORED_CHOICES);
618  return (scoredChoicesMessage?.data as ScoredChoice[]) || [];
619};
620```
621 
622### Search-Specific Mock Requirements
623Essential mocks for search testing:
624 
625```typescript
626// Required search mocks
627vi.mock('./search', () => ({
628  invokeSearch: vi.fn(),
629  setChoices: vi.fn(),
630  setShortcodes: vi.fn(),
631}));
632 
633vi.mock('./messages', () => ({
634  cacheChoices: vi.fn()
635}));
636 
637vi.mock('./state', () => ({
638  kitCache: {
639    choices: [],
640    scripts: [],
641    triggers: new Map(),
642    keywords: new Map(),
643    shortcodes: new Map(),
644  },
645  kitState: {
646    kenvEnv: {
647      KIT_SEARCH_MAX_ITERATIONS: '3',
648      KIT_SEARCH_MIN_SCORE: '0.6',
649    },
650  },
651}));
652 
653// Fix lodash debounce mock for search
654vi.mock('lodash-es', () => ({
655  debounce: vi.fn((fn) => {
656    const mockDebounced = vi.fn(fn) as any;
657    mockDebounced.cancel = vi.fn();
658    return mockDebounced;
659  }),
660}));
661```
662 
663### Search Test Object Requirements
664Ensure complete test objects for search functionality:
665 
666```typescript
667// Complete KitPrompt mock for search testing
668const mockPrompt: KitPrompt = {
669  ui: UI.arg,
670  pid: 12345,
671  scriptPath: '/test/script.ts',
672  sendToPrompt: mockSendToPrompt,
673  kitSearch: {
674    input: '',
675    inputRegex: undefined,
676    keyword: '',
677    keywordCleared: false,
678    generated: false,
679    flaggedValue: '',
680    choices: [],
681    scripts: [],
682    qs: null,
683    hasGroup: false, // Critical for info choice handling
684    keys: ['name', 'keyword', 'tag'],
685    keywords: new Map(),
686    triggers: new Map(),
687    postfixes: new Map(),
688    shortcodes: new Map(),
689  },
690  flagSearch: {
691    input: '',
692    choices: [],
693    hasGroup: false,
694    qs: null,
695  },
696  updateShortcodes: vi.fn(),
697} as unknown as KitPrompt;
698```
699 
700## Performance Testing Success Metrics
701 
702### Search Performance Targets
703- ✅ Average search time < 50ms (10,000 choices)
704- ✅ Individual searches < 100ms
705- ✅ Progressive typing responsive (< 75ms average)
706- ✅ Memory growth < 200MB during intensive testing
707- ✅ <10% of searches should be considered "slow" (>100ms)
708- ✅ Scaling: Max 0.02ms per choice
709 
710### Integration Testing Coverage
711- ✅ UI input → IPC → Search → Results flow
712- ✅ Search state management (hasGroup, choices, etc.)
713- ✅ Info choice prioritization
714- ✅ Hide-without-input behavior
715- ✅ Empty/whitespace input handling
716- ✅ Progressive typing simulation
717- ✅ Memory leak prevention
718 
719### Example Performance Test Files
720- [search-performance.bench.ts](mdc:src/main/search-performance.bench.ts) - Pure benchmarks
721- [search-performance.test.ts](mdc:src/main/search-performance.test.ts) - Performance tests with assertions
722- [search-integration.test.ts](mdc:src/main/search-integration.test.ts) - End-to-end integration
723- [ipc-input.test.ts](mdc:src/main/ipc-input.test.ts) - IPC message handling
724