Two files, one repository
mobile-dev-inc/Maestro ships 2 formats across 2 indexed files. The question worth asking is whether the second one says anything the first does not.
CompareAGENTS.md ↔ CLAUDE.md
| Dimension | Shared | Only in A | Only in B | Overlap |
|---|---|---|---|---|
| Sections | 0 | 13 | 17 | 0% |
| Commands | 0 | 6 | 5 | 0% |
| Section tags | 3 | 2 | 2 | 43% |
What each file covers
Sections
0 shared · 13 only in A · 17 only in B- − AGENTS.md — Maestro
- − Module map
- − E2E test fixtures (`e2e/`)
- − `passing/` vs `failing/` suites
- − `test-e2e.yaml` workflow contract
- − Testing
- − Unit tests (per module, `src/test/kotlin/`)
- − Integration tests (`maestro-test/`)
- − E2E tests (`e2e/`)
- − MCP server evals (`maestro-cli/src/test/mcp/`)
- − Conventions
- − Where Claude Code resources live
- − What NOT to do
- + CLAUDE.md
- + What This Is
- + Build Commands
- + Run the app (requires a connected device or emulator)
- + Build Android APK
- + Build iOS simulator app
- + Analyze code
- + Maestro Flow Commands
- + Run all flows
- + Run a single flow
- + Run flows with a specific tag
- + Architecture
- + Flutter App (`lib/`)
- + Maestro Flows (`.maestro/`)
- + App ID
- + Testing Permissions
- + Adding New Test Screens
Commands
0 shared · 6 only in A · 5 only in B- − ./gradlew :maestro-orchestra:test
- − ./gradlew test
- − ./gradlew :maestro-test:test
- − gh workflow run test-e2e.yaml --ref <branch> -f android_version=android-<N>
- − gh workflow run test-e2e.yaml --ref <branch> -f android_version=<...>
- − npx
- + flutter run
- + flutter build apk
- + flutter build ios --simulator
- + flutter analyze
- + flutter build ios
Section tags
3 shared · 2 only in A · 2 only in B- − testing-strategy
- − api
- + build
- + security
- test
- code-style
- agent-behaviour
Line diff
mobile-dev-inc/Maestro · AGENTS.md
@@ −1 @@
1# AGENTS.md — Maestro
2
3Shared context for any Claude Code skill or subagent operating in this repo. Skills (`.claude/skills/*`) reference this file rather than restating module roles; if a description here drifts from reality, fix it here once and every skill follows.
4
5## Module map
6
7Top-level Gradle modules. Code lives under each module's `src/main/`.
8
9| Module | Role |
10|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
11| `maestro-android/` | On-device Android driver. Kotlin sources compile to two checked-in APKs (`maestro-app.apk`, `maestro-server.apk`) consumed by `maestro-client/`. The build's `copyMaestroAndroid` / `copyMaestroServer` finalizers update those APKs plus a `maestro-android-source.sha256` checksum. |
12| `maestro-ios-driver/` | Host side of iOS driver wrapper (Kotlin). The actual XCTest runner lives in `maestro-ios-xctest-runner/`. |
13| `maestro-ios-xctest-runner/` | Swift XCTest runner that runs on the iOS device/simulator. The compiled artifacts (`maestro-driver-ios*.zip`) are checked in under `maestro-ios-driver/src/main/resources/driver-iPhoneSimulator/Debug-iphonesimulator/`. |
14| `maestro-ios/` | iOS host-side glue (small — most iOS host code lives in `maestro-client/`). |
15| `maestro-client/` | Host-side Kotlin SDK that drives devices. Platform drivers live in `src/main/java/maestro/drivers/`: `AndroidDriver.kt`, `IOSDriver.kt`, `WebDriver.kt`, `CdpWebDriver.kt`. This is where most "auto-grant", "auto-dismiss", system-dialog handling and platform-specific quirks belong. |
16| `maestro-orchestra/` | Command execution layer. `Orchestra.kt` interprets each Maestro command, applies retries, manages the command lifecycle. Sub-packages: `error/`, `filter/`, `workspace/`, `yaml/`. |
17| `maestro-orchestra-models/` | Shared command/data models (used by `maestro-orchestra/` and consumers). |
18| `maestro-cli/` | CLI entry point + MCP server. Mixed Kotlin (~100 files) + Swift (~56 files for iOS-related CLI bits). |
19| `maestro-utils/` | Shared utilities. |
20| `maestro-web/` | Web (browser) driver pieces. |
21| `maestro-proto/` | Protobuf definitions shared across modules. |
22| `maestro-test/` | Cross-module tests that doesn't require devices. |
23
24## E2E test fixtures (`e2e/`)
25
26Shipped fixtures used by `.github/workflows/test-e2e.yaml`. Run via `e2e/run_tests <android|ios|web>` (see `e2e/run_tests` for env-var inputs `MAESTRO_APP`, `MAESTRO_FLOW_PATH`).
27
28| Path | Role |
29|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
30| `e2e/demo_app/` | Flutter demo app whose only purpose is to exercise Maestro features. Contains its own `CLAUDE.md`. Built binaries are uploaded to a GCS bucket and re-downloaded by CI. |
31| `e2e/demo_app/.maestro/` | Maestro flow YAMLs that drive the demo app. |
32| `e2e/workspaces/` | Additional app workspaces (e.g. `simple_web_view`, `wikipedia`). |
33| `e2e/run_tests` | Test driver invoked by the workflow. |
34
35### `passing/` vs `failing/` suites
36
37Tag-based filters inside the YAML flows split test runs into two suites at execution time:
38
39- `passing/` — flows tagged `passing`. **Expected to pass.** Any failure here is a real regression. This is the only suite the diagnose agent reads.
40- `failing/` — flows tagged `failing`. **Expected to fail** (negative-path coverage: assertions that should not match, commands that should error). The workflow inverts the success check on this suite. Do not treat `failing/` artifacts as regressions.
41
42Artifacts land at `<artifact_root>/tests/<app>/<suite>/`:
43
44## `test-e2e.yaml` workflow contract
45
46`.github/workflows/test-e2e.yaml` is the validation harness for both PR triggers and manual `workflow_dispatch` (e.g. validating a new Android API level or iOS version). Contract:
47
48- **`workflow_dispatch` inputs** — `android_version` (choice enum), `app` (string, default `demo_app`), `flow` (string, optional single-flow). The `validate-inputs` job rejects `android_version <= android-29`, missing `app` workspace, or ambiguous `flow`. (See PR #3226.)
49- **`pull_request` triggers** are byte-identical to the prior behaviour; manual dispatches use the new narrowing knobs.
50- **`test-android` job** boots an emulator on `system-images;${android_version};google_apis;x86_64` and runs `e2e/run_tests android`.
51
52Skills that bump platform versions (Android API levels, iOS versions) drive this workflow via `gh workflow run test-e2e.yaml --ref <branch> -f android_version=<...>`.
53
54## Testing
55
56Three layers — unit, integration, E2E — plus MCP-specific evals. Each layer has a different cost/coverage trade-off; default to the lowest layer that can express the test.
57
58### Unit tests (per module, `src/test/kotlin/`)
59
60Standard per-class tests. Stack: **JUnit 5** (`junit-jupiter-api` + `-params` + `-engine`), **Google Truth** for assertions, **MockK** for mocks. Each module's `build.gradle.kts` enables the platform via `tasks.named<Test>("test") { useJUnitPlatform() }`.
61
62```bash
63./gradlew :maestro-orchestra:test # one module
64./gradlew test # all modules
65```
66
67### Integration tests (`maestro-test/`)
68
69Cross-module tests for behaviour that does **not** require a device or simulator — JS engine integration points, command orchestration end-to-end, cancellation / coroutine semantics. Notable suites:
70
71- `IntegrationTest.kt` — full `Maestro` orchestration against an in-process `FakeDriver` (defined in `maestro-test/src/main/kotlin/maestro/test/drivers/`: `FakeDriver`, `FakeLayoutElement`, `FakeTimer`). Covers test-run cancellation (`CancellationException`, `withTimeout`, supervisor scopes) and the full command lifecycle without a real device.
72- `GraalJsEngineTest.kt` / shared `JsEngineTest.kt` — Maestro's JS extension points (`evalScript`, JS-evaluated assertions/conditions). Exercises `org.graalvm.polyglot` directly.
73- `FlowControllerTest.kt`, `DeepestMatchingElementTest.kt` — orchestration and view-hierarchy logic.
74
75Stack: **JUnit 5**, **Google Truth**, **WireMock JRE8** (HTTP fakes), plus the in-house `FakeDriver` fixtures listed above. No mocks of Maestro's own classes — tests run real `Maestro` against the fakes.
76
77```bash
78./gradlew :maestro-test:test
79```
80
81### E2E tests (`e2e/`)
82
83Smoke-test every Maestro command across Android, iOS, and Web on real fixture apps. Maestro is its own dogfood harness: the CLI executes Maestro flow YAMLs against the fixtures, asserting both the framework's commands and the platform drivers behave correctly.
84
85Stack: **Maestro CLI itself** (dogfood) + `e2e/run_tests` shell driver + GHA workflow (`.github/workflows/test-e2e.yaml`). Fixture and suite layout is in "E2E test fixtures (`e2e/`)" above.
86
87```bash
88cd e2e && ./run_tests <android|ios|web> # local
89gh workflow run test-e2e.yaml --ref <branch> -f android_version=android-<N> # CI
90```
91
92**Two roles for the same E2E setup.** The same suite serves both purposes — treat them identically:
93
941. **Regression smoke** — every PR that touches Maestro source runs the suite on the current platform versions, catching behaviour breakage on existing platforms.
952. **New-OS validation** — when launching a new Android API level or iOS version, the same flows are dispatched against the new system image to confirm Maestro still works. This is what `bump-android-version` (and the planned `bump-ios-version`) drives.
96
97A flow breaking for either reason is a real regression — fix in `maestro-android/`, `maestro-client/`, or `e2e/demo_app/`, not in `test-e2e.yaml` (see "What NOT to do").
98
99**Multiple apps for framework-specific coverage.** `demo_app/` (Flutter) is the default fixture and exercises every Maestro command. When a target is **framework-specific** (SwiftUI, React Native, Jetpack Compose specifics, WebView quirks, etc.), add a separate workspace under `e2e/workspaces/<app>/` with its own `.maestro/` flow YAMLs and a binary under `e2e/apps/`. Existing examples: `simple_web_view` (WebView coverage), `wikipedia` (real-world third-party app). The workflow's `app` input narrows a manual dispatch to one workspace: `... -f app=simple_web_view`.
100
101### MCP server evals (`maestro-cli/src/test/mcp/`)
102
103LLM-behaviour evaluations and tool-functionality tests for the MCP server inside `maestro-cli`. Stack: **`mcp-server-tester`** (npm package, run via `npx`) consuming YAML definitions (`full-evals.yaml`, `inspect-screen-evals.yaml`, `tool-tests-{with,without}-device.yaml`) plus per-platform setup scripts under `setup/`. See `maestro-cli/src/test/mcp/README.md` for the model list, scorers, and how to run.
104
105```bash
106./run_mcp_tool_tests.sh ios # tool-functionality (fast)
107./run_mcp_evals.sh ios # LLM behaviour (slower)
108```
109
110## Conventions
111
112- Kotlin 1.9 / JVM 17. Gradle. No DI framework — services are constructed manually.
113- Protobuf for the on-device wire format (`maestro-proto/`).
114- Coroutines with explicit dispatchers; `runBlocking` only at entry points.
115- Exposed exceptions classify failures (retryable vs terminal) — see `maestro-orchestra/src/main/java/maestro/orchestra/error/`.
116- **Temp files and directories go through `maestro.utils.TempFileHandler`**, not `java.nio.file.Files.createTempFile/createTempDirectory` directly. `TempFileHandler` is a `Closeable` that recursively cleans up everything it allocated on `close()`. Direct `Files.createTempFile(...)` skips that lifecycle and leaks `/tmp` content (especially painful on long-lived JVMs like the cloud worker). Construct a `TempFileHandler` near the lifecycle owner, call its `createTempFile` / `createTempDirectory`, and `close()` it in a `finally`.
117
118## Where Claude Code resources live
119
120- `.claude/skills/*` — skills (workflows). Each skill's `SKILL.md` references this file for module roles.
121- `.claude/agents/*.md` — subagents (e.g. `diagnose-maestro-failure.md`). Their input/output contracts are documented in each file.
122
123## What NOT to do
124
125- Don't fix driver-behaviour gaps by patching `.github/workflows/test-e2e.yaml` (e.g. extra `adb shell settings put …`, command-line tweaks, AVD pre-config). Workflow band-aids hide the regression from users running Maestro outside our CI. Fix `maestro-android/`, `maestro-client/`, or `e2e/demo_app/` instead so the fix ships with the driver APKs. Workflow edits are valid for shape-changes (matrix, retention, dispatch inputs) and the narrow third-party-FRE exception documented in skill files.
126- Don't edit checked-in driver artifacts (`maestro-app.apk`, `maestro-server.apk`, `maestro-android-source.sha256`, `maestro-driver-ios*.zip`) by hand — they are gradle finalizer outputs.
127- Don't commit local changes to the iOS driver zips. A local build regenerates two zips — `maestro-driver-ios.zip` and `maestro-driver-iosUITests-Runner.zip` — under `maestro-ios-driver/src/main/resources/driver-iPhoneSimulator/Debug-iphonesimulator/`. This is normal: they're used by local builds. But although they're checked into the repo, they're managed exclusively by CI, so leave any local modifications to them out of your commits.
128- Don't modify existing flows in `failing/` to make them pass — that's the negative-path suite by design.
129
mobile-dev-inc/Maestro · e2e/demo_app/CLAUDE.md
@@ +1 @@
1# CLAUDE.md
2
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
5## What This Is
6
7A Flutter demo app used as the target app for end-to-end testing of the [Maestro](https://github.com/mobile-dev-inc/maestro) mobile UI testing framework. It is not a production app — its screens exist to exercise specific Maestro features and reproduce specific bugs.
8
9When new screens or behaviors are needed to test a Maestro feature, they are added here.
10
11## Build Commands
12
13```sh
14# Run the app (requires a connected device or emulator)
15flutter run
16
17# Build Android APK
18flutter build apk
19
20# Build iOS simulator app
21flutter build ios --simulator
22
23# Analyze code
24flutter analyze
25```
26
27## Maestro Flow Commands
28
29Flows live in `.maestro/`. **Prefer the Maestro MCP** for authoring, running, and debugging flows interactively (`list_devices` → `inspect_screen` / `take_screenshot` → `run`): it returns the view hierarchy and screenshots inline, which is far more effective for iterating than parsing CLI output. Use the CLI below for scripted or CI-style runs, or when no MCP is available.
30
31```sh
32# Run all flows
33maestro test .maestro/
34
35# Run a single flow
36maestro test .maestro/fill_form.yaml
37
38# Run flows with a specific tag
39maestro test --include-tags passing .maestro/
40```
41
42**The MCP runs a *built* Maestro, not your working tree.** If you change Maestro framework code (anything outside `e2e/`) and want to validate it through the MCP against this app, the MCP will keep using the old build until Maestro is rebuilt **and the MCP is reconnected** (e.g. `/mcp reconnect maestro`). Rebuild, reconnect, then re-run. This is separate from rebuilding the demo app itself (see Build Commands) — a Dart/iOS/Android change to this app needs the app rebuilt and reinstalled on the device before the MCP will see it.
43
44## Architecture
45
46### Flutter App (`lib/`)
47
48`main.dart` is the home screen with buttons navigating to each test screen. Each screen is a standalone Dart file targeting a specific testing scenario:
49
50| File | Purpose |
51|---|---|
52| `form_screen.dart` | Login form with email/password validation |
53| `input_screen.dart` | Keyboard and text input behaviors |
54| `swiping_screen.dart` | Swipe gesture testing |
55| `nesting_screen.dart` | Deeply nested widget hierarchies |
56| `location_screen.dart` | GPS location via `geolocator`, streams position updates |
57| `sensors_screen.dart` | Device sensors (Android only) |
58| `webview.dart` | Embedded WebView via `webview_flutter` |
59| `defects_screen.dart` | Intentional UI quirks for defect regression |
60| `cropped_screenshot_screen.dart` | Screenshot cropping edge cases |
61| `notifications_permission_screen.dart` | Permission request flows |
62| `permission_check_screen.dart` | Passively displays permission status (location, all-files) via `permission_handler` — never calls `requestPermission()`, so it reflects a pre-granted state deterministically |
63| `issue_1619_repro.dart`, `issue_1677_repro.dart` | Bug reproductions |
64
65The app reads launch arguments via `flutter_launch_arguments` (e.g., `initialCounter`, `delay`) so Maestro flows can configure app state at launch.
66
67### Maestro Flows (`.maestro/`)
68
69- **Root flows** (`*.yaml`): Main passing/failing test cases, tagged `passing` or used to assert expected failures.
70- **`commands/`**: Reusable Maestro command definitions (e.g., `assertVisible.yaml`, `inputText.yaml`).
71- **`android_device_configuration/`** and **`ios_device_configuration/`**: Device setup flows run before tests (disable autocorrect, set timezone, enable sensors, etc.).
72- **`web_flows/`**: Flows targeting web/WebView scenarios.
73- **`issues/`**: Flows specifically reproducing reported Maestro bugs.
74- **`experimental/`**: Unstable/in-progress flows not included in CI.
75- **`scripts/`**: JavaScript helpers used by `evalScript` commands.
76
77`config.yaml` configures which flow directories Maestro includes when running `maestro test .maestro/`.
78
79**Platform targeting.** Write flows to run on both Android and iOS by default. Add an `android` or `ios` tag only when the behaviour is genuinely platform-specific — a flow with no platform tag runs on every platform. Prefer keeping a single cross-platform flow over splitting into per-platform files: use `${maestro.platform == "android" ? ... : ...}` for platform-specific values, and `runFlow` with `when: platform:` to guard platform-specific steps.
80
81### App ID
82
83All flows target `appId: com.example.example`.
84
85## Testing Permissions
86
87Non-obvious gotchas when writing permission flows against this app:
88
89- **iOS: each `permission_handler` permission must be enabled in `ios/Podfile`.** A permission's handler is compiled in only when its macro is set in `GCC_PREPROCESSOR_DEFINITIONS` (e.g. `PERMISSION_LOCATION=1`). Without it, that permission's `.status` **silently returns denied on iOS** regardless of the real authorization. Enabled today: notifications, location. After editing the Podfile, run `pod install` in `ios/` before `flutter build ios` — a Podfile edit alone won't trigger it.
90- **Android: runtime permissions must be declared in `AndroidManifest.xml`** or they can't be granted (`pm grant` throws). Declared today: INTERNET, ACCESS_FINE/COARSE_LOCATION, MANAGE_EXTERNAL_STORAGE. (So e.g. POST_NOTIFICATIONS can't be granted here.)
91- **Permission values are platform-specific.** Android uses `allow`/`deny`/`unset`; iOS `location` uses `always`/`inuse`/`never`/`unset`. iOS *validates* location values and throws on anything else; Android silently falls back to revoke for unknown/empty values. For cross-platform flows, pick per platform: `location: ${maestro.platform == "android" ? "allow" : "always"}`.
92- **`launchApp` with no `permissions:` block defaults to `all: allow`.**
93- **Observe passively.** `permission_check_screen.dart` reads status without requesting — use it to assert a pre-granted state on both platforms. Avoid `location_screen.dart` (geolocator) for that: it actively calls `requestPermission()`, popping a dialog (Android does not auto-dismiss it) and resolving position asynchronously.
94- **`MANAGE_EXTERNAL_STORAGE` is an appOps permission** on Android (special path), not a standard `pm grant` runtime permission.
95
96## Adding New Test Screens
97
981. Create a new `lib/<feature>_screen.dart` with a `StatefulWidget`.
992. Add a navigation button in `lib/main.dart`.
1003. Write the flow that exercises the **Maestro feature** you are testing, tagged `[passing]`. The flow is the point — the screen exists only to make that Maestro behaviour observable, not to be tested itself. Add a platform tag only if the behaviour is platform-specific (see **Platform targeting** above).
1014. For location or sensor tests, ensure relevant device configuration flows exist in the platform-specific subdirectories.
102
@@ −1 +1 @@
1−# AGENTS.md — Maestro
1+# CLAUDE.md
22
3−Shared context for any Claude Code skill or subagent operating in this repo. Skills (`.claude/skills/*`) reference this file rather than restating module roles; if a description here drifts from reality, fix it here once and every skill follows.
3+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
44
5−## Module map
5+## What This Is
66
7−Top-level Gradle modules. Code lives under each module's `src/main/`.
7+A Flutter demo app used as the target app for end-to-end testing of the [Maestro](https://github.com/mobile-dev-inc/maestro) mobile UI testing framework. It is not a production app — its screens exist to exercise specific Maestro features and reproduce specific bugs.
88
9−| Module | Role |
10−|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
11−| `maestro-android/` | On-device Android driver. Kotlin sources compile to two checked-in APKs (`maestro-app.apk`, `maestro-server.apk`) consumed by `maestro-client/`. The build's `copyMaestroAndroid` / `copyMaestroServer` finalizers update those APKs plus a `maestro-android-source.sha256` checksum. |
12−| `maestro-ios-driver/` | Host side of iOS driver wrapper (Kotlin). The actual XCTest runner lives in `maestro-ios-xctest-runner/`. |
13−| `maestro-ios-xctest-runner/` | Swift XCTest runner that runs on the iOS device/simulator. The compiled artifacts (`maestro-driver-ios*.zip`) are checked in under `maestro-ios-driver/src/main/resources/driver-iPhoneSimulator/Debug-iphonesimulator/`. |
14−| `maestro-ios/` | iOS host-side glue (small — most iOS host code lives in `maestro-client/`). |
15−| `maestro-client/` | Host-side Kotlin SDK that drives devices. Platform drivers live in `src/main/java/maestro/drivers/`: `AndroidDriver.kt`, `IOSDriver.kt`, `WebDriver.kt`, `CdpWebDriver.kt`. This is where most "auto-grant", "auto-dismiss", system-dialog handling and platform-specific quirks belong. |
16−| `maestro-orchestra/` | Command execution layer. `Orchestra.kt` interprets each Maestro command, applies retries, manages the command lifecycle. Sub-packages: `error/`, `filter/`, `workspace/`, `yaml/`. |
17−| `maestro-orchestra-models/` | Shared command/data models (used by `maestro-orchestra/` and consumers). |
18−| `maestro-cli/` | CLI entry point + MCP server. Mixed Kotlin (~100 files) + Swift (~56 files for iOS-related CLI bits). |
19−| `maestro-utils/` | Shared utilities. |
20−| `maestro-web/` | Web (browser) driver pieces. |
21−| `maestro-proto/` | Protobuf definitions shared across modules. |
22−| `maestro-test/` | Cross-module tests that doesn't require devices. |
9+When new screens or behaviors are needed to test a Maestro feature, they are added here.
2310
24−## E2E test fixtures (`e2e/`)
11+## Build Commands
2512
26−Shipped fixtures used by `.github/workflows/test-e2e.yaml`. Run via `e2e/run_tests <android|ios|web>` (see `e2e/run_tests` for env-var inputs `MAESTRO_APP`, `MAESTRO_FLOW_PATH`).
13+```sh
14+# Run the app (requires a connected device or emulator)
15+flutter run
2716
28−| Path | Role |
29−|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
30−| `e2e/demo_app/` | Flutter demo app whose only purpose is to exercise Maestro features. Contains its own `CLAUDE.md`. Built binaries are uploaded to a GCS bucket and re-downloaded by CI. |
31−| `e2e/demo_app/.maestro/` | Maestro flow YAMLs that drive the demo app. |
32−| `e2e/workspaces/` | Additional app workspaces (e.g. `simple_web_view`, `wikipedia`). |
33−| `e2e/run_tests` | Test driver invoked by the workflow. |
17+# Build Android APK
18+flutter build apk
3419
35−### `passing/` vs `failing/` suites
20+# Build iOS simulator app
21+flutter build ios --simulator
3622
37−Tag-based filters inside the YAML flows split test runs into two suites at execution time:
38−
39−- `passing/` — flows tagged `passing`. **Expected to pass.** Any failure here is a real regression. This is the only suite the diagnose agent reads.
40−- `failing/` — flows tagged `failing`. **Expected to fail** (negative-path coverage: assertions that should not match, commands that should error). The workflow inverts the success check on this suite. Do not treat `failing/` artifacts as regressions.
41−
42−Artifacts land at `<artifact_root>/tests/<app>/<suite>/`:
43−
44−## `test-e2e.yaml` workflow contract
45−
46−`.github/workflows/test-e2e.yaml` is the validation harness for both PR triggers and manual `workflow_dispatch` (e.g. validating a new Android API level or iOS version). Contract:
47−
48−- **`workflow_dispatch` inputs** — `android_version` (choice enum), `app` (string, default `demo_app`), `flow` (string, optional single-flow). The `validate-inputs` job rejects `android_version <= android-29`, missing `app` workspace, or ambiguous `flow`. (See PR #3226.)
49−- **`pull_request` triggers** are byte-identical to the prior behaviour; manual dispatches use the new narrowing knobs.
50−- **`test-android` job** boots an emulator on `system-images;${android_version};google_apis;x86_64` and runs `e2e/run_tests android`.
51−
52−Skills that bump platform versions (Android API levels, iOS versions) drive this workflow via `gh workflow run test-e2e.yaml --ref <branch> -f android_version=<...>`.
53−
54−## Testing
55−
56−Three layers — unit, integration, E2E — plus MCP-specific evals. Each layer has a different cost/coverage trade-off; default to the lowest layer that can express the test.
57−
58−### Unit tests (per module, `src/test/kotlin/`)
59−
60−Standard per-class tests. Stack: **JUnit 5** (`junit-jupiter-api` + `-params` + `-engine`), **Google Truth** for assertions, **MockK** for mocks. Each module's `build.gradle.kts` enables the platform via `tasks.named<Test>("test") { useJUnitPlatform() }`.
61−
62−```bash
63−./gradlew :maestro-orchestra:test # one module
64−./gradlew test # all modules
23+# Analyze code
24+flutter analyze
6525 ```
6626
67−### Integration tests (`maestro-test/`)
27+## Maestro Flow Commands
6828
69−Cross-module tests for behaviour that does **not** require a device or simulator — JS engine integration points, command orchestration end-to-end, cancellation / coroutine semantics. Notable suites:
29+Flows live in `.maestro/`. **Prefer the Maestro MCP** for authoring, running, and debugging flows interactively (`list_devices` → `inspect_screen` / `take_screenshot` → `run`): it returns the view hierarchy and screenshots inline, which is far more effective for iterating than parsing CLI output. Use the CLI below for scripted or CI-style runs, or when no MCP is available.
7030
71−- `IntegrationTest.kt` — full `Maestro` orchestration against an in-process `FakeDriver` (defined in `maestro-test/src/main/kotlin/maestro/test/drivers/`: `FakeDriver`, `FakeLayoutElement`, `FakeTimer`). Covers test-run cancellation (`CancellationException`, `withTimeout`, supervisor scopes) and the full command lifecycle without a real device.
72−- `GraalJsEngineTest.kt` / shared `JsEngineTest.kt` — Maestro's JS extension points (`evalScript`, JS-evaluated assertions/conditions). Exercises `org.graalvm.polyglot` directly.
73−- `FlowControllerTest.kt`, `DeepestMatchingElementTest.kt` — orchestration and view-hierarchy logic.
31+```sh
32+# Run all flows
33+maestro test .maestro/
7434
75−Stack: **JUnit 5**, **Google Truth**, **WireMock JRE8** (HTTP fakes), plus the in-house `FakeDriver` fixtures listed above. No mocks of Maestro's own classes — tests run real `Maestro` against the fakes.
35+# Run a single flow
36+maestro test .maestro/fill_form.yaml
7637
77−```bash
78−./gradlew :maestro-test:test
38+# Run flows with a specific tag
39+maestro test --include-tags passing .maestro/
7940 ```
8041
81−### E2E tests (`e2e/`)
42+**The MCP runs a *built* Maestro, not your working tree.** If you change Maestro framework code (anything outside `e2e/`) and want to validate it through the MCP against this app, the MCP will keep using the old build until Maestro is rebuilt **and the MCP is reconnected** (e.g. `/mcp reconnect maestro`). Rebuild, reconnect, then re-run. This is separate from rebuilding the demo app itself (see Build Commands) — a Dart/iOS/Android change to this app needs the app rebuilt and reinstalled on the device before the MCP will see it.
8243
83−Smoke-test every Maestro command across Android, iOS, and Web on real fixture apps. Maestro is its own dogfood harness: the CLI executes Maestro flow YAMLs against the fixtures, asserting both the framework's commands and the platform drivers behave correctly.
44+## Architecture
8445
85−Stack: **Maestro CLI itself** (dogfood) + `e2e/run_tests` shell driver + GHA workflow (`.github/workflows/test-e2e.yaml`). Fixture and suite layout is in "E2E test fixtures (`e2e/`)" above.
46+### Flutter App (`lib/`)
8647
87−```bash
88−cd e2e && ./run_tests <android|ios|web> # local
89−gh workflow run test-e2e.yaml --ref <branch> -f android_version=android-<N> # CI
90−```
48+`main.dart` is the home screen with buttons navigating to each test screen. Each screen is a standalone Dart file targeting a specific testing scenario:
9149
92−**Two roles for the same E2E setup.** The same suite serves both purposes — treat them identically:
50+| File | Purpose |
51+|---|---|
52+| `form_screen.dart` | Login form with email/password validation |
53+| `input_screen.dart` | Keyboard and text input behaviors |
54+| `swiping_screen.dart` | Swipe gesture testing |
55+| `nesting_screen.dart` | Deeply nested widget hierarchies |
56+| `location_screen.dart` | GPS location via `geolocator`, streams position updates |
57+| `sensors_screen.dart` | Device sensors (Android only) |
58+| `webview.dart` | Embedded WebView via `webview_flutter` |
59+| `defects_screen.dart` | Intentional UI quirks for defect regression |
60+| `cropped_screenshot_screen.dart` | Screenshot cropping edge cases |
61+| `notifications_permission_screen.dart` | Permission request flows |
62+| `permission_check_screen.dart` | Passively displays permission status (location, all-files) via `permission_handler` — never calls `requestPermission()`, so it reflects a pre-granted state deterministically |
63+| `issue_1619_repro.dart`, `issue_1677_repro.dart` | Bug reproductions |
9364
94−1. **Regression smoke** — every PR that touches Maestro source runs the suite on the current platform versions, catching behaviour breakage on existing platforms.
95−2. **New-OS validation** — when launching a new Android API level or iOS version, the same flows are dispatched against the new system image to confirm Maestro still works. This is what `bump-android-version` (and the planned `bump-ios-version`) drives.
65+The app reads launch arguments via `flutter_launch_arguments` (e.g., `initialCounter`, `delay`) so Maestro flows can configure app state at launch.
9666
97−A flow breaking for either reason is a real regression — fix in `maestro-android/`, `maestro-client/`, or `e2e/demo_app/`, not in `test-e2e.yaml` (see "What NOT to do").
67+### Maestro Flows (`.maestro/`)
9868
99−**Multiple apps for framework-specific coverage.** `demo_app/` (Flutter) is the default fixture and exercises every Maestro command. When a target is **framework-specific** (SwiftUI, React Native, Jetpack Compose specifics, WebView quirks, etc.), add a separate workspace under `e2e/workspaces/<app>/` with its own `.maestro/` flow YAMLs and a binary under `e2e/apps/`. Existing examples: `simple_web_view` (WebView coverage), `wikipedia` (real-world third-party app). The workflow's `app` input narrows a manual dispatch to one workspace: `... -f app=simple_web_view`.
69+- **Root flows** (`*.yaml`): Main passing/failing test cases, tagged `passing` or used to assert expected failures.
70+- **`commands/`**: Reusable Maestro command definitions (e.g., `assertVisible.yaml`, `inputText.yaml`).
71+- **`android_device_configuration/`** and **`ios_device_configuration/`**: Device setup flows run before tests (disable autocorrect, set timezone, enable sensors, etc.).
72+- **`web_flows/`**: Flows targeting web/WebView scenarios.
73+- **`issues/`**: Flows specifically reproducing reported Maestro bugs.
74+- **`experimental/`**: Unstable/in-progress flows not included in CI.
75+- **`scripts/`**: JavaScript helpers used by `evalScript` commands.
10076
101−### MCP server evals (`maestro-cli/src/test/mcp/`)
77+`config.yaml` configures which flow directories Maestro includes when running `maestro test .maestro/`.
10278
103−LLM-behaviour evaluations and tool-functionality tests for the MCP server inside `maestro-cli`. Stack: **`mcp-server-tester`** (npm package, run via `npx`) consuming YAML definitions (`full-evals.yaml`, `inspect-screen-evals.yaml`, `tool-tests-{with,without}-device.yaml`) plus per-platform setup scripts under `setup/`. See `maestro-cli/src/test/mcp/README.md` for the model list, scorers, and how to run.
79+**Platform targeting.** Write flows to run on both Android and iOS by default. Add an `android` or `ios` tag only when the behaviour is genuinely platform-specific — a flow with no platform tag runs on every platform. Prefer keeping a single cross-platform flow over splitting into per-platform files: use `${maestro.platform == "android" ? ... : ...}` for platform-specific values, and `runFlow` with `when: platform:` to guard platform-specific steps.
10480
105−```bash
106−./run_mcp_tool_tests.sh ios # tool-functionality (fast)
107−./run_mcp_evals.sh ios # LLM behaviour (slower)
108−```
81+### App ID
10982
110−## Conventions
83+All flows target `appId: com.example.example`.
11184
112−- Kotlin 1.9 / JVM 17. Gradle. No DI framework — services are constructed manually.
113−- Protobuf for the on-device wire format (`maestro-proto/`).
114−- Coroutines with explicit dispatchers; `runBlocking` only at entry points.
115−- Exposed exceptions classify failures (retryable vs terminal) — see `maestro-orchestra/src/main/java/maestro/orchestra/error/`.
116−- **Temp files and directories go through `maestro.utils.TempFileHandler`**, not `java.nio.file.Files.createTempFile/createTempDirectory` directly. `TempFileHandler` is a `Closeable` that recursively cleans up everything it allocated on `close()`. Direct `Files.createTempFile(...)` skips that lifecycle and leaks `/tmp` content (especially painful on long-lived JVMs like the cloud worker). Construct a `TempFileHandler` near the lifecycle owner, call its `createTempFile` / `createTempDirectory`, and `close()` it in a `finally`.
85+## Testing Permissions
11786
118−## Where Claude Code resources live
87+Non-obvious gotchas when writing permission flows against this app:
11988
120−- `.claude/skills/*` — skills (workflows). Each skill's `SKILL.md` references this file for module roles.
121−- `.claude/agents/*.md` — subagents (e.g. `diagnose-maestro-failure.md`). Their input/output contracts are documented in each file.
89+- **iOS: each `permission_handler` permission must be enabled in `ios/Podfile`.** A permission's handler is compiled in only when its macro is set in `GCC_PREPROCESSOR_DEFINITIONS` (e.g. `PERMISSION_LOCATION=1`). Without it, that permission's `.status` **silently returns denied on iOS** regardless of the real authorization. Enabled today: notifications, location. After editing the Podfile, run `pod install` in `ios/` before `flutter build ios` — a Podfile edit alone won't trigger it.
90+- **Android: runtime permissions must be declared in `AndroidManifest.xml`** or they can't be granted (`pm grant` throws). Declared today: INTERNET, ACCESS_FINE/COARSE_LOCATION, MANAGE_EXTERNAL_STORAGE. (So e.g. POST_NOTIFICATIONS can't be granted here.)
91+- **Permission values are platform-specific.** Android uses `allow`/`deny`/`unset`; iOS `location` uses `always`/`inuse`/`never`/`unset`. iOS *validates* location values and throws on anything else; Android silently falls back to revoke for unknown/empty values. For cross-platform flows, pick per platform: `location: ${maestro.platform == "android" ? "allow" : "always"}`.
92+- **`launchApp` with no `permissions:` block defaults to `all: allow`.**
93+- **Observe passively.** `permission_check_screen.dart` reads status without requesting — use it to assert a pre-granted state on both platforms. Avoid `location_screen.dart` (geolocator) for that: it actively calls `requestPermission()`, popping a dialog (Android does not auto-dismiss it) and resolving position asynchronously.
94+- **`MANAGE_EXTERNAL_STORAGE` is an appOps permission** on Android (special path), not a standard `pm grant` runtime permission.
12295
123−## What NOT to do
96+## Adding New Test Screens
12497
125−- Don't fix driver-behaviour gaps by patching `.github/workflows/test-e2e.yaml` (e.g. extra `adb shell settings put …`, command-line tweaks, AVD pre-config). Workflow band-aids hide the regression from users running Maestro outside our CI. Fix `maestro-android/`, `maestro-client/`, or `e2e/demo_app/` instead so the fix ships with the driver APKs. Workflow edits are valid for shape-changes (matrix, retention, dispatch inputs) and the narrow third-party-FRE exception documented in skill files.
126−- Don't edit checked-in driver artifacts (`maestro-app.apk`, `maestro-server.apk`, `maestro-android-source.sha256`, `maestro-driver-ios*.zip`) by hand — they are gradle finalizer outputs.
127−- Don't commit local changes to the iOS driver zips. A local build regenerates two zips — `maestro-driver-ios.zip` and `maestro-driver-iosUITests-Runner.zip` — under `maestro-ios-driver/src/main/resources/driver-iPhoneSimulator/Debug-iphonesimulator/`. This is normal: they're used by local builds. But although they're checked into the repo, they're managed exclusively by CI, so leave any local modifications to them out of your commits.
128−- Don't modify existing flows in `failing/` to make them pass — that's the negative-path suite by design.
98+1. Create a new `lib/<feature>_screen.dart` with a `StatefulWidget`.
99+2. Add a navigation button in `lib/main.dart`.
100+3. Write the flow that exercises the **Maestro feature** you are testing, tagged `[passing]`. The flow is the point — the screen exists only to make that Maestro behaviour observable, not to be tested itself. Add a platform tag only if the behaviour is platform-specific (see **Platform targeting** above).
101+4. For location or sensor tests, ensure relevant device configuration flows exist in the platform-specific subdirectories.
129102
