AGENTS.md

1## Pytest Integration Tests
2 
3 
4- Look to `app/factories/` to generate any required database state
5  - Here's an example of how to create + persist a factory `DistributionFactory.save(domain=PYTHON_TEST_SERVER_HOST)`
6- Add the `server` factory to each test
7- Use the `faker` factory to generate emails, etc.
8- Don't add obvious `assert` descriptions
9- Do not use the `db_session` fixture here. Instead, use `with test_session():` if you need to setup complex database state
10- if a UI timeout is occuring, it could be because it cannot find a element because the rendering has changed. Check the failure screenshot and see if you can correct the test assertion.
11- The integration tests can take a very long time to run. Do not abort them if they are taking a long time.
12- Use `expect(page.get_by_text("Screening is fully booked")).to_be_visible()` instead of `expect(page.get_by_role("heading")).to_contain_text("Screening is fully booked")`. It's less brittle.
13- Do not use `client` fixtures in an integration test. Integration tests should only use the frontend of the website to interact with the application, not the API.
14- Use `with page.expect_response("https://example.com/resource") as response_info:` to assert against network activity.
15- Do not `next_button.evaluate("el => el.click()")` instead, just `locator.click()`. If this doesn't work, stop your work and let me know.
16- Only use `wait_for_loading(page)` if a `LONG_INTEGRATION_TEST_TIMEOUT` on an expectation does not work: `expect(page.get_by_text("Your Matched Doctors")).to_be_visible(timeout=LONG_INTEGRATION_TEST_TIMEOUT)`
17  - `LONG_INTEGRATION_TEST_TIMEOUT` should only be used as a last resort. If you have many of these in a test, let me know and I will debug it.
18- Prefer fewer integration tests that cover more functionality. Unlike unit tests, where each test is designed to test a very particular piece of functionality, I want integration tests to cover entire workflows. It's preferred to add more steps to an integration test to test an entire workflow.
19- Prefer simple locators. If a `filter`, `or_`, etc is required to capture a button in multiple states it indicates something is wrong in the code.
20- Use `react_router_url` to generate the frontend url path and do not set `base_url`.
21- End all Playwright tests with `from pytest_playwright_artifacts import assert_no_console_errors` and `assert_no_console_errors(request)` (capture is the plugin's `playwright_console_logging` fixture).
22  - Test-Specific Ignores: Pass `ignore=[...]` to `assert_no_console_errors` per `pytest-playwright-artifacts` (regex strings, compiled patterns, or `{"file": "...", "message": "..."}` dicts); add a comment explaining why.
23  - Global Ignores: Use `playwright_console_ignore` under `[tool.pytest.ini_options]` in `pyproject.toml` (see `pytest-playwright-artifacts` README).
24 
25### Example Integration Test
26 
27Below is an example test. Notice the following:
28 
29- Code comments are used to describe the key user steps that are being tested
30- We avoid long timeouts or wait commands
31- Fixtures and factories are generated at the beginning of the test
32- We assert against database state after each major user action
33- Some of the comments (i.e. comments on included fixtures) are included for instructional purposes only and should not be included in the tests you write
34 
35```python
36from pytest_playwright_artifacts import assert_no_console_errors
37 
38 
39def test_streaming_checkout_creates_user_and_links_order(
40    # this fixture ensures that the underlying python server is started
41    server,
42    faker,
43    page: Page,
44    # if you need to create objects (like factories) tied to a common session, include this fixture
45    db_truncate_session,
46    # for asserting against the console logs
47    request: FixtureRequest,
48) -> None:
49    distribution = DistributionWithWebhooksFactory.save()
50    test_email = clerk_test_email()
51 
52    # 1) Run through streaming checkout
53    page.goto(react_router_url("/streaming"))
54 
55    page.get_by_placeholder("your@email.com").first.fill(test_email)
56    page.get_by_placeholder("your@email.com").last.fill(test_email)
57 
58    # Check TOS
59    page.get_by_role("checkbox").last.check()
60 
61    fill_stripe_checkout(page)
62 
63    # 2) Complete checkout
64    # the stripe checkout form will expand and cause the purchase button to move below the screen
65    safely_scroll_then_click(page.get_by_role("button", name="Complete Purchase"))
66 
67    expect(page.get_by_role("heading", name="You're ready to watch!")).to_be_visible()
68    page.get_by_role("link", name="Login & Start Watching").click()
69 
70    # 3) Assert against database and Clerk state
71 
72    # One StreamingOrder should be created for this distribution and email
73    assert StreamingOrder.count() == 1
74    streaming_order = StreamingOrder.get(email=test_email)
75    assert streaming_order
76 
77    assert streaming_order.status.value == "completed"
78    assert streaming_order.user_id is not None
79 
80    # 4) Login to the app via the Clerk login page
81    clerk_login_and_verify(page, test_email)
82 
83    expect(page.get_by_text("Triumph of the Heart").first).to_be_visible()
84 
85    # 5) Play the video
86    page.get_by_role("button").filter(has_text="Play").click()
87 
88    # wait for the page to completely load
89    wait_for_loading(page)
90 
91    expect(page.get_by_text("Triumph of the Heart").first).to_be_visible()
92    expect(page.get_by_text("Play")).not_to_be_visible()
93 
94    assert_no_console_errors(
95        request,
96        ignore=[
97            {
98              # if there are console errors specific to the project, exclude them here. Match to the specific URL if you can.
99              "file": r"https://iframe.cloudflarestream.com/.*",
100              "message": "the server responded with a status of 403",
101            }
102        ],
103    )
104```
105 
106 
107## Pytest Tests
108 
109 
110- Look first to `app.factories.*` instead of `app.models.*` to generate any required database state
111  - For example, to create and persist a `Distribution` record `DistributionFactory.save()`
112  - If a factory doesn't exist for the model you are working with, create one.
113  - You can customize one or more params in a factory using `DistributionFactory.save(host="custom_host.com)`
114- Use `faker` factory to generate emails, etc.
115- Do not mock or patch unless I instruct you to. Test as much of the application stack as possible in each test.
116- If you get lazy attribute errors, or need a database session to share across logic, use the `db_session` fixture to fix the issue.
117  - Note that when writing route tests a `db_session` is not needed for the logic inside of the route.
118- When testing Stripe, use the sandbox API. Never mock out Stripe interactions unless explicitly told to.
119- Omit obvious docstrs and comments. Add comments for non-obvious but easy-to-miss lines that are key to what the test is checking.
120- If a docstring needs formatting, use markdown. Use Google Style.
121 
122### Example Test
123 
124Below is an example test, you'll notice the following:
125 
126- Docstr is omitted since the purpose of the test is obvious
127- Comment about the `county` is added since it's the main point of the test
128- Newline between test setup, functionality under test, and assertions against result
129- `api_app_url_path_for` helper is used instead of hardcoded routes
130 
131```python
132from app.generated.fastapi_typed_routes import api_app_url_path_for
133import json
134 
135def test_calculate_quote_unknown_county(client):
136    payload = {
137        "subscriber": {"age": 35, "gender": "M"},
138        # fake county to ensure error is thrown
139        "county": "NotACounty",
140    }
141 
142    response = client.post(
143        api_app_url_path_for("composite_quote"),
144        json=payload,
145    )
146 
147    assert_status(response, status.HTTP_422_UNPROCESSABLE_CONTENT)
148```
149 
150### File Structure
151 
152* If there's more than a handful of tests in a folder, you should probably create subfolders.
153* File name should be related to the file or primary class / functionality the test is covering. Do not add a component to the test name that exists in the test file path.
154  * Example: `app/routes/unauthenticated/quote.py` should be `tests/routes/unauthenticated/quote_test.py`
155* `tests/routes/{unauthenticated,authenticated}` and a handful of top-level test files for fastapi API route testing.
156* `tests/integration/` for browser tests
157 
158 
159## Python Route Tests
160 
161 
162- Polyfactory is the [factory](app/factories/) library in use. `ModelNameFactory.build()` is how you generate factories.
163- Use `assert_status(response)` instead of `assert response.status_code == status.HTTP_200_OK`
164- Do not reference routes by raw strings. Instead, use the typed route helpers defined in `app/generated/fastapi_typed_routes.py`.
165