CLAUDE.md

1When working with a branch, do not use rebase or amend - add new commits instead.
2 
3Do not commit to the master branch. Create a new branch for every task.
4 
5Do not create stacked pull requests. Every pull request must target `master` directly (or, in rare cases, a release branch), not another feature branch. CI only runs for pull requests whose base is `master` or a release branch, so a pull request stacked on another feature branch gets no checks. If a change depends on unmerged work, wait for that work to merge into `master` first, or include all the changes in a single pull request.
6 
7When writing text such as documentation, comments, or commit messages, wrap literal names from ClickHouse SQL language, classes and functions, or literal excerpts from log messages inside inline code blocks, such as: `MergeTree`.
8 
9When adding headers to documentation files under `docs/`, every header must include an explicit anchor in the form `{#kebab-case-anchor}` at the end of the header line, e.g. `## My Section {#my-section}`. This is mandatory for all heading levels. New documentation files must also include a frontmatter block at the top (before the first heading) with `description`, `sidebar_label`, `sidebar_position`, `slug`, `title`, and `doc_type` fields, modelled on existing files such as `docs/en/development/continuous-integration.md`.
10 
11When writing text such as documentation, comments, or commit messages, write names of functions and methods as `f` instead of `f()` - we prefer it for mathematical purity when it refers a function itself rather than its application.
12 
13Only `docs/en/` is hand-editable documentation source. Never edit the translated trees (`docs/ar/`, `docs/es/`, `docs/fr/`, `docs/ja/`, `docs/ko/`, `docs/pt-BR/`, `docs/ru/`, `docs/zh/`) or the generated trees (`docs/reference/`, `docs/_site/`, `docs/snippets/`); they are produced from the English source and from C++ definitions by separate translation and generation pipelines, and any direct edit is overwritten by the next sync. The CI check "No direct edits to generated or read-only docs" fails edits inside `AUTOGENERATED` regions in the generated trees. When a change needs documentation updates, edit only `docs/en/` (and the relevant C++ embedded documentation strings) and leave every other `docs/` tree untouched.
14 
15Whenever changes are added, modified, or deleted that relate to the `Native` format - its wire/serialization format, type encodings (e.g. `LowCardinality`, `Array`, `Map`, `Variant`, `Dynamic`, `JSON`), the block/column structure, the compression frame, the `NativeReader`/`NativeWriter`, or the user-facing doc `docs/en/interfaces/formats/Native.md` - also update the official specification `docs/en/interfaces/specs/NativeFormat.md` (slug `/interfaces/specs/NativeFormat`) accordingly in the same change. The spec is the single source of truth for the format; if unsure whether the spec needs updating, flag it.
16 
17When mentioning logical errors, say "exception" instead of "crash", because they don't crash the server in the release build.
18 
19Links to ClickHouse CI should be analyzed using the tool at `.claude/tools/fetch_ci_report.js`, which directly fetches the underlying JSON data without requiring a browser. It accepts GitHub PR URLs (fetches all CI reports) or direct S3/CI HTML URLs.
20 
21```bash
22# Fetch all CI reports for a PR
23node .claude/tools/fetch_ci_report.js "https://github.com/ClickHouse/ClickHouse/pull/12345"
24 
25# Show only failed tests with CIDB links
26node .claude/tools/fetch_ci_report.js "https://github.com/ClickHouse/ClickHouse/pull/12345" --failed --cidb
27 
28# Fetch only a specific report from a PR (by index)
29node .claude/tools/fetch_ci_report.js "https://github.com/ClickHouse/ClickHouse/pull/12345" --report 2
30 
31# Filter by test name, show artifact links
32node .claude/tools/fetch_ci_report.js "<url>" --test peak_memory --links
33 
34# Download logs and show failed tests
35node .claude/tools/fetch_ci_report.js "<url>" --failed --download-logs
36 
37# Options:
38#   --test <name>               Filter tests by name
39#   --failed                    Show only failed tests
40#   --all                       Show all test results
41#   --links                     Show artifact links (logs.tar.gz, etc.)
42#   --cidb                      Show CIDB links for failed tests
43#   --report <number>           For PR URLs: fetch only one specific report
44#   --download-logs [path]      Download logs to path (default: /tmp/ci_logs.tar.{gz,zst})
45#   --credentials <user,password>  HTTP Basic Auth for private repositories
46```
47 
48After downloading logs, extract specific test logs:
49```bash
50tar -xzf /tmp/ci_logs.tar.gz ci/tmp/pytest_parallel.jsonl
51grep &quot;test_name&quot; ci/tmp/pytest_parallel.jsonl | python3 -c &quot;import sys,json; [print(json.loads(l).get('longrepr','')) for l in sys.stdin if 'failed' in l]&quot;
52```
53 
54To analyze CI performance comparison results (slower/faster queries, unstable queries), use the tool at `.claude/tools/fetch_perf_report.py`. It fetches the machine-readable `all-query-metrics.tsv` from S3 for each performance shard, filters to `client_time`, and classifies queries as changed or unstable using the same thresholds as `compare.sh`.
55 
56```bash
57# Show performance changes for a PR (default: changed + unstable queries only)
58python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot;
59 
60# Filter by architecture
61python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot; --arch amd
62 
63# Show only per-shard summary (no individual queries)
64python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot; --summary
65 
66# Filter by test name
67python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot; --test group_by
68 
69# Show all queries (not just changes)
70python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot; --all --sort times
71 
72# JSON output for structured analysis
73python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot; --json
74 
75# TSV output for piping
76python3 .claude/tools/fetch_perf_report.py &quot;https://github.com/ClickHouse/ClickHouse/pull/12345&quot; --tsv
77 
78# Also accepts CI HTML URLs
79python3 .claude/tools/fetch_perf_report.py &quot;https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=12345&amp;sha=abc123&quot;
80```
81 
82Key options: `--arch <amd|arm|all>` to filter architecture, `--metric <name>` to change metric (default `client_time`), `--shard <n>` for a specific shard, `--test <name>` / `--query <text>` for substring filtering, `--sort <diff|times|threshold|test>` for ordering, `--summary` for shard-level overview only, `--json` / `--tsv` for machine-readable output.
83 
84To compile and run C++ code snippets against the ClickHouse codebase without modifying any source files, use the tool at `.claude/tools/cppexpr.sh`. This is a wrapper around `utils/c++expr` that auto-detects build directories and handles working directory setup. When asked about the size, layout, or alignment of ClickHouse data structures, or asked to compare performance of code snippets, use this tool to get a definitive answer instead of guessing.
85 
86```bash
87# Query the size of a ClickHouse data structure
88.claude/tools/cppexpr.sh -i Core/Block.h 'OUT(sizeof(DB::Block))'
89 
90# Query multiple expressions at once
91.claude/tools/cppexpr.sh -i Core/Field.h 'OUT(sizeof(DB::Field)) OUT(sizeof(DB::Array))'
92 
93# Use global code for helper functions or custom types
94.claude/tools/cppexpr.sh -g 'struct Foo { int a; double b; };' 'OUT(sizeof(Foo)) OUT(alignof(Foo))'
95 
96# Benchmark a code snippet (100000 iterations, 5 tests)
97.claude/tools/cppexpr.sh -i Common/Stopwatch.h -b 100000 'Stopwatch sw;'
98 
99# Standalone mode (no ClickHouse headers, just standard C++)
100.claude/tools/cppexpr.sh --plain 'OUT(sizeof(std::string))'
101```
102 
103Key options: `-i HEADER` to include headers, `-g 'CODE'` for global-scope code, `-b STEPS` for benchmarking, `-l LIB` to link extra libraries, `--plain` for standalone compilation without ClickHouse. The `OUT(expr)` macro prints `expr -> value`.
104 
105When asked to analyze assembly, inspect generated code, find register spills, check branch density, compare codegen between builds, or investigate optimization opportunities in compiled functions, use the tool at `.claude/tools/analyze-assembly.py`. It disassembles functions from a compiled binary, builds a CFG, computes metrics (spill/branch/call density), and reports findings. Use it instead of manually running `llvm-objdump` or `llvm-nm`.
106 
107```bash
108# Basic analysis of a function
109python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;&lt;function_name&gt;&quot;
110 
111# Search for overloaded/templated functions by regex
112python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;insertRangeFrom&quot; --search
113 
114# Pick a specific overload from ambiguous results
115python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;insertRangeFrom&quot; --search --select 3
116 
117# JSON output for structured analysis
118python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;&lt;function_name&gt;&quot; --format json
119 
120# Source-interleaved disassembly (needs debug info)
121python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;&lt;function_name&gt;&quot; --source
122 
123# Microarchitectural analysis of loop bodies (--mcpu is required)
124python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;&lt;function_name&gt;&quot; --mca --mcpu=znver3
125 
126# Profile-weighted analysis (re-ranks findings by runtime impact)
127python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;&lt;function_name&gt;&quot; --perf-map tmp/perf.map.jsonl
128 
129# Compare codegen between two builds
130python3 .claude/tools/analyze-assembly.py --before &lt;old_binary&gt; --after &lt;new_binary&gt; &quot;&lt;function_name&gt;&quot;
131 
132# Analyze function at a specific address (useful for heavily-templated symbols)
133python3 .claude/tools/analyze-assembly.py &lt;binary&gt; 0x0dc7c780
134 
135# Verbose mode to see tool commands
136python3 .claude/tools/analyze-assembly.py &lt;binary&gt; &quot;&lt;function_name&gt;&quot; -v
137```
138 
139Key options: `--search` for regex matching, `--fuzzy` for substring matching, `--select N` to pick from ambiguous results, `--all` to analyze all matches, `--context N` to show surrounding symbols, `--max-instructions N` to control output size, `--mca --mcpu=<model>` for llvm-mca throughput analysis, `--perf-map <file>` for runtime-weighted scoring, `--before`/`--after` for diff mode. Hex addresses (e.g. `0x0dc7c780`) are resolved to the enclosing symbol automatically — useful when symbol names are too long for regex matching. The tool caches symbol tables by build-id for fast repeated queries.
140 
141You can build multiple versions of ClickHouse inside `build_*` directories, such as `build`, `build_debug`, `build_asan`, etc.
142 
143You can run integration tests as in `tests/integration/README.md` using: `python -m ci.praktika run "integration" --test <selectors>` invoked from the repository root.
144 
145When writing tests, do not add "no-*" tags (like "no-parallel") unless strictly necessarily.
146 
147When writing tests in tests/queries, prefer adding a new test instead of extending existing ones.
148 
149When removing a feature, do not write tests asserting that the feature no longer exists (for example, a test checking that a removed function or setting now throws an error). Instead, delete the tests of the removed feature. Such tests only pin down the absence of something and become noise.
150 
151When adding a new test, use `./tests/queries/0_stateless/add-test <name>` for `.sql` tests or `./tests/queries/0_stateless/add-test <name>.sh` for `.sh` tests. It assigns the next available number prefix and creates both the test and reference files.
152 
153When writing C++ code, always use Allman-style braces (opening brace on a new line). This is enforced by the style check in CI.
154 
155Never use sleep in C++ code to fix race conditions - this is stupid and not acceptable!
156 
157Avoid fallback paths. When an operation fails, prefer letting the error propagate over silently substituting a default value or alternate behavior. Fallbacks hide bugs and make incidents harder to diagnose. If a fallback is genuinely needed, follow the fail-close principle: never perform a destructive, expensive, or otherwise consequential action on the fallback path. Skip the operation and surface the error instead — for example, when label-attribution data is unavailable, do not assume "human-added" and create backports anyway; let the run fail and retry once the data is available.
158 
159When writing messages, say ASan, not ASAN, and similar (because there are two words: Address Sanitizer).
160 
161When checking the CI status, pay attention to the comment from robot with the links first. Look at the Praktika reports first. The logs of GitHub actions usually contain less info.
162 
163Do not use `-j` argument with ninja; do not use `nproc` - let it decide automatically.
164 
165When building ClickHouse (running ninja), always redirect output to the build log file in the build directory. Always use a subagent to analyze the log and return only a concise summary.
166 
167When running tests, always redirect output to a log file in the build directory (e.g. `<build_directory>/test_<test_name>.log`). Use unique file names per test so multiple tests can run in parallel. Always use a subagent to analyze each log and return only a concise summary.
168 
169If I provided a URL with the CI report, logs, or examples, include it in the commit message. If the link has `PR=...`, also add a link to the corresponding PR.
170 
171When creating or updating a pull request, use `.github/PULL_REQUEST_TEMPLATE.md` as the PR body template. The body should contain: a short description of the change and motivation, then the Changelog category (leave one from the list), then the Changelog entry. Do not invent a custom "## Summary" or "## Test plan" structure — follow the template exactly. The "Bug Fix" category should be used only for real bug fixes, while for fixing CI reports you can use the "CI Fix or improvement" category. Include the URL to CI report I provided if any. If the PR is about a CI failure, search for the corresponding open issues and provide a link in the PR description.
172 
173Link related pull requests and issues explicitly, using full GitHub URLs, one relationship per line:
174- When a pull request fixes an issue, put `Closes: <full link to the issue>` on its own line in the pull request description. GitHub renders this as `Closes: #<number>` and closes the issue automatically when the pull request is merged into the default branch (auto-close only fires when targeting the default branch).
175- When an issue was caused by a pull request (a regression), put `Caused by: <full link to the pull request>` on its own line in the issue.
176- For any other relevant pull request or issue, put `Related: <full link>` on its own line.
177 
178Use the keyword `Closes` (not `Fixes` or `Resolves`) for consistency, even though GitHub also auto-closes on `Fixes` and `Resolves`. `Caused by` and `Related` are not GitHub keywords and trigger no automatic closing; they are conventions for humans and tooling. Issues never close pull requests, so the issue side uses only `Caused by` and `Related`.
179 
180ARM machines in CI are not slow. They are similar to x86 in performance.
181 
182Use `tmp` subdirectory in the current directory for temporary files (logs, downloads, scripts, etc.), do not use `/tmp`. Create the directory if needed.
183 
184