| Dimension | Shared | Only in A | Only in B | Overlap |
|---|---|---|---|---|
| Sections | 2 | 21 | 15 | 5% |
| Commands | 0 | 3 | 7 | 0% |
| Section tags | 4 | 0 | 5 | 44% |
What each file covers
Sections
2 shared · 21 only in A · 15 only in B- − torch/_dynamo
- − Key Abstractions
- − VariableTracker (`variables/`)
- − Source (`source.py`)
- − Guards (`guards.py`)
- − Side Effects (`side_effects.py`)
- − Other key files
- − Graph Breaks
- − Common patterns
- − Debugging
- − TORCH_LOGS
- − Reproducing crashes
- − Structured tracing (for production)
- − Compile-time profiling
- − comptime.breakpoint() (`comptime.py`)
- − Bytecode Debugger (`bytecode_debugger.py`)
- − C++ Runtime (`torch/csrc/dynamo/`)
- − Frame Evaluation
- − Cache
- − Guard Evaluation Tree (`guards.cpp`)
- − Other C++ files
- + PyTorch Copilot Instructions
- + Core Components
- + The Code Generation Workflow
- + Development Workflows
- + Building from Source
- + Linting
- + Project-Specific Conventions
- + Memory and Storage
- + Python-C++ Integration (`torch/csrc/`)
- + Dispatch System
- + Git Workflow (AI Agent Specific)
- + Resolve conflicts if necessary
- + Common Gotchas
- + Key Files Reference
- + Performance Debugging
- Architecture Overview
- Testing
Commands
0 shared · 3 only in A · 7 only in B- − python test/dynamo/test_misc.py
- − python test/dynamo/test_misc.py MiscTests.test_foo
- − python test/dynamo/test_misc.py -k test_foo
- + python -m pip install --no-build-isolation -v -e .
- + python test/test_torch.py TestTorch.test_specific_case
- + git stash -u
- + git reset --hard $(cat /tmp/orig_work.txt)
- + git stash pop
- + ninja
- + pip install ninja
Section tags
4 shared · 0 only in A · 5 only in B- + setup
- + git-pr
- + performance
- + do-not
- + agent-behaviour
- build
- test
- code-style
- architecture
Line diff
pytorch/pytorch · torch/_dynamo/CLAUDE.md
@@ −1 @@
1# torch/_dynamo
2
3TorchDynamo is a Python-level JIT compiler that captures PyTorch programs into
4FX graphs by symbolically executing Python bytecode. It hooks into CPython's
5PEP 523 frame evaluation API to intercept execution, traces operations into an
6FX graph, compiles the graph with a backend (e.g. Inductor), and generates new
7bytecode that calls the compiled code.
8
9## Architecture Overview
10
11The compilation pipeline, in execution order:
12
131. **`eval_frame.py`** — Runtime entry point. `torch.compile()` wraps a
14 function in an `OptimizedModule`. At runtime, the C extension
15 (`torch._C._dynamo.eval_frame`) intercepts Python frames via PEP 523.
162. **`convert_frame.py`** — `ConvertFrameAssert.__call__` checks caches,
17 handles recompilation limits, calls `_compile()` → `trace_frame()`.
183. **`symbolic_convert.py`** — The heart of Dynamo. `InstructionTranslator`
19 symbolically executes bytecode instruction-by-instruction. Maintains a
20 symbolic `stack` (list of `VariableTracker`s) and `symbolic_locals` (dict of
21 name → `VariableTracker`). Opcodes are dispatched via a `dispatch_table`
22 built by `BytecodeDispatchTableMeta`.
234. **`output_graph.py`** — `OutputGraph` owns the FX graph being built (via
24 `SubgraphTracer`), the `SideEffects` tracker, guards, shape environment,
25 and graph args. `compile_subgraph()` finalizes the graph, calls the backend,
26 and generates output bytecode.
275. **`codegen.py`** — `PyCodegen` emits output bytecode: loads graph inputs
28 (via `Source.reconstruct()`), calls the compiled graph, unpacks outputs, and
29 replays side effects.
306. **`resume_execution.py`** — Generates continuation functions for execution
31 after graph breaks.
32
33## Key Abstractions
34
35### VariableTracker (`variables/`)
36
37Every Python value encountered during tracing is wrapped in a `VariableTracker`
38subclass. Key interface: `as_python_constant()`, `as_proxy()`,
39`call_function()`, `call_method()`, `getattro_impl()`, `reconstruct()`.
40
41Key fields: `source` (where the value came from, for guards) and
42`mutation_type` (whether/how mutations are tracked).
43
44**Factory**: `VariableTracker.build(tx, value, source=...)` dispatches to
45`VariableBuilder` (sourced values needing guards) or `SourcelessBuilder`
46(values created during tracing).
47
48Key subclass families in `variables/`: `TensorVariable` / `SymNodeVariable`
49(tensor.py), `ConstantVariable` (constant.py), `ListVariable` /
50`TupleVariable` (lists.py), `ConstDictVariable` (dicts.py), `SetVariable` (sets.py),
51`UserFunctionVariable` (functions.py), `BuiltinVariable` (builtin.py),
52`NNModuleVariable` (nn_module.py), `UserDefinedObjectVariable`
53(user_defined.py), `TorchHigherOrderOperatorVariable` (higher_order_ops.py),
54`LazyVariableTracker` (lazy.py). `VariableBuilder` and `SourcelessBuilder` are
55in builder.py.
56
57### Source (`source.py`)
58
59Tracks value provenance — how to access a value at runtime. Used for guard
60generation (`source.make_guard(GuardBuilder.XXX)`) and bytecode reconstruction
61(`source.reconstruct(codegen)`). Root sources: `LocalSource`, `GlobalSource`.
62Chained sources: `AttrSource`, `GetItemSource`, `NNModuleSource`, etc.
63
64### Guards (`guards.py`)
65
66Runtime conditions that must hold for cached compiled code to be reused.
67Install via `install_guard(source.make_guard(GuardBuilder.TYPE_MATCH))`.
68Common types: `TYPE_MATCH`, `ID_MATCH`, `EQUALS_MATCH`, `TENSOR_MATCH`,
69`SEQUENCE_LENGTH`. At finalization, `CheckFunctionManager` builds a tree of
70C++ `GuardManager` objects for fast runtime checking.
71
72### Side Effects (`side_effects.py`)
73
74Tracks mutations during tracing (attribute stores, list mutations, cell
75variable updates, tensor hooks) and replays them as bytecode after graph
76execution. The `MutationType` system (`variables/base.py`) controls what
77mutations are allowed: `ValueMutationNew/Existing`,
78`AttributeMutationNew/Existing`, or `None` (immutable). The `scope` field
79prevents cross-scope mutations inside higher-order operators.
80
81### Other key files
82
83- `trace_rules.py` — inline/skip/graph-break decisions per function
84- `exc.py` — exception hierarchy: `Unsupported` (graph break), `RestartAnalysis`
85 (restart tracing), `ObservedException` (user exceptions during tracing),
86 `BackendCompilerFailed`
87- `config.py` — configuration flags, supports `config.patch()` context
88 manager/decorator
89- `bytecode_transformation.py` / `bytecode_analysis.py` — low-level bytecode
90 manipulation, liveness analysis
91- `pgo.py` — profile-guided optimization for dynamic shapes
92- `polyfills/` — traceable replacements for stdlib functions
93- `repro/` — reproduction/minification tools
94
95## Graph Breaks
96
97Call `unimplemented()` (from `exc.py`) to trigger a graph break:
98
99```python
100from torch._dynamo.exc import unimplemented
101from torch._dynamo import graph_break_hints
102
103unimplemented(
104 gb_type="short_category_name",
105 context=f"dynamic details: {value}",
106 explanation="Human-readable explanation of why this breaks the graph.",
107 hints=[*graph_break_hints.SUPPORTABLE],
108)
109```
110
111- `gb_type`: Context-free category (no dynamic strings).
112- `context`: Developer-facing details (can be dynamic).
113- `explanation`: User-facing explanation (can be dynamic).
114- `hints`: Use constants from `graph_break_hints.py`: `SUPPORTABLE`,
115 `FUNDAMENTAL`, `DIFFICULT`, `DYNAMO_BUG`, `USER_ERROR`,
116 `CAUSED_BY_EARLIER_GRAPH_BREAK`.
117
118The `break_graph_if_unsupported` decorator on instruction handlers catches
119`Unsupported`, logs the graph break, updates the `SpeculationLog`, and restarts
120analysis. On the second pass, the partial graph is compiled at the break point
121and a resume function handles the rest.
122
123## Testing
124
125Tests live in `test/dynamo/`. Use `torch._dynamo.test_case.TestCase` as base
126class — it calls `torch._dynamo.reset()` in setUp/tearDown and patches config
127for strict error checking.
128
129```bash
130python test/dynamo/test_misc.py # whole file
131python test/dynamo/test_misc.py MiscTests.test_foo # single test
132python test/dynamo/test_misc.py -k test_foo # pattern match
133```
134
135### Common patterns
136
137The default backend to `torch.compile()` is `backend="eager"`.
138
139**CompileCounter** — count compilations and graph ops:
140```python
141cnt = torch._dynamo.testing.CompileCounter()
142
143@torch.compile(backend=cnt)
144def fn(x):
145 return x + 1
146
147fn(torch.randn(10))
148self.assertEqual(cnt.frame_count, 1)
149self.assertEqual(cnt.op_count, 1)
150```
151
152**fullgraph=True** — assert no graph breaks:
153```python
154torch.compile(fn, backend="eager", fullgraph=True)(x)
155```
156
157**EagerAndRecordGraphs** — inspect captured FX graphs:
158```python
159backend = torch._dynamo.testing.EagerAndRecordGraphs()
160torch.compile(fn, backend=backend)(x)
161graph = backend.graphs[0]
162```
163
164**normalize_gm + assertExpectedInline** — snapshot test graph output:
165```python
166from torch._dynamo.testing import normalize_gm
167self.assertExpectedInline(
168 normalize_gm(backend.graphs[0].print_readable(False)),
169 """\
170expected output here
171""",
172)
173```
174
175Call `torch._dynamo.reset()` within a test when testing multiple compilation
176scenarios in a single test method. The base class handles setUp/tearDown reset
177automatically.
178
179## Debugging
180
181### TORCH_LOGS
182
183```bash
184TORCH_LOGS="graph_breaks" python script.py # see graph breaks
185TORCH_LOGS="guards,recompiles" python script.py # see guards and recompilation reasons
186TORCH_LOGS="graph_code" python script.py # see captured FX graph code
187TORCH_LOGS="+dynamo" python script.py # full debug logging
188TORCH_LOGS="bytecode" python script.py # see bytecode transformations
189```
190
191### Reproducing crashes
192
193When writing a minimal repro for a Dynamo crash:
194
1951. **Capture `TORCH_LOGS="+dynamo"` for the known-failing case.** Look for
196 the frame ID (e.g. `[3/0_1]`), INLINING/FAILED/COMPILING/Restart events,
197 speculation behavior, and graph break reasons.
1982. **Capture the same logs for your repro attempt** and diff against the
199 failing case. The divergence point tells you what condition you're missing.
2003. **Match each condition from the crash traceback:**
201 - Does the code need a speculation checkpoint? Add tensor ops before the
202 failing code so `compile_subgraph` creates one.
203 - Does the function need to be inlined on retry (not skipped)? The graph
204 break type matters — `step_unsupported` keeps inlining on retry while
205 `unimplemented` may skip the function.
206 - Does the crash happen in a resume function? Add a `graph_break()` earlier
207 so PEP 523 compiles the resume as a fresh frame.
208
209### Structured tracing (for production)
210
211```bash
212TORCH_TRACE=/path/to/dir python script.py # explicit trace directory
213```
214
215Analyze with `tlparse`.
216
217### Compile-time profiling
218
219`TORCH_COMPILE_DYNAMO_PROFILER=1` prints per-function cumtime/tottime
220(cProfile-style) showing where Dynamo spends time during tracing. Set to a
221file path instead to save a profile loadable by `snakeviz`.
222
223### comptime.breakpoint() (`comptime.py`)
224
225Drops into pdb during **compilation** to inspect Dynamo state. Call
226`comptime.breakpoint()` in user code; in the pdb session use `ctx`
227(`ComptimeContext`) to call `print_locals()`, `print_bt()`, `print_graph()`,
228or `get_local("x").as_fake()`.
229
230### Bytecode Debugger (`bytecode_debugger.py`)
231
232pdb-like debugger for stepping through Dynamo-generated bytecode. Useful for
233debugging segfaults (no Python traceback) and codegen errors.
234
235```python
236with torch._dynamo.bytecode_debugger.debug():
237 my_compiled_fn(x)
238```
239
240**Programmatic breakpoints** (no graph break): call
241`torch._dynamo.bytecode_debugger.breakpoint()` in user code, or
242`codegen.extend_output(create_breakpoint())` in codegen. Auto-activates
243without an explicit `debug()` wrapper.
244
245**Segfault debugging**: `v` (verbose) then `c` (continue) — every instruction
246is printed with `flush=True` before execution, so the last line before a crash
247is the culprit. On exceptions, the debugger stops at the faulting instruction
248automatically.
249
250## C++ Runtime (`torch/csrc/dynamo/`)
251
252The C/C++ layer implements the PEP 523 frame evaluation hook, the cache, and
253the guard evaluation tree. Performance-critical runtime on every Python frame.
254
255### Frame Evaluation
256
257**`eval_frame.c`** — Installs a custom frame evaluation function via
258`_PyInterpreterState_SetEvalFrameFunc`. A thread-local callback controls
259behavior: `None` (disabled), `Py_False` (run-only / cache lookup), or a
260callable (full Dynamo).
261
262**`eval_frame_cpp.cpp`** — `dynamo__custom_eval_frame` is called for every
263frame: gets `ExtraState` from the code object, builds a `FrameLocalsMapping`
264(O(1) access to locals without dict materialization), evaluates guards via
265`run_root_guard_manager()` across all `CacheEntry`s (LRU ordered). On cache
266hit, executes compiled code via a shadow frame (`dynamo_eval_custom_code_impl`
267copies `localsplus` into a new frame with the compiled code object). On miss,
268calls the Python callback to trigger compilation.
269
270### Cache
271
272**`extra_state.cpp/.h`** — `ExtraState` is attached per code object via
273`_PyCode_SetExtra`. Contains a `cache_entry_list` (LRU linked list),
274`frame_state` (dynamic shapes detection), and `FrameExecStrategy`.
275
276**`cache_entry.cpp/.h`** — Each `CacheEntry` stores a `RootGuardManager*`
277(raw C++ pointer for fast guard eval), the compiled code object, and the
278backend.
279
280### Guard Evaluation Tree (`guards.cpp`)
281
282Guards are organized as a C++ tree (~7800 lines) mirroring the data access
283pattern. `RootGuardManager` is the root, receiving a `FrameLocalsMapping`.
284Each `GuardManager` node has leaf guards and child accessors.
285
286**LeafGuard** subclasses: `TYPE_MATCH` (Py_TYPE pointer comparison),
287`ID_MATCH`, `EQUALS_MATCH`, `TENSOR_MATCH` (dtype/device/shape/strides/dispatch
288keys in C++), `DICT_VERSION`, `GLOBAL_STATE` (grad mode, autocast, etc.).
289
290**GuardAccessor** subclasses define tree edges: `FrameLocalsGuardAccessor`
291(O(1) index), `GetAttrGuardAccessor`, `DictGetItemGuardAccessor`,
292`GlobalsGuardAccessor`, etc.
293
294Key optimizations: fail-fast accessor reordering, dict version tag matching to
295skip subtrees, `FrameLocalsMapping` avoids dict construction,
296`check_nopybind()` avoids pybind11 overhead.
297
298### Other C++ files
299
300- `framelocals_mapping.cpp` — O(1) frame locals/cells/freevars access
301- `cpython_defs.c` — copied CPython internals for frame manipulation
302- `init.cpp` — `torch._C._dynamo` module and pybind11 bindings
303- `debug_macros.h` — `DEBUG_TRACE`, `NULL_CHECK`, `INSPECT(...)` (drops into
304 pdb from C)
305- `compiled_autograd.cpp/.h` — compiled autograd engine
306
pytorch/pytorch · .github/copilot-instructions.md
@@ +1 @@
1# PyTorch Copilot Instructions
2
3This is the PyTorch machine learning framework codebase. These instructions help AI agents navigate and contribute effectively.
4
5## Architecture Overview
6
7### Core Components
8
9- **c10/** - Core library (C++-10 compatible) for essential, binary-size-conscious functionality
10- **aten/** - ATen tensor library (C++), PyTorch's foundation without autograd
11 - `aten/src/ATen/native/` - Modern operator implementations (CPU/CUDA/MPS/sparse)
12 - `aten/src/ATen/native/native_functions.yaml` - **Critical**: Declarative operator registry
13- **torch/** - Python bindings and public API
14 - `torch/csrc/` - C++ Python bindings (hand-written and generated)
15 - `torch/csrc/autograd/` - Reverse-mode automatic differentiation
16 - `torch/csrc/jit/` - TorchScript JIT compiler
17- **torchgen/** - Code generation tooling that reads `native_functions.yaml`
18- **tools/** - Build scripts, autograd derivatives, code generation
19
20### The Code Generation Workflow
21
22**Most operator changes require editing `native_functions.yaml`**, not direct C++ files. This YAML file:
231. Declares operator signatures, variants (function/method), and dispatch behavior
242. Gets processed by `torchgen/` to generate C++/Python bindings
253. Produces headers in `build/aten/src/ATen/` during compilation
26
27Example entry structure:
28```yaml
29- func: my_op(Tensor self, Scalar alpha=1) -> Tensor
30 variants: function, method
31 dispatch:
32 CPU: my_op_cpu
33 CUDA: my_op_cuda
34```
35
36After editing `native_functions.yaml`, implement kernels in `aten/src/ATen/native/` (see `aten/src/ATen/native/README.md`).
37
38## Development Workflows
39
40### Building from Source
41
42**Never run `setup.py` directly** - use pip with editable install:
43```bash
44python -m pip install --no-build-isolation -v -e .
45```
46
47Speed up builds:
48- `DEBUG=1` - Debug symbols with `-g -O0`
49- `USE_CUDA=0` - Skip CUDA compilation
50- `BUILD_TEST=0` - Skip C++ test binaries
51- Install `ninja` (`pip install ninja`) for faster builds
52- Use `ccache` for incremental compilation caching
53
54Rebuild specific targets: `(cd build && ninja <target>)`
55
56### Testing
57
58**Critical**: DO NOT run entire test suites. Run specific tests only:
59```bash
60python test/test_torch.py TestTorch.test_specific_case
61```
62
63**Test structure**: All tests use `torch.testing._internal.common_utils`:
64```python
65from torch.testing._internal.common_utils import run_tests, TestCase
66
67class TestFeature(TestCase):
68 def test_something(self):
69 # Use self.assertEqual for tensor comparisons
70 pass
71
72if __name__ == "__main__":
73 run_tests()
74```
75
76**For bug fixes**: Create a standalone reproduction script first, verify it fails, then fix and add to appropriate test file.
77
78### Linting
79
80Run linter (not pre-commit): `lintrunner -a` (auto-applies fixes)
81
82## Project-Specific Conventions
83
84### Memory and Storage
85- **Storage is never nullptr** (but `StorageImpl.data` may be nullptr for unallocated outputs)
86- CUDA device info lives in storage objects
87
88### Python-C++ Integration (`torch/csrc/`)
89- Always include `Python.h` **first** to avoid `_XOPEN_SOURCE` redefinition errors
90- Use `pybind11::gil_scoped_acquire` before calling Python API or using `THPObjectPtr`
91- Wrap entry points with `HANDLE_TH_ERRORS` / `END_HANDLE_TH_ERRORS` for exception conversion
92
93### Dispatch System
94- PyTorch uses operator dispatch to route calls to backend-specific kernels
95- Prefer `CompositeExplicitAutograd` dispatch when writing device-agnostic compound ops
96- See `aten/src/ATen/native/README.md` for dispatch keyword guidance
97
98## Git Workflow (AI Agent Specific)
99
100When preparing PRs from this environment:
101```bash
102git stash -u
103git reset --hard $(cat /tmp/orig_work.txt) # Reset to LOCAL branch
104git stash pop
105# Resolve conflicts if necessary
106```
107
108## Common Gotchas
109
1101. **Editing generated files** - If it's in `build/`, don't edit it. Edit the source template or `native_functions.yaml`
1112. **NVCC template compilation** - NVCC is stricter about C++ than gcc/clang; code working on Linux may fail Windows CI
1123. **Windows symbol visibility** - Use `TORCH_API` macros for exported symbols (required on Windows, optional on Linux)
1134. **No internet access** - DO NOT attempt to install dependencies during development
114
115## Key Files Reference
116
117- `AGENTS.md` - Instructions specific to AI coding agents
118- `CONTRIBUTING.md` - Comprehensive human contributor guide
119- `GLOSSARY.md` - Terminology (ATen, kernels, operations, JIT, TorchScript)
120- `aten/src/ATen/native/README.md` - Operator implementation guide
121- `tools/autograd/derivatives.yaml` - Gradient definitions for autograd
122
123## Performance Debugging
124
125Use `TORCH_SHOW_CPP_STACKTRACES=1` for C++ traces in Python errors. For profiling, prefer `py-spy` over manual instrumentation.
126
@@ −1 +1 @@
1−# torch/_dynamo
1+# PyTorch Copilot Instructions
22
3−TorchDynamo is a Python-level JIT compiler that captures PyTorch programs into
4−FX graphs by symbolically executing Python bytecode. It hooks into CPython's
5−PEP 523 frame evaluation API to intercept execution, traces operations into an
6−FX graph, compiles the graph with a backend (e.g. Inductor), and generates new
7−bytecode that calls the compiled code.
3+This is the PyTorch machine learning framework codebase. These instructions help AI agents navigate and contribute effectively.
84
95 ## Architecture Overview
106
11−The compilation pipeline, in execution order:
7+### Core Components
128
13−1. **`eval_frame.py`** — Runtime entry point. `torch.compile()` wraps a
14− function in an `OptimizedModule`. At runtime, the C extension
15− (`torch._C._dynamo.eval_frame`) intercepts Python frames via PEP 523.
16−2. **`convert_frame.py`** — `ConvertFrameAssert.__call__` checks caches,
17− handles recompilation limits, calls `_compile()` → `trace_frame()`.
18−3. **`symbolic_convert.py`** — The heart of Dynamo. `InstructionTranslator`
19− symbolically executes bytecode instruction-by-instruction. Maintains a
20− symbolic `stack` (list of `VariableTracker`s) and `symbolic_locals` (dict of
21− name → `VariableTracker`). Opcodes are dispatched via a `dispatch_table`
22− built by `BytecodeDispatchTableMeta`.
23−4. **`output_graph.py`** — `OutputGraph` owns the FX graph being built (via
24− `SubgraphTracer`), the `SideEffects` tracker, guards, shape environment,
25− and graph args. `compile_subgraph()` finalizes the graph, calls the backend,
26− and generates output bytecode.
27−5. **`codegen.py`** — `PyCodegen` emits output bytecode: loads graph inputs
28− (via `Source.reconstruct()`), calls the compiled graph, unpacks outputs, and
29− replays side effects.
30−6. **`resume_execution.py`** — Generates continuation functions for execution
31− after graph breaks.
9+- **c10/** - Core library (C++-10 compatible) for essential, binary-size-conscious functionality
10+- **aten/** - ATen tensor library (C++), PyTorch's foundation without autograd
11+ - `aten/src/ATen/native/` - Modern operator implementations (CPU/CUDA/MPS/sparse)
12+ - `aten/src/ATen/native/native_functions.yaml` - **Critical**: Declarative operator registry
13+- **torch/** - Python bindings and public API
14+ - `torch/csrc/` - C++ Python bindings (hand-written and generated)
15+ - `torch/csrc/autograd/` - Reverse-mode automatic differentiation
16+ - `torch/csrc/jit/` - TorchScript JIT compiler
17+- **torchgen/** - Code generation tooling that reads `native_functions.yaml`
18+- **tools/** - Build scripts, autograd derivatives, code generation
3219
33−## Key Abstractions
20+### The Code Generation Workflow
3421
35−### VariableTracker (`variables/`)
22+**Most operator changes require editing `native_functions.yaml`**, not direct C++ files. This YAML file:
23+1. Declares operator signatures, variants (function/method), and dispatch behavior
24+2. Gets processed by `torchgen/` to generate C++/Python bindings
25+3. Produces headers in `build/aten/src/ATen/` during compilation
3626
37−Every Python value encountered during tracing is wrapped in a `VariableTracker`
38−subclass. Key interface: `as_python_constant()`, `as_proxy()`,
39−`call_function()`, `call_method()`, `getattro_impl()`, `reconstruct()`.
40−
41−Key fields: `source` (where the value came from, for guards) and
42−`mutation_type` (whether/how mutations are tracked).
43−
44−**Factory**: `VariableTracker.build(tx, value, source=...)` dispatches to
45−`VariableBuilder` (sourced values needing guards) or `SourcelessBuilder`
46−(values created during tracing).
47−
48−Key subclass families in `variables/`: `TensorVariable` / `SymNodeVariable`
49−(tensor.py), `ConstantVariable` (constant.py), `ListVariable` /
50−`TupleVariable` (lists.py), `ConstDictVariable` (dicts.py), `SetVariable` (sets.py),
51−`UserFunctionVariable` (functions.py), `BuiltinVariable` (builtin.py),
52−`NNModuleVariable` (nn_module.py), `UserDefinedObjectVariable`
53−(user_defined.py), `TorchHigherOrderOperatorVariable` (higher_order_ops.py),
54−`LazyVariableTracker` (lazy.py). `VariableBuilder` and `SourcelessBuilder` are
55−in builder.py.
56−
57−### Source (`source.py`)
58−
59−Tracks value provenance — how to access a value at runtime. Used for guard
60−generation (`source.make_guard(GuardBuilder.XXX)`) and bytecode reconstruction
61−(`source.reconstruct(codegen)`). Root sources: `LocalSource`, `GlobalSource`.
62−Chained sources: `AttrSource`, `GetItemSource`, `NNModuleSource`, etc.
63−
64−### Guards (`guards.py`)
65−
66−Runtime conditions that must hold for cached compiled code to be reused.
67−Install via `install_guard(source.make_guard(GuardBuilder.TYPE_MATCH))`.
68−Common types: `TYPE_MATCH`, `ID_MATCH`, `EQUALS_MATCH`, `TENSOR_MATCH`,
69−`SEQUENCE_LENGTH`. At finalization, `CheckFunctionManager` builds a tree of
70−C++ `GuardManager` objects for fast runtime checking.
71−
72−### Side Effects (`side_effects.py`)
73−
74−Tracks mutations during tracing (attribute stores, list mutations, cell
75−variable updates, tensor hooks) and replays them as bytecode after graph
76−execution. The `MutationType` system (`variables/base.py`) controls what
77−mutations are allowed: `ValueMutationNew/Existing`,
78−`AttributeMutationNew/Existing`, or `None` (immutable). The `scope` field
79−prevents cross-scope mutations inside higher-order operators.
80−
81−### Other key files
82−
83−- `trace_rules.py` — inline/skip/graph-break decisions per function
84−- `exc.py` — exception hierarchy: `Unsupported` (graph break), `RestartAnalysis`
85− (restart tracing), `ObservedException` (user exceptions during tracing),
86− `BackendCompilerFailed`
87−- `config.py` — configuration flags, supports `config.patch()` context
88− manager/decorator
89−- `bytecode_transformation.py` / `bytecode_analysis.py` — low-level bytecode
90− manipulation, liveness analysis
91−- `pgo.py` — profile-guided optimization for dynamic shapes
92−- `polyfills/` — traceable replacements for stdlib functions
93−- `repro/` — reproduction/minification tools
94−
95−## Graph Breaks
96−
97−Call `unimplemented()` (from `exc.py`) to trigger a graph break:
98−
99−```python
100−from torch._dynamo.exc import unimplemented
101−from torch._dynamo import graph_break_hints
102−
103−unimplemented(
104− gb_type="short_category_name",
105− context=f"dynamic details: {value}",
106− explanation="Human-readable explanation of why this breaks the graph.",
107− hints=[*graph_break_hints.SUPPORTABLE],
108−)
27+Example entry structure:
28+```yaml
29+- func: my_op(Tensor self, Scalar alpha=1) -> Tensor
30+ variants: function, method
31+ dispatch:
32+ CPU: my_op_cpu
33+ CUDA: my_op_cuda
10934 ```
11035
111−- `gb_type`: Context-free category (no dynamic strings).
112−- `context`: Developer-facing details (can be dynamic).
113−- `explanation`: User-facing explanation (can be dynamic).
114−- `hints`: Use constants from `graph_break_hints.py`: `SUPPORTABLE`,
115− `FUNDAMENTAL`, `DIFFICULT`, `DYNAMO_BUG`, `USER_ERROR`,
116− `CAUSED_BY_EARLIER_GRAPH_BREAK`.
36+After editing `native_functions.yaml`, implement kernels in `aten/src/ATen/native/` (see `aten/src/ATen/native/README.md`).
11737
118−The `break_graph_if_unsupported` decorator on instruction handlers catches
119−`Unsupported`, logs the graph break, updates the `SpeculationLog`, and restarts
120−analysis. On the second pass, the partial graph is compiled at the break point
121−and a resume function handles the rest.
38+## Development Workflows
12239
123−## Testing
40+### Building from Source
12441
125−Tests live in `test/dynamo/`. Use `torch._dynamo.test_case.TestCase` as base
126−class — it calls `torch._dynamo.reset()` in setUp/tearDown and patches config
127−for strict error checking.
128−
42+**Never run `setup.py` directly** - use pip with editable install:
12943 ```bash
130−python test/dynamo/test_misc.py # whole file
131−python test/dynamo/test_misc.py MiscTests.test_foo # single test
132−python test/dynamo/test_misc.py -k test_foo # pattern match
44+python -m pip install --no-build-isolation -v -e .
13345 ```
13446
135−### Common patterns
47+Speed up builds:
48+- `DEBUG=1` - Debug symbols with `-g -O0`
49+- `USE_CUDA=0` - Skip CUDA compilation
50+- `BUILD_TEST=0` - Skip C++ test binaries
51+- Install `ninja` (`pip install ninja`) for faster builds
52+- Use `ccache` for incremental compilation caching
13653
137−The default backend to `torch.compile()` is `backend="eager"`.
54+Rebuild specific targets: `(cd build && ninja <target>)`
13855
139−**CompileCounter** — count compilations and graph ops:
140−```python
141−cnt = torch._dynamo.testing.CompileCounter()
56+### Testing
14257
143−@torch.compile(backend=cnt)
144−def fn(x):
145− return x + 1
146−
147−fn(torch.randn(10))
148−self.assertEqual(cnt.frame_count, 1)
149−self.assertEqual(cnt.op_count, 1)
58+**Critical**: DO NOT run entire test suites. Run specific tests only:
59+```bash
60+python test/test_torch.py TestTorch.test_specific_case
15061 ```
15162
152−**fullgraph=True** — assert no graph breaks:
63+**Test structure**: All tests use `torch.testing._internal.common_utils`:
15364 ```python
154−torch.compile(fn, backend="eager", fullgraph=True)(x)
155−```
65+from torch.testing._internal.common_utils import run_tests, TestCase
15666
157−**EagerAndRecordGraphs** — inspect captured FX graphs:
158−```python
159−backend = torch._dynamo.testing.EagerAndRecordGraphs()
160−torch.compile(fn, backend=backend)(x)
161−graph = backend.graphs[0]
162−```
67+class TestFeature(TestCase):
68+ def test_something(self):
69+ # Use self.assertEqual for tensor comparisons
70+ pass
16371
164−**normalize_gm + assertExpectedInline** — snapshot test graph output:
165−```python
166−from torch._dynamo.testing import normalize_gm
167−self.assertExpectedInline(
168− normalize_gm(backend.graphs[0].print_readable(False)),
169− """\
170−expected output here
171−""",
172−)
72+if __name__ == "__main__":
73+ run_tests()
17374 ```
17475
175−Call `torch._dynamo.reset()` within a test when testing multiple compilation
176−scenarios in a single test method. The base class handles setUp/tearDown reset
177−automatically.
76+**For bug fixes**: Create a standalone reproduction script first, verify it fails, then fix and add to appropriate test file.
17877
179−## Debugging
78+### Linting
18079
181−### TORCH_LOGS
80+Run linter (not pre-commit): `lintrunner -a` (auto-applies fixes)
18281
183−```bash
184−TORCH_LOGS="graph_breaks" python script.py # see graph breaks
185−TORCH_LOGS="guards,recompiles" python script.py # see guards and recompilation reasons
186−TORCH_LOGS="graph_code" python script.py # see captured FX graph code
187−TORCH_LOGS="+dynamo" python script.py # full debug logging
188−TORCH_LOGS="bytecode" python script.py # see bytecode transformations
189−```
82+## Project-Specific Conventions
19083
191−### Reproducing crashes
84+### Memory and Storage
85+- **Storage is never nullptr** (but `StorageImpl.data` may be nullptr for unallocated outputs)
86+- CUDA device info lives in storage objects
19287
193−When writing a minimal repro for a Dynamo crash:
88+### Python-C++ Integration (`torch/csrc/`)
89+- Always include `Python.h` **first** to avoid `_XOPEN_SOURCE` redefinition errors
90+- Use `pybind11::gil_scoped_acquire` before calling Python API or using `THPObjectPtr`
91+- Wrap entry points with `HANDLE_TH_ERRORS` / `END_HANDLE_TH_ERRORS` for exception conversion
19492
195−1. **Capture `TORCH_LOGS="+dynamo"` for the known-failing case.** Look for
196− the frame ID (e.g. `[3/0_1]`), INLINING/FAILED/COMPILING/Restart events,
197− speculation behavior, and graph break reasons.
198−2. **Capture the same logs for your repro attempt** and diff against the
199− failing case. The divergence point tells you what condition you're missing.
200−3. **Match each condition from the crash traceback:**
201− - Does the code need a speculation checkpoint? Add tensor ops before the
202− failing code so `compile_subgraph` creates one.
203− - Does the function need to be inlined on retry (not skipped)? The graph
204− break type matters — `step_unsupported` keeps inlining on retry while
205− `unimplemented` may skip the function.
206− - Does the crash happen in a resume function? Add a `graph_break()` earlier
207− so PEP 523 compiles the resume as a fresh frame.
93+### Dispatch System
94+- PyTorch uses operator dispatch to route calls to backend-specific kernels
95+- Prefer `CompositeExplicitAutograd` dispatch when writing device-agnostic compound ops
96+- See `aten/src/ATen/native/README.md` for dispatch keyword guidance
20897
209−### Structured tracing (for production)
98+## Git Workflow (AI Agent Specific)
21099
100+When preparing PRs from this environment:
211101 ```bash
212−TORCH_TRACE=/path/to/dir python script.py # explicit trace directory
102+git stash -u
103+git reset --hard $(cat /tmp/orig_work.txt) # Reset to LOCAL branch
104+git stash pop
105+# Resolve conflicts if necessary
213106 ```
214107
215−Analyze with `tlparse`.
108+## Common Gotchas
216109
217−### Compile-time profiling
110+1. **Editing generated files** - If it's in `build/`, don't edit it. Edit the source template or `native_functions.yaml`
111+2. **NVCC template compilation** - NVCC is stricter about C++ than gcc/clang; code working on Linux may fail Windows CI
112+3. **Windows symbol visibility** - Use `TORCH_API` macros for exported symbols (required on Windows, optional on Linux)
113+4. **No internet access** - DO NOT attempt to install dependencies during development
218114
219−`TORCH_COMPILE_DYNAMO_PROFILER=1` prints per-function cumtime/tottime
220−(cProfile-style) showing where Dynamo spends time during tracing. Set to a
221−file path instead to save a profile loadable by `snakeviz`.
115+## Key Files Reference
222116
223−### comptime.breakpoint() (`comptime.py`)
117+- `AGENTS.md` - Instructions specific to AI coding agents
118+- `CONTRIBUTING.md` - Comprehensive human contributor guide
119+- `GLOSSARY.md` - Terminology (ATen, kernels, operations, JIT, TorchScript)
120+- `aten/src/ATen/native/README.md` - Operator implementation guide
121+- `tools/autograd/derivatives.yaml` - Gradient definitions for autograd
224122
225−Drops into pdb during **compilation** to inspect Dynamo state. Call
226−`comptime.breakpoint()` in user code; in the pdb session use `ctx`
227−(`ComptimeContext`) to call `print_locals()`, `print_bt()`, `print_graph()`,
228−or `get_local("x").as_fake()`.
123+## Performance Debugging
229124
230−### Bytecode Debugger (`bytecode_debugger.py`)
231−
232−pdb-like debugger for stepping through Dynamo-generated bytecode. Useful for
233−debugging segfaults (no Python traceback) and codegen errors.
234−
235−```python
236−with torch._dynamo.bytecode_debugger.debug():
237− my_compiled_fn(x)
238−```
239−
240−**Programmatic breakpoints** (no graph break): call
241−`torch._dynamo.bytecode_debugger.breakpoint()` in user code, or
242−`codegen.extend_output(create_breakpoint())` in codegen. Auto-activates
243−without an explicit `debug()` wrapper.
244−
245−**Segfault debugging**: `v` (verbose) then `c` (continue) — every instruction
246−is printed with `flush=True` before execution, so the last line before a crash
247−is the culprit. On exceptions, the debugger stops at the faulting instruction
248−automatically.
249−
250−## C++ Runtime (`torch/csrc/dynamo/`)
251−
252−The C/C++ layer implements the PEP 523 frame evaluation hook, the cache, and
253−the guard evaluation tree. Performance-critical runtime on every Python frame.
254−
255−### Frame Evaluation
256−
257−**`eval_frame.c`** — Installs a custom frame evaluation function via
258−`_PyInterpreterState_SetEvalFrameFunc`. A thread-local callback controls
259−behavior: `None` (disabled), `Py_False` (run-only / cache lookup), or a
260−callable (full Dynamo).
261−
262−**`eval_frame_cpp.cpp`** — `dynamo__custom_eval_frame` is called for every
263−frame: gets `ExtraState` from the code object, builds a `FrameLocalsMapping`
264−(O(1) access to locals without dict materialization), evaluates guards via
265−`run_root_guard_manager()` across all `CacheEntry`s (LRU ordered). On cache
266−hit, executes compiled code via a shadow frame (`dynamo_eval_custom_code_impl`
267−copies `localsplus` into a new frame with the compiled code object). On miss,
268−calls the Python callback to trigger compilation.
269−
270−### Cache
271−
272−**`extra_state.cpp/.h`** — `ExtraState` is attached per code object via
273−`_PyCode_SetExtra`. Contains a `cache_entry_list` (LRU linked list),
274−`frame_state` (dynamic shapes detection), and `FrameExecStrategy`.
275−
276−**`cache_entry.cpp/.h`** — Each `CacheEntry` stores a `RootGuardManager*`
277−(raw C++ pointer for fast guard eval), the compiled code object, and the
278−backend.
279−
280−### Guard Evaluation Tree (`guards.cpp`)
281−
282−Guards are organized as a C++ tree (~7800 lines) mirroring the data access
283−pattern. `RootGuardManager` is the root, receiving a `FrameLocalsMapping`.
284−Each `GuardManager` node has leaf guards and child accessors.
285−
286−**LeafGuard** subclasses: `TYPE_MATCH` (Py_TYPE pointer comparison),
287−`ID_MATCH`, `EQUALS_MATCH`, `TENSOR_MATCH` (dtype/device/shape/strides/dispatch
288−keys in C++), `DICT_VERSION`, `GLOBAL_STATE` (grad mode, autocast, etc.).
289−
290−**GuardAccessor** subclasses define tree edges: `FrameLocalsGuardAccessor`
291−(O(1) index), `GetAttrGuardAccessor`, `DictGetItemGuardAccessor`,
292−`GlobalsGuardAccessor`, etc.
293−
294−Key optimizations: fail-fast accessor reordering, dict version tag matching to
295−skip subtrees, `FrameLocalsMapping` avoids dict construction,
296−`check_nopybind()` avoids pybind11 overhead.
297−
298−### Other C++ files
299−
300−- `framelocals_mapping.cpp` — O(1) frame locals/cells/freevars access
301−- `cpython_defs.c` — copied CPython internals for frame manipulation
302−- `init.cpp` — `torch._C._dynamo` module and pybind11 bindings
303−- `debug_macros.h` — `DEBUG_TRACE`, `NULL_CHECK`, `INSPECT(...)` (drops into
304− pdb from C)
305−- `compiled_autograd.cpp/.h` — compiled autograd engine
125+Use `TORCH_SHOW_CPP_STACKTRACES=1` for C++ traces in Python errors. For profiling, prefer `py-spy` over manual instrumentation.
306126
