CLAUDE.md

1# CLAUDE.md
2 
3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4 
5## Commands
6 
7```bash
8# Code style (auto-fix)
9make style
10 
11# Code quality check (no modifications)
12make quality
13 
14# Run all tests
15make test
16 
17# Run a single test file
18WANDB_DISABLED=true pytest -vv --import-mode=importlib tests/path/to/test_file.py
19 
20# Run tests matching a pattern
21WANDB_DISABLED=true pytest -vv --import-mode=importlib tests/ -k "test_name"
22 
23# License header check
24make license
25 
26# Build package
27make build
28```
29 
30The project uses `uv` as the preferred package manager. Commands automatically use `uv run` / `uvx` if `uv` is available.
31 
32## Architecture
33 
34LlamaFactory has two parallel architectures controlled by the `USE_V1` environment variable:
35 
36- **v0 (default):** `api, webui > chat, eval, train > data, model > hparams > extras`
37- **v1 (experimental, `USE_V1=1`):** `trainers > core > accelerator, plugins, config > utils`
38 
39Most active development happens in v0. The v1 architecture lives in `src/llamafactory/v1/`.
40 
41### Entry Points
42 
43CLI entry point is `llamafactory-cli` / `lmf` → `src/llamafactory/cli.py:main()`, which dispatches to `launcher.py` based on `USE_V1`.
44 
45Available subcommands: `train`, `chat`, `api`, `export`, `webchat`, `webui`, `env`, `version`, `help`.
46 
47### Training Flow (v0)
48 
49```
50run_exp() [tuner.py]
51  → read_args() → parse YAML/JSON config
52  → get_train_args() → produces typed argument dataclasses
53  → routes to: run_sft / run_dpo / run_ppo / run_rm / run_pt / run_kto
54  → optional: export_model()
55```
56 
57Training is invoked with a YAML config: `llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml`
58 
59### Configuration System
60 
61All training parameters are YAML/JSON config files. Argument parsing in `src/llamafactory/hparams/parser.py` produces four typed dataclasses:
62- `ModelArguments` — model/tokenizer selection, quantization
63- `DataArguments` — datasets, templates, preprocessing
64- `FinetuningArguments` — LoRA rank/target, training method (sft/dpo/ppo/rm/pt/kto)
65- `TrainingArguments` — extends HuggingFace's `TrainingArguments`
66 
67### Key Modules
68 
69| Module | Purpose |
70|--------|---------|
71| `src/llamafactory/model/loader.py` | Loads model + tokenizer; applies quantization, LoRA, patches |
72| `src/llamafactory/model/patcher.py` | Model-specific compatibility patches |
73| `src/llamafactory/data/template.py` | Prompt templates; `TEMPLATES` dict maps model family → format |
74| `src/llamafactory/data/mm_plugin.py` | Multi-modal (image/video/audio) data handling |
75| `src/llamafactory/data/processor/` | Per-stage data processors (supervised, pairwise, pretrain, etc.) |
76| `src/llamafactory/train/sft/` | SFT trainer; other stages follow same structure |
77| `src/llamafactory/chat/` | Inference engines: `hf_engine`, `vllm_engine`, `sglang_engine`, `kt_engine` |
78| `src/llamafactory/extras/constants.py` | Enums and constants used across the project |
79 
80### Adding Support for a New Model
81 
821. Add a prompt template to `src/llamafactory/data/template.py` in the `TEMPLATES` dict
832. Add any necessary model patches in `src/llamafactory/model/patcher.py`
843. Add multi-modal support in `src/llamafactory/data/mm_plugin.py` if needed
85 
86### Distributed Training
87 
88Multi-GPU automatically uses `torchrun`. Additional backends:
89- **Ray:** Optional Ray cluster support
90- **HyperParallel FSDP2:** `src/llamafactory/train/hyper_parallel/`
91- **Megatron-core:** `src/llamafactory/train/mca/`
92 
93### Testing
94 
95- `tests/` — v0 tests; `tests_v1/` — v1 tests
96- Most training tests require GPU hardware
97- pytest markers: `@pytest.mark.slow`, `@pytest.mark.runs_on(['cuda'])`
98- Always set `WANDB_DISABLED=true` when running tests
99 
100### Code Style
101 
102- Ruff for linting and formatting (line length 119, Google-style docstrings)
103- Python 3.11+ syntax
104- Double quotes for strings
105- All new files must include Apache 2.0 license header (checked by `make license`)
106