Copilot instructions

1# GitHub Copilot Instructions for LLaMA Factory
2 
3## Project Overview
4 
5LLaMA Factory is an efficient fine-tuning framework for 100+ large language models (LLMs). It provides:
6- Support for various models: LLaMA, LLaVA, Mistral, Qwen, DeepSeek, Yi, Gemma, ChatGLM, Phi, etc.
7- Multiple training methods: pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO
8- Scalable resources: 16-bit full-tuning, freeze-tuning, LoRA and QLoRA variants
9- Advanced algorithms: GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, etc.
10- Web UI (LLaMA Board) and CLI interfaces
11 
12### Architecture Versions
13 
14LLaMA Factory has two parallel architectures that can be switched via the `USE_V1` environment variable:
15 
16**v0 (default)** - File hierarchy:
17- `api`, `webui` → `chat`, `eval`, `train` → `data`, `model` → `hparams` → `extras`
18 
19**v1** - File hierarchy:
20- `trainers` → `core` → `accelerator`, `plugins`, `config` → `utils`
21 
22Set `USE_V1=1` to enable v1 architecture.
23 
24## Code Structure
25 
26### v0 Architecture (Default)
27 
28- `src/llamafactory/` - Main package directory
29  - `api/` - OpenAI-style API implementation
30  - `chat/` - Chat interface implementation
31  - `cli.py` - Command-line interface
32  - `data/` - Data processing and dataset handling
33  - `eval/` - Model evaluation utilities
34  - `extras/` - Additional utilities and helpers
35  - `hparams/` - Hyperparameter definitions
36  - `model/` - Model loading, patching, and utilities
37  - `train/` - Training pipeline implementation
38  - `webui/` - Gradio-based web interface
39- `src/train.py` - Training entry script (delegates to `llamafactory.train.tuner`)
40- `src/webui.py` - Web UI entry script (delegates to `llamafactory.webui.interface`)
41- `src/api.py` - API server entry script (delegates to `llamafactory.api.app`)
42- `tests/` - Test suite
43- `examples/` - Example configurations for various training scenarios
44- `data/` - Dataset definitions and examples
45 
46### v1 Architecture (USE_V1=1)
47 
48- `src/llamafactory/v1/` - Version 1 package directory
49  - `trainers/` - Training implementations
50  - `core/` - Core training utilities
51  - `accelerator/` - Acceleration and distributed training
52  - `plugins/` - Pluggable components (model, data, sampler, trainer)
53  - `config/` - Configuration management
54  - `utils/` - Utility functions
55 
56## Development Practices
57 
58### Code Style
59 
60- Follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)
61- Use ruff for linting and formatting
62- Line length: 119 characters
63- Indentation: 4 spaces
64- Quote style: double quotes
65- Use Google-style docstrings for documentation
66 
67### Import Organization
68 
69- Known first-party: `llamafactory`
70- Known third-party: `accelerate`, `datasets`, `gradio`, `numpy`, `peft`, `torch`, `transformers`, `trl`
71- Use 2 blank lines after imports
72 
73### Quality Checks
74 
75Before committing code, run:
76```bash
77make style      # Auto-fix style issues
78make quality    # Check code quality
79make test       # Run test suite
80```
81 
82Or use the combined command:
83```bash
84make commit     # Run pre-commit hooks
85```
86 
87### Testing
88 
89- Use pytest for testing
90- Tests are located in `tests/` and `tests_v1/` directories
91- Run tests with: `make test` (which runs `WANDB_DISABLED=true pytest -vv --import-mode=importlib tests/ tests_v1/`)
92- Disable wandb during testing to avoid external dependencies
93- **Note**: Training configurations require GPU machines, so training is typically not tested end-to-end. Use `make test` to validate file-level functionality.
94 
95### Building
96 
97Build the package with:
98```bash
99pip3 install build && python3 -m build
100```
101 
102### License
103 
104- All source files must include the Apache 2.0 license header
105- Check license headers with: `make license`
106 
107## Common Patterns
108 
109### Configuration Files
110 
111- Training configurations are typically YAML or JSON files in `examples/` directory
112- Hyperparameters are defined using dataclasses in `src/llamafactory/hparams/`
113 
114### Model Support
115 
116- New model support is added through model patches in `src/llamafactory/model/`
117- Visual models use the visual utilities in `src/llamafactory/model/model_utils/visual.py`
118- Quantization support is in `src/llamafactory/model/model_utils/quantization.py`
119 
120### Data Processing
121 
122- Dataset definitions are in `data/dataset_info.json`
123- Data templates and processors are in `src/llamafactory/data/`
124 
125### Training
126 
127- Training pipelines are in `src/llamafactory/train/`
128- Support for different training methods: SFT, DPO, PPO, RM, PT, KTO, ORPO
129 
130## Key Dependencies
131 
132- Python >= 3.9.0
133- PyTorch and transformers for model handling
134- datasets for data processing
135- peft for parameter-efficient fine-tuning
136- accelerate for distributed training
137- gradio for web UI
138- trl for reinforcement learning
139- Optional: vllm/sglang for inference, flash-attention-2, unsloth, liger-kernel
140 
141## Entry Points
142 
143- **CLI Training**: `llamafactory-cli train --config examples/train_lora/llama3_lora_sft.yaml`
144- **Web UI**: `llamafactory-cli webui` or `python src/webui.py`
145- **API Server**: `llamafactory-cli api` or `python src/api.py`
146- **Chat Interface**: `llamafactory-cli chat --model_name_or_path MODEL_PATH`
147 
148## Environment Setup
149 
150For development:
151```bash
152pip install -e ".[dev]"
153```
154 
155## Important Notes
156 
157- The project supports multiple backends: default PyTorch, vLLM, SGLang
158- Megatron-core training is supported via mcore_adapter
159- SwanLab and W&B are supported for experiment tracking
160- Docker support is available with pre-built images
161- Day-0/Day-1 support for latest cutting-edge models
162- Multi-modal support for vision and audio understanding tasks
163 
164## Contribution Guidelines
165 
1661. Fork the repository
1672. Create a development branch
1683. Set up development environment with `pip install -e ".[dev]"`
1694. Make changes following the style guide
1705. Run quality checks: `make style && make quality`
1716. Run tests: `make test`
1727. Submit a pull request
173 
174## Common Commands
175 
176- `make style` - Format code
177- `make quality` - Run linters
178- `make test` - Run tests
179- `make commit` - Install and run pre-commit hooks
180- `make license` - Check license headers
181