Copilot instructions

1# Tesseract OCR - GitHub Copilot Instructions
2 
3## Repository Overview
4 
5Tesseract is an open-source **OCR (Optical Character Recognition) engine** that recognizes text from images. This repository contains:
6 
7- **libtesseract**: C++ OCR library with C API wrapper
8- **tesseract**: Command-line OCR program
9- **Training tools**: For creating custom language models
10 
11**Key Facts:**
12- Primary language: **C++17** (requires C++17-compliant compiler)
13- Size: Large (~100MB+ with submodules)
14- License: Apache 2.0
15- Maintained by: Stefan Weil (lead), Zdenko Podobny (maintainer)
16 
17## Build Systems
18 
19Tesseract supports **two build systems**. Both are actively maintained and tested in CI.
20 
21### 1. Autotools (Traditional, POSIX Systems)
22 
23**When to use:** Linux, macOS (command-line), MSYS2 on Windows
24 
25**Build sequence:**
26```bash
27./autogen.sh                    # Generate configure script (only needed after git clone)
28./configure                      # Configure build (creates Makefiles)
29make                            # Build library and CLI
30sudo make install               # Install to system
31sudo ldconfig                   # Update library cache (Linux only)
32make training                   # Build training tools (optional)
33sudo make training-install      # Install training tools
34```
35 
36**Important:**
37- ALWAYS run `./autogen.sh` first if building from git clone
38- Use `make -j N` for parallel builds (N = number of CPU cores)
39- Check `configure --help` for build options
40- To clean: `make clean` or `make distclean` (complete cleanup)
41 
42### 2. CMake (Modern, Cross-platform)
43 
44**When to use:** Windows (MSVC, MinGW), cross-platform, modern development
45 
46**Build sequence:**
47```bash
48mkdir build                     # MUST use out-of-source build
49cd build
50cmake ..                        # Configure (add options here)
51make                            # Or: cmake --build .
52sudo make install               # Install to system
53```
54 
55**Important CMake options:**
56- `BUILD_TRAINING_TOOLS=ON` - Enable training tools build
57- `CMAKE_BUILD_TYPE=Release` - Release build (default is RelWithDebInfo)
58- `GRAPHICS_DISABLED=ON` - Disable ScrollView (GUI debugger)
59- `ENABLE_NATIVE=OFF` - Disable CPU-specific optimizations (for portability)
60 
61**CMake enforces out-of-source builds** - you cannot build in the source directory. If you get an error about this, remove `CMakeCache.txt` and build in a separate directory.
62 
63## Dependencies
64 
65### Core Required Dependencies
66 
67- **Leptonica 1.74.2+** (REQUIRED) - Image I/O library
68  - Without this, build will fail
69  - Usually installed via package manager: `libleptonica-dev` (Ubuntu) or `leptonica` (Homebrew)
70 
71- **C++17 compiler:**
72  - GCC 7+, Clang 5+, MSVC 2017+
73  - Verified compilers: gcc-11, gcc-12, gcc-14, clang-15, clang++
74 
75### Training Tools Dependencies
76 
77Only needed if building training tools (`make training` or `-DBUILD_TRAINING_TOOLS=ON`):
78 
79- pango-devel / libpango1.0-dev
80- cairo-devel
81- icu-devel
82 
83### Optional Dependencies
84 
85- **libarchive-dev**, **libcurl4-openssl-dev** - For advanced features
86- **OpenMP** - For parallel processing (enabled by default if available)
87- **cabextract** - For testing with CAB archives
88 
89### Traineddata Files
90 
91Tesseract requires **traineddata files** to function. Minimum required:
92- `eng.traineddata` (English)
93- `osd.traineddata` (Orientation and Script Detection)
94 
95**Installation:**
96```bash
97# Download individual files (to /usr/local/share/tessdata/ or your TESSDATA_PREFIX path)
98cd /usr/local/share/tessdata/  # Or wherever you want to install
99wget https://github.com/tesseract-ocr/tessdata/raw/main/eng.traineddata
100wget https://github.com/tesseract-ocr/tessdata/raw/main/osd.traineddata
101 
102# Or clone all languages (WARNING: 1.2+ GB)
103git clone https://github.com/tesseract-ocr/tessdata.git
104```
105 
106**Set environment variable:**
107```bash
108export TESSDATA_PREFIX=/usr/local/share/tessdata/
109```
110 
111Verify with: `tesseract --list-langs`
112 
113## Testing
114 
115### Running Unit Tests
116 
117**With autotools:**
118```bash
119./autogen.sh
120./configure
121make
122make check                      # Runs all unit tests
123```
124 
125**With CMake:**
126```bash
127mkdir build && cd build
128cmake ..
129make
130ctest                          # Or: cmake --build . --target test
131```
132 
133**Important:**
134- Tests require `googletest` submodule: `git submodule update --init --recursive`
135- Tests require tessdata files (eng, osd minimum)
136- Test results in `test-suite.log` (autotools) or CTest output (CMake)
137 
138### Running Tesseract CLI
139 
140Basic test commands:
141```bash
142# After installation:
143tesseract --version
144tesseract --list-langs
145tesseract input.png output      # OCR image, creates output.txt
146tesseract input.png output pdf  # Create searchable PDF
147```
148 
149Test files available in `test/testing/` (requires test submodule):
150- `phototest.tif` - English test image
151- `devatest.png` - Hindi/Devanagari test image (different format intentional)
152 
153## Project Structure
154 
155### Source Code Layout
156 
157```
158src/
159├── api/               # Public C/C++ API (baseapi.h, capi.h)
160├── ccmain/            # Main OCR control logic
161├── lstm/              # LSTM neural network engine (Tesseract 4+)
162├── ccutil/, cutil/    # Core utilities, data structures
163├── classify/          # Character classifier
164├── dict/              # Dictionary and language model
165├── textord/           # Text line and word detection
166├── wordrec/           # Word recognition
167├── training/          # Training tools (lstmtraining, text2image, etc.)
168└── tesseract.cpp      # CLI main() entry point
169 
170include/tesseract/     # Public header files
171unittest/              # Unit tests (requires googletest)
172test/testing/          # Test images and data
173tessdata/              # Default location for traineddata files
174doc/                   # Documentation
175```
176 
177### Key Files
178 
179- **src/api/baseapi.h** - Main C++ API class (`TessBaseAPI`)
180- **src/api/capi.h** - C wrapper API
181- **src/tesseract.cpp** - Command-line tool
182- **CMakeLists.txt**, **configure.ac**, **Makefile.am** - Build configuration
183- **VERSION** - Current version string
184 
185### Configuration Files
186 
187- **.clang-format** - Code formatting rules (LLVM style)
188- **tesseract.pc.in** - pkg-config template
189- **.github/workflows/** - CI/CD definitions
190 
191## CI/CD Workflows
192 
193### Active Workflows
194 
1951. **cmake.yml** - CMake builds on Ubuntu/macOS, 6 configurations
1962. **autotools.yml** - Autotools builds, comprehensive testing
1973. **unittest.yml** - Unit tests with sanitizers (ASAN, UBSAN)
1984. **codeql-analysis.yml** - Security static analysis
1995. **vcpkg.yml**, **msys2.yml**, **cmake-win64.yml** - Windows builds
200 
201### Validation Requirements
202 
203All PRs trigger:
204- **Build tests** on multiple platforms (Ubuntu 22.04, 24.04, macOS 14, 15)
205- **Compiler tests** (GCC 11-14, Clang 15)
206- **Unit tests** with sanitizers
207- **CodeQL** security scan
208 
209**Expect ~10-30 minutes** for full CI validation.
210 
211### Common CI Failures
212 
213- **Missing dependencies:** Check workflow files for required packages
214- **Test failures:** Often due to missing tessdata files
215- **Sanitizer errors:** Memory leaks, undefined behavior
216- **CodeQL alerts:** Security vulnerabilities in code
217 
218## Common Build Issues & Workarounds
219 
220### Issue: "configure: error: Leptonica not found"
221**Solution:** Install leptonica development package
222```bash
223# Ubuntu/Debian:
224sudo apt-get install libleptonica-dev
225# macOS:
226brew install leptonica
227```
228 
229### Issue: "CMake Error: cannot build in source directory"
230**Solution:** CMake requires out-of-source builds
231```bash
232rm -f CMakeCache.txt
233mkdir build && cd build && cmake ..
234```
235 
236### Issue: "make check" fails with "cannot find tessdata"
237**Solution:** Set TESSDATA_PREFIX or download files
238```bash
239export TESSDATA_PREFIX=/usr/local/share/tessdata/
240# Or copy files to /usr/local/share/tessdata/
241```
242 
243### Issue: Submodule errors (googletest, test)
244**Solution:** Initialize submodules
245```bash
246git submodule update --init --recursive
247```
248 
249### Issue: Old Tesseract version conflicts
250**Solution:** Remove previous installation before building
251```bash
252# Find installed files:
253which tesseract
254pkg-config --modversion tesseract
255# Uninstall old version, then rebuild
256```
257 
258### Issue: Training tools not building
259**Solution:** Install pango, cairo, icu dependencies
260```bash
261sudo apt-get install libpango1.0-dev libcairo2-dev libicu-dev
262```
263 
264## Validation Steps for Code Changes
265 
266When making code changes, follow these steps:
267 
2681. **Build the project** (choose one):
269```bash
270   # Autotools:
271   ./autogen.sh && ./configure && make
272   # CMake:
273   mkdir build && cd build && cmake .. && make
274```
275 
2762. **Run unit tests**:
277```bash
278   # Autotools:
279   make check
280   # CMake:
281   ctest
282```
283 
2843. **Test CLI manually**:
285```bash
286   tesseract test/testing/phototest.tif output
287   cat output.txt  # Verify OCR output
288```
289 
2904. **Check for memory issues** (if modifying C++ code):
291```bash
292   # Build with sanitizers:
293   CXXFLAGS="-g -O2 -fsanitize=address,undefined" ./configure
294   make && make check
295```
296 
2975. **Run CodeQL** (security check):
298   - Will run automatically in CI
299   - Or use GitHub Code Scanning locally
300 
3016. **Verify documentation** (if API changes):
302   - Update header comments in `include/tesseract/`
303   - Update relevant docs in `doc/`
304 
305## Code Style & Conventions
306 
307- **Formatting:** Use clang-format with `.clang-format` config (LLVM style)
308- **Naming:** 
309  - Classes: `CamelCase` (e.g., `TessBaseAPI`)
310  - Functions: `CamelCase` (e.g., `ProcessPage`)
311  - Variables: `snake_case` or `lower_case`
312- **Headers:** Use include guards, document public APIs
313- **Comments:** Focus on "why", not "what"
314- **Commits:** Use meaningful messages, reference issue numbers
315 
316## Important Notes for AI Coding Agents
317 
3181. **Always use out-of-source builds with CMake** - in-source builds are blocked
3192. **Check for Leptonica** before building - it's a hard requirement
3203. **Initialize git submodules** before running tests
3214. **Set TESSDATA_PREFIX** or tests will fail
3225. **Building takes time** - allow 2-5 minutes for full build
3236. **Testing takes time** - `make check` can take 5-10 minutes
3247. **Don't remove existing tests** - they're critical for preventing regressions
3258. **Check CI workflows** for platform-specific requirements
3269. **Sanitizer builds are slower** - 2-3x slower than normal builds
32710. **Training tools are optional** - only build if needed for the task
328 
329## Useful Commands Reference
330 
331```bash
332# Quick build and test (autotools):
333./autogen.sh && ./configure && make -j8 && make check
334 
335# Quick build and test (CMake):
336mkdir build && cd build && cmake .. && make -j8 && ctest
337 
338# Format code:
339find src -name '*.cpp' -o -name '*.h' | xargs clang-format -i
340 
341# Check test results:
342cat test-suite.log                    # autotools
343ctest --output-on-failure             # CMake
344 
345# Install only library (no training):
346make install                          # After ./configure && make
347 
348# Clean builds:
349make clean                            # Partial clean
350make distclean                        # Complete clean (autotools)
351rm -rf build                          # Complete clean (CMake)
352 
353# Check installed version:
354tesseract --version
355pkg-config --modversion tesseract
356 
357# Debug OCR on specific image:
358tesseract input.png output -l eng --psm 6 -c debug_file=/dev/null
359```
360 
361---
362 
363**Trust these instructions.** Only search for additional information if these instructions are incomplete, outdated, or if you encounter an error not covered here. The workflows and build procedures are tested daily in CI and represent current best practices for this repository.
364