Copilot instructions

1---
2applyTo: "**/mlops/**/*.py,**/sagemaker/**/*.py,**/pipelines/**/*.py,**/*.pipeline.py"
3---
4 
5# MLOps Pipeline Standards
6 
7## Model Lifecycle
8 
9- All models must pass through: Experiment → Registered → Approved → Deployed → Monitored → Deprecated
10- Never deploy a model that has not been registered and approved in the Model Registry
11- Record with every model version: training data version, evaluation metrics, training date, approver
12- Define model expiry policy — retrain or retire models older than the data freshness threshold
13 
14## Pipeline Design
15 
16- Use SageMaker Pipelines or equivalent for all production ML workflows — not ad-hoc scripts
17- Every pipeline must be reproducible: same inputs → same outputs (deterministic data processing and fixed random seeds)
18- Implement quality gates as pipeline steps — do not promote models that fail the quality gate
19- Version all pipeline definitions alongside the model code
20 
21## Feature Engineering
22 
23- Ensure training/serving feature parity — feature computation must be identical in both contexts
24- Store feature transformation logic in a versioned artifact alongside the model
25- Implement point-in-time correctness for all join operations in training data preparation
26- Validate feature schema before training and serving — schema drift is a silent failure mode
27 
28## Model Monitoring
29 
30- Implement data quality monitoring: detect input feature distribution shift
31- Implement model quality monitoring: track prediction accuracy against ground truth labels
32- Define alert thresholds for each monitored metric — alerts must route to the responsible team
33- Implement bias monitoring for all models with demographic inputs or user-facing decisions
34 
35## Retraining
36 
37- Define trigger conditions for automated retraining: drift threshold breach, scheduled cadence, new data volume
38- Automated retraining pipelines must still require human approval before production deployment
39- Test retraining pipeline on a schedule even when drift has not been detected — prevents pipeline rot
40- Implement champion/challenger deployment for new model versions — do not flip 100% traffic immediately
41 
42## Governance
43 
44- Produce a model card for every model version before production approval
45- Maintain audit log: who approved the model, on what data, with what metrics, when
46- Never delete training data for a model still in production — it is the audit evidence
47- Classify all models by EU AI Act risk tier; high-risk models require additional governance controls
48