Copilot instructions

1---
2description: Rails Background Jobs Guide (Solid Queue)
3applyTo: "app/jobs/**/*.rb"
4---
5 
6# Rails Background Jobs Guide (Solid Queue)
7 
8## Core Philosophy
9 
10- Use database-backed queuing for simplicity and reliability
11- Avoid external dependencies like Redis when possible
12- Design jobs to be idempotent and retryable
13- Keep background processing simple and focused
14- Leverage Rails' built-in Active Job framework
15 
16## Solid Queue Principles
17 
18- Database-backed job storage using FOR UPDATE SKIP LOCKED
19- No external dependencies required for basic operation
20- Built-in support for priorities, concurrency limits, and recurring jobs
21- Seamless integration with Active Job interface
22- Production-ready performance with PostgreSQL or MySQL
23 
24## Configuration Strategy
25 
26- Use separate database for queue in high-volume production environments
27- Configure workers based on job priorities and server resources
28- Set appropriate polling intervals for job urgency
29- Design thread counts based on database connection limits
30- Monitor queue depth and processing times
31 
32## Queue Organization
33 
34- Design queues based on priority and processing requirements
35- Use semantic queue names that indicate purpose
36- Configure dedicated workers for high-priority queues
37- Set appropriate concurrency limits per queue type
38- Use wildcards sparingly - prefer explicit queue names
39 
40## Job Design Principles
41 
42- Make all jobs idempotent - safe to run multiple times
43- Keep jobs small and focused on single tasks
44- Pass simple arguments - prefer IDs over objects
45- Design for eventual consistency
46- Handle failures gracefully with proper error reporting
47 
48## Error Handling Strategy
49 
50- Configure retry behavior based on job importance
51- Use exponential backoff for transient failures
52- Discard jobs that cannot succeed after retries
53- Log errors with sufficient context for debugging
54- Monitor failed job patterns for system issues
55 
56## Concurrency Controls
57 
58- Use `limits_concurrency` to prevent resource exhaustion
59- Define concurrency keys based on resource constraints
60- Set appropriate duration for concurrency locks
61- Group related jobs with concurrency controls
62- Monitor for concurrency bottlenecks
63 
64## Recurring Jobs
65 
66- Define recurring jobs in configuration, not code
67- Use cron-like syntax for scheduling
68- Keep recurring job logic simple and fast
69- Monitor recurring job execution patterns
70- Handle overlapping executions appropriately
71 
72## Performance Optimization
73 
74- Use exact queue names for better polling performance
75- Configure batch sizes based on job characteristics
76- Set polling intervals according to latency requirements
77- Monitor database load from job queries
78- Index job tables appropriately
79 
80## Database Configuration
81 
82- Use separate database for Solid Queue in production
83- Configure appropriate connection pool sizes
84- Set transaction isolation levels correctly
85- Monitor database performance metrics
86- Plan for job table maintenance
87 
88## Worker Management
89 
90- Use Puma plugin for integrated process management
91- Configure supervisor for automatic restarts
92- Set appropriate number of worker processes
93- Monitor worker health and performance
94- Handle graceful shutdowns properly
95 
96## Job Arguments
97 
98- Pass primitive types and simple data structures
99- Avoid passing full ActiveRecord objects
100- Use GlobalID for model references
101- Keep argument size reasonable
102- Version job arguments for backwards compatibility
103 
104## Testing Strategy
105 
106- Test job logic separately from queueing
107- Use perform_now in tests for synchronous execution
108- Test retry and failure scenarios
109- Verify idempotency with multiple executions
110- Mock external dependencies appropriately
111 
112## Monitoring Best Practices
113 
114- Track job queue depth and latency
115- Monitor job success/failure rates
116- Alert on queue backlogs
117- Track job execution times
118- Monitor worker resource usage
119 
120## Deployment Considerations
121 
122- Use Kamal for container-based deployments
123- Configure health checks for workers
124- Implement zero-downtime deployments
125- Handle long-running jobs during deploys
126- Monitor deployment impact on queues
127 
128## Migration Strategy
129 
130- Plan migration from other queue backends carefully
131- Run both systems in parallel during transition
132- Migrate jobs gradually by type
133- Monitor for issues during migration
134- Have rollback plan ready
135 
136## Debugging Techniques
137 
138- Use Rails console to inspect job state
139- Query job tables directly when needed
140- Add structured logging to jobs
141- Use APM tools for job performance
142- Track job execution paths
143 
144## Lifecycle Hooks
145 
146- Use Solid Queue callbacks for monitoring
147- Track job state transitions
148- Implement custom instrumentation
149- Monitor for stuck jobs
150- Alert on abnormal patterns
151 
152## Best Practices
153 
154- Keep jobs simple and focused
155- Design for failure and retry
156- Monitor everything important
157- Document job behavior clearly
158- Test edge cases thoroughly
159 
160## Common Patterns
161 
162- Email delivery jobs with rate limiting
163- Report generation with progress tracking
164- Data import/export with chunking
165- Webhook processing with retries
166- Scheduled maintenance tasks
167 
168## Anti-Patterns to Avoid
169 
170- Long-running jobs without heartbeats
171- Jobs that modify shared state unsafely
172- Recursive job spawning without limits
173- Jobs with external dependencies in transactions
174- Overly complex job orchestration
175 
176Remember: Solid Queue brings reliability and simplicity to background processing. Use it to keep your application responsive while handling work asynchronously.
177