- Added new fields to the mirror_jobs table for job resilience, including job_type, batch_id, total_items, completed_items, item_ids, completed_item_ids, in_progress, started_at, completed_at, and last_checkpoint.
- Implemented database migration scripts to update the mirror_jobs table schema.
- Introduced processWithResilience utility for handling item processing with checkpointing and recovery capabilities.
- Updated API routes for mirroring organizations and repositories to utilize the new resilience features.
- Created recovery system to detect and resume interrupted jobs on application startup.
- Added middleware to initialize the recovery system when the server starts.
- Updated `useSSE` hook to include max reconnect attempts and exponential backoff for reconnections.
- Improved error handling for SSE messages and connection errors.
- Added connection status reset on successful connection.
fix: improve SQLite database connection handling
- Simplified database initialization and connection logic.
- Ensured the database file is created if it doesn't exist.
fix: enhance Redis client connection with retry strategy
- Implemented exponential backoff for Redis connection retries.
- Added event handlers for connection success and error handling.
feat: improve SSE API endpoint with robust Redis connection management
- Added connection retry logic for Redis in the SSE API.
- Implemented heartbeat messages to keep the connection alive.
- Enhanced error handling for Redis subscription and connection attempts.
