Commit Graph

57 Commits

Author SHA1 Message Date
Arunavo Ray
c2f6e73054 Testing Authentik SSO Issues 2025-09-07 19:09:00 +05:30
Arunavo Ray
f54a7e6d71 update default configs 2025-08-28 13:45:49 +05:30
Arunavo Ray
d49599ff05 Org ignore 2025-08-28 13:27:10 +05:30
Arunavo Ray
d99f597988 Update the Ignore Repo 2025-08-28 12:58:58 +05:30
Arunavo Ray
b3856b4223 More tsc issues 2025-08-28 08:34:41 +05:30
Arunavo Ray
ad7418aef2 tsc issues 2025-08-28 08:34:27 +05:30
Arunavo Ray
38a0d1b494 repository cleanup functionality 2025-08-27 19:12:52 +05:30
Arunavo Ray
698eb0b507 fix: Complete Issue #72 - Fix automatic mirroring and repository cleanup
Major fixes for Docker environment variable issues and cleanup functionality:

🔧 **Duration Parser & Scheduler Fixes**
- Add comprehensive duration parser supporting "8h", "30m", "24h" formats
- Fix GITEA_MIRROR_INTERVAL environment variable mapping to scheduler
- Auto-enable scheduler when GITEA_MIRROR_INTERVAL is set
- Improve scheduler logging to clarify timing behavior (from last run, not startup)

🧹 **Repository Cleanup Service**
- Complete repository cleanup service for orphaned repos (unstarred, deleted)
- Fix cleanup configuration logic - now works with CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
- Auto-enable cleanup when deleteIfNotInGitHub is enabled
- Add manual cleanup trigger API endpoint (/api/cleanup/trigger)
- Support archive/delete actions with dry-run mode and protected repos

🐛 **Environment Variable Integration**
- Fix scheduler not recognizing GITEA_MIRROR_INTERVAL=8h
- Fix cleanup requiring both CLEANUP_DELETE_FROM_GITEA and CLEANUP_DELETE_IF_NOT_IN_GITHUB
- Auto-enable services when relevant environment variables are set
- Better error logging and debugging information

📚 **Documentation Updates**
- Update .env.example with auto-enabling behavior notes
- Update ENVIRONMENT_VARIABLES.md with clarified functionality
- Add comprehensive tests for duration parsing

This resolves the core issues where:
1. GITEA_MIRROR_INTERVAL=8h was not working for automatic mirroring
2. Repository cleanup was not working despite CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
3. Users had no visibility into why scheduling/cleanup wasn't working

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-20 11:06:21 +05:30
Arunavo Ray
de314cf174 Fixed Tests 2025-07-27 19:09:56 +05:30
Arunavo Ray
e637d573a2 Fixes 2025-07-27 00:25:19 +05:30
Arunavo Ray
5f45a9a03d updates 2025-07-26 22:06:29 +05:30
Arunavo Ray
0920314679 More fixes in SSO 2025-07-26 20:33:26 +05:30
Arunavo Ray
1f6add5fff Updates to SSO Testing 2025-07-26 19:45:20 +05:30
Arunavo Ray
d4aa665873 more SSO and OIDC fixes 2025-07-21 12:09:38 +05:30
Arunavo Ray
0244133e7b Fix: Starred Repos Organization Bug | Organization Repos Routing 2025-07-21 10:39:48 +05:30
Arunavo Ray
3458891511 Updates for starred and personal repos 2025-07-18 09:37:38 +05:30
Arunavo Ray
7bd862606b More fixes 2025-07-18 00:52:03 +05:30
Arunavo Ray
251baeb1aa Fixed Private Repo Issues 2025-07-17 23:46:01 +05:30
Arunavo Ray
e6a31512ac some more fixes 2025-07-17 23:31:45 +05:30
Arunavo Ray
cad77320f3 Added Header Authentication 2025-07-17 16:19:56 +05:30
Arunavo Ray
f83711ecd6 Potential security fixes 2025-07-17 13:41:17 +05:30
Arunavo Ray
beedbaf9a4 Added Encryptions to All stored token and passwords 2025-07-16 16:02:34 +05:30
Arunavo Ray
938a909787 tsc fixes 2025-07-11 01:17:54 +05:30
Arunavo Ray
fad78516ef Added SSO and OIDC 2025-07-11 01:04:50 +05:30
Arunavo Ray
6cfe43932f Fixing issues with Better Auth 2025-07-11 00:00:37 +05:30
Arunavo Ray
b838310872 Added Better Auth 2025-07-10 23:15:37 +05:30
Arunavo Ray
46cf117bdf Migrate to Drizzle kit 2025-07-10 21:44:35 +05:30
Arunavo Ray
983b47fa76 fix: update repository references from arunavo4 to RayLabsHQ 2025-07-07 10:52:33 +05:30
Arunavo Ray
e6c4ca0731 feat: implement InlineDestinationEditor for repository destination management and add API support for updating destination organization 2025-06-24 12:11:10 +05:30
Arunavo Ray
cfe65cadca feat: add organization destination update API and enhance organization list editing functionality 2025-06-24 11:16:44 +05:30
Arunavo Ray
68108b8383 feat: add personal repositories organization override and update related configurations 2025-06-24 11:02:57 +05:30
Arunavo Ray
50972713a3 refactor: improve version comparison logic in health API 2025-06-17 15:11:05 +05:30
Arunavo Ray
e94de5c9ca fix: mirror to one org issue 2025-06-17 11:35:09 +05:30
Arunavo Ray
7af1f6da17 fix: implement proper mirror strategies for starred and org repos 2025-06-17 09:26:26 +05:30
Arunavo Ray
e9c12bb9ff feat: enhance API config handling by adding mapping functions for UI and database structures 2025-06-15 13:31:06 +05:30
Arunavo Ray
2eda800a7c refactor: update test files to use bun:test and remove vitest configuration 2025-06-12 10:24:55 +05:30
Arunavo Ray
0d60c2fdf1 feat: implement createSecureErrorResponse for consistent error handling across API routes 2025-06-12 09:50:43 +05:30
Arunavo Ray
0f671a4088 feat: add support for mirroring wiki pages in configuration 2025-06-11 19:48:24 +05:30
Arunavo Ray
5b60cffaae Add fork tags to repository UI and enhance organization cards with repository breakdown
- Add fork tags to repository table and dashboard list components
- Display 'Fork' badge for repositories where isForked is true
- Enhance organization cards to show breakdown of public, private, and fork repositories
- Update organization API to respect user configuration filters (private repos, forks)
- Add visual indicators with colored dots for each repository type
- Ensure consistent filtering between repository and organization APIs
- Fix issue where private repositories weren't showing due to configuration filtering
2025-05-28 12:53:32 +05:30
Arunavo Ray
cba421d606 feat: enhance error logging for better debugging of JSON parsing issues
- Add comprehensive error logging in mirror-repo API endpoint
- Enhance HTTP client error handling with detailed response information
- Improve concurrency utility error reporting with context
- Add specific detection and guidance for JSON parsing errors
- Include troubleshooting information in error responses
- Update tests to accommodate enhanced logging

This will help users diagnose issues like 'JSON Parse error: Unexpected EOF'
by providing detailed information about what responses are being received
from the Gitea API and what might be causing the failures.
2025-05-28 10:13:41 +05:30
Arunavo Ray
38e0fb33b9 fix: resolve JSON parsing error and standardize HTTP client usage
- Fix JSON parsing error in getOrCreateGiteaOrg function (#19)
  - Add content-type validation before JSON parsing
  - Add response cloning for better error debugging
  - Enhance error messages with actual response content
  - Add comprehensive logging for troubleshooting

- Standardize HTTP client usage across codebase
  - Create new http-client.ts utility with consistent error handling
  - Replace all superagent calls with fetch-based functions
  - Replace all axios calls with fetch-based functions
  - Remove superagent, axios, and @types/superagent dependencies
  - Update tests to mock new HTTP client
  - Maintain backward compatibility

- Benefits:
  - Smaller bundle size (removed 3 HTTP client libraries)
  - Better performance (leveraging Bun's optimized fetch)
  - Consistent error handling across all HTTP operations
  - Improved debugging with detailed error messages
  - Easier maintenance with single HTTP client pattern
2025-05-28 09:56:59 +05:30
Arunavo Ray
a28a766f8b refactor: update cleanup and schedule config to use seconds for retentionDays and improve nextRun calculation 2025-05-24 20:12:27 +05:30
Arunavo Ray
d925b3c155 refactor: Remove unnecessary console logs from event polling and retrieval 2025-05-24 18:41:55 +05:30
Arunavo Ray
47e1c7b493 feat: Implement automatic database cleanup feature with configuration options and API support 2025-05-24 18:33:59 +05:30
Arunavo Ray
90944a40c6 feat: Remove health API tests to streamline codebase 2025-05-24 16:14:22 +05:30
Arunavo Ray
a988be1028 feat: Implement comprehensive job recovery and resume process improvements
- Added a startup recovery script to handle interrupted jobs before application startup.
- Enhanced recovery system with database connection validation and stale job cleanup.
- Improved middleware to check for recovery needs and handle recovery during requests.
- Updated health check endpoint to include recovery system status and metrics.
- Introduced test scripts for verifying recovery functionality and job state management.
- Enhanced logging and error handling throughout the recovery process.
2025-05-24 13:45:25 +05:30
Arunavo Ray
98610482ae feat: enhance event management by adding duplicate removal, cleanup functionality, and improving activity logging 2025-05-24 13:25:58 +05:30
Arunavo Ray
309f8c4341 feat: add version information component and integrate version check in health API 2025-05-22 18:51:11 +05:30
Arunavo Ray
894be88a28 feat: migrate testing framework to Bun and update test configurations
- Updated GitHub Actions workflow to use Bun's test runner and coverage reporting.
- Added comprehensive testing documentation for the Gitea Mirror project.
- Refactored test scripts in package.json to align with Bun's testing commands.
- Created new test files for database, Gitea, GitHub, health, and mirroring APIs.
- Implemented mock functions for API tests to handle various scenarios and responses.
- Established a test setup file for consistent test environment configuration.
2025-05-22 18:08:51 +05:30
Arunavo Ray
abe3113755 feat: enhance job resilience with new database schema and recovery mechanisms
- Added new fields to the mirror_jobs table for job resilience, including job_type, batch_id, total_items, completed_items, item_ids, completed_item_ids, in_progress, started_at, completed_at, and last_checkpoint.
- Implemented database migration scripts to update the mirror_jobs table schema.
- Introduced processWithResilience utility for handling item processing with checkpointing and recovery capabilities.
- Updated API routes for mirroring organizations and repositories to utilize the new resilience features.
- Created recovery system to detect and resume interrupted jobs on application startup.
- Added middleware to initialize the recovery system when the server starts.
2025-05-22 14:33:03 +05:30