Commit Graph

13 Commits

Author SHA1 Message Date
ARUNAVO RAY
d0693206c3 feat: selective starred repo mirroring with autoMirrorStarred toggle (#208)
* feat: add autoMirrorStarred toggle for selective starred repo mirroring (#205)

Add `githubConfig.autoMirrorStarred` (default: false) to control whether
starred repos are included in automatic mirroring operations. Manual
per-repo actions always work regardless of this toggle.

Bug fixes:
- Cleanup service no longer orphans starred repos when includeStarred is
  disabled (prevents data loss)
- First-boot auto-start now gates initial mirror behind autoMirror config
  (previously mirrored everything unconditionally)
- "Mirror All" button now respects autoMirrorStarred setting
- Bulk mirror and getAvailableActions now include pending-approval status

Changes span schema, config mapping, env loader, scheduler, cleanup
service, UI settings toggle, and repository components.

* fix: log activity when repos are auto-imported during scheduled sync

Auto-discovered repositories (including newly starred ones) were inserted
into the database without creating activity log entries, so they appeared
in the dashboard but not in the activity log.

* ci: set 10-minute timeout on all CI jobs
2026-03-04 08:22:44 +05:30
ARUNAVO RAY
98da7065e0 feat: smart force-push protection with backup strategies (#206)
* feat: smart force-push protection with backup strategies (#187)

Replace blunt `backupBeforeSync` boolean with `backupStrategy` enum
offering four modes: disabled, always, on-force-push (default), and
block-on-force-push. This dramatically reduces backup storage for large
mirror collections by only creating snapshots when force-pushes are
actually detected.

Detection works by comparing branch SHAs between Gitea and GitHub APIs
before each sync — no git cloning required. Fail-open design ensures
detection errors never block sync.

Key changes:
- Add force-push detection module (branch SHA comparison via APIs)
- Add backup strategy resolver with backward-compat migration
- Add pending-approval repo status with approve/dismiss UI + API
- Add block-on-force-push mode requiring manual approval
- Fix checkAncestry to only treat 404 as confirmed force-push
  (transient errors skip branch instead of false-positive blocking)
- Fix approve-sync to bypass detection gate (skipForcePushDetection)
- Fix backup execution to not be hard-gated by deprecated flag
- Persist backupStrategy through config-mapper round-trip

* fix: resolve four bugs in smart force-push protection

P0: Approve flow re-blocks itself — approve-sync now calls
syncGiteaRepoEnhanced with skipForcePushDetection: true so the
detection+block gate is bypassed on approved syncs.

P1: backupStrategy not persisted — added to both directions of the
config-mapper. Don't inject a default in the mapper; let
resolveBackupStrategy handle fallback so legacy backupBeforeSync
still works for E2E tests and existing configs.

P1: Backup hard-gated by deprecated backupBeforeSync — added force
flag to createPreSyncBundleBackup; strategy-driven callers and
approve-sync pass force: true to bypass the legacy guard.

P1: checkAncestry false positives — now only returns false for
404/422 (confirmed force-push). Transient errors (rate limits, 500s)
are rethrown so detectForcePush skips that branch (fail-open).

* test(e2e): migrate backup tests from backupBeforeSync to backupStrategy

Update E2E tests to use the new backupStrategy enum ("always",
"disabled") instead of the deprecated backupBeforeSync boolean.

* docs: add backup strategy UI screenshot

* refactor(ui): move Destructive Update Protection to GitHub config tab

Relocates the backup strategy section from GiteaConfigForm to
GitHubConfigForm since it protects against GitHub-side force-pushes.
Adds ShieldAlert icon to match other section header patterns.

* docs: add force-push protection documentation and Beta badge

Add docs/FORCE_PUSH_PROTECTION.md covering detection mechanism,
backup strategies, API usage, and troubleshooting. Link it from
README features list and support section. Mark the feature as Beta
in the UI with an outline badge.

* fix(ui): match Beta badge style to Git LFS badge
2026-03-02 15:48:59 +05:30
ARUNAVO RAY
08da526ddd fix(github): keep disabled repos from cleanup while skipping new imports (#191)
* fix: preserve disabled repos while skipping new imports

* ci: upgrade bun to 1.3.6 for test workflow
2026-02-26 10:19:28 +05:30
Arunavo Ray
8d96e176b4 fix: prevent duplicate orgs and repos 2025-10-27 08:44:45 +05:30
Arunavo Ray
5add8766a4 fix(scheduler,config): preserve ENV schedule; add AUTO_MIRROR_REPOS auto-mirroring
- Prevent Automation UI from overriding schedule:
      - mapDbScheduleToUi now parses intervals robustly (cron/duration/seconds) via parseInterval
      - mapUiScheduleToDb merges with existing config and stores interval as seconds (no lossy cron conversion)
      - /api/config passes existing scheduleConfig to preserve ENV-sourced values
      - schedule-sync endpoint uses parseInterval for nextRun calculation
  - Add AUTO_MIRROR_REPOS support and scheduled auto-mirror phase:
      - scheduleConfig schema includes autoImport and autoMirror
      - env-config-loader reads AUTO_MIRROR_REPOS and carries through to DB
      - scheduler auto-mirrors imported/pending/failed repos when autoMirror is enabled before regular sync
      - docker-compose and ENV docs updated with AUTO_MIRROR_REPOS
  - Tests pass and build succeeds
2025-09-14 08:31:31 +05:30
Arunavo Ray
18ecdbc252 fix(sync): batch inserts + normalize nulls to avoid SQLite param mismatch
- Batch repository inserts with dynamic sizing under SQLite 999-param limit
- Normalize undefined → null to keep multi-row insert shapes consistent
- De-duplicate owned + starred repos by fullName (prefer starred variant)
- Enforce uniqueness via (user_id, full_name) + onConflictDoNothing
- Handle starred name collisions (suffix/prefix) across mirror + metadata
- Add repo-utils helpers + tests; guard Octokit.plugin in tests
- Remove manual unique index from entrypoint; rely on drizzle-kit migrations
2025-09-13 23:38:50 +05:30
Arunavo Ray
89ca5abe7d fix: resolve SQLite field mismatch for large starred repo imports (#90)
- Add missing database fields (language, description, mirroredLocation, destinationOrg) to repository operations
  - Add missing organization fields (publicRepositoryCount, privateRepositoryCount, forkRepositoryCount) to schema
  - Update GitRepo interface to include all required database fields
  - Fix GitHub data fetching functions to map all fields correctly
  - Update all sync endpoints (main, repository, organization, scheduler) to handle new fields

  This fixes the "SQLite query expected X values, received Y" error when importing
  large numbers (4.6k+) of starred repositories by ensuring all database fields
  are properly mapped from GitHub API responses through to database insertion.
2025-09-09 09:56:18 +05:30
Arunavo Ray
fab4efd93a Auto-start on boot 2025-09-07 16:29:23 +05:30
Arunavo Ray
b956b71c5f Fixed #87 where the Release Notes was missing 2025-09-07 13:14:41 +05:30
Arunavo Ray
18855f09c4 Imporved a bunch of things in Mirror and sync Automation 2025-08-29 17:49:44 +05:30
Arunavo Ray
ad7418aef2 tsc issues 2025-08-28 08:34:27 +05:30
Arunavo Ray
38a0d1b494 repository cleanup functionality 2025-08-27 19:12:52 +05:30
Arunavo Ray
698eb0b507 fix: Complete Issue #72 - Fix automatic mirroring and repository cleanup
Major fixes for Docker environment variable issues and cleanup functionality:

🔧 **Duration Parser & Scheduler Fixes**
- Add comprehensive duration parser supporting "8h", "30m", "24h" formats
- Fix GITEA_MIRROR_INTERVAL environment variable mapping to scheduler
- Auto-enable scheduler when GITEA_MIRROR_INTERVAL is set
- Improve scheduler logging to clarify timing behavior (from last run, not startup)

🧹 **Repository Cleanup Service**
- Complete repository cleanup service for orphaned repos (unstarred, deleted)
- Fix cleanup configuration logic - now works with CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
- Auto-enable cleanup when deleteIfNotInGitHub is enabled
- Add manual cleanup trigger API endpoint (/api/cleanup/trigger)
- Support archive/delete actions with dry-run mode and protected repos

🐛 **Environment Variable Integration**
- Fix scheduler not recognizing GITEA_MIRROR_INTERVAL=8h
- Fix cleanup requiring both CLEANUP_DELETE_FROM_GITEA and CLEANUP_DELETE_IF_NOT_IN_GITHUB
- Auto-enable services when relevant environment variables are set
- Better error logging and debugging information

📚 **Documentation Updates**
- Update .env.example with auto-enabling behavior notes
- Update ENVIRONMENT_VARIABLES.md with clarified functionality
- Add comprehensive tests for duration parsing

This resolves the core issues where:
1. GITEA_MIRROR_INTERVAL=8h was not working for automatic mirroring
2. Repository cleanup was not working despite CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
3. Users had no visibility into why scheduling/cleanup wasn't working

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-20 11:06:21 +05:30