Compare commits

..

11 Commits

Author SHA1 Message Date
Arunavo Ray
a5b4482c8a working on a fix for SSO issue 2025-09-14 10:18:37 +05:30
Arunavo Ray
5add8766a4 fix(scheduler,config): preserve ENV schedule; add AUTO_MIRROR_REPOS auto-mirroring
- Prevent Automation UI from overriding schedule:
      - mapDbScheduleToUi now parses intervals robustly (cron/duration/seconds) via parseInterval
      - mapUiScheduleToDb merges with existing config and stores interval as seconds (no lossy cron conversion)
      - /api/config passes existing scheduleConfig to preserve ENV-sourced values
      - schedule-sync endpoint uses parseInterval for nextRun calculation
  - Add AUTO_MIRROR_REPOS support and scheduled auto-mirror phase:
      - scheduleConfig schema includes autoImport and autoMirror
      - env-config-loader reads AUTO_MIRROR_REPOS and carries through to DB
      - scheduler auto-mirrors imported/pending/failed repos when autoMirror is enabled before regular sync
      - docker-compose and ENV docs updated with AUTO_MIRROR_REPOS
  - Tests pass and build succeeds
2025-09-14 08:31:31 +05:30
Arunavo Ray
6ce70bb5bf chore(version): bump to 3.7.1\n\ncleanup: attempt fix for orphaned repo archiving (refs #84)\n- Sanitize mirror rename to satisfy AlphaDashDot; timestamped fallback\n- Resolve Gitea owner robustly via mirroredLocation/strategy; verify presence\n- Add 'archived' status to Zod enums; set isArchived on archive\n- Update CHANGELOG entry without closing keyword 2025-09-14 07:53:36 +05:30
Arunavo Ray
f3aae2ec94 fix for repo name collison 2025-09-14 00:13:13 +05:30
Arunavo Ray
46d5ec46fc Updated deisgn for 'Duplicate collision strategy' 2025-09-13 23:54:14 +05:30
Arunavo Ray
0caa53b67f v3.7.0 2025-09-13 23:39:50 +05:30
Arunavo Ray
18ecdbc252 fix(sync): batch inserts + normalize nulls to avoid SQLite param mismatch
- Batch repository inserts with dynamic sizing under SQLite 999-param limit
- Normalize undefined → null to keep multi-row insert shapes consistent
- De-duplicate owned + starred repos by fullName (prefer starred variant)
- Enforce uniqueness via (user_id, full_name) + onConflictDoNothing
- Handle starred name collisions (suffix/prefix) across mirror + metadata
- Add repo-utils helpers + tests; guard Octokit.plugin in tests
- Remove manual unique index from entrypoint; rely on drizzle-kit migrations
2025-09-13 23:38:50 +05:30
Arunavo Ray
51a6c8ca58 Added product hunt badge on website 2025-09-12 01:44:13 +05:30
Arunavo Ray
41b8806268 update packages 2025-09-10 09:49:08 +05:30
ARUNAVO RAY
ac5c7800c1 Merge pull request #93 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-73ea615029
Bump vite from 6.3.5 to 6.3.6 in /www in the npm_and_yarn group across 1 directory
2025-09-10 09:46:02 +05:30
dependabot[bot]
13e7661f07 Bump vite in /www in the npm_and_yarn group across 1 directory
Bumps the npm_and_yarn group with 1 update in the /www directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 6.3.5 to 6.3.6
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.3.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.3.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.3.6
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-10 02:49:30 +00:00
42 changed files with 36827 additions and 366 deletions

View File

@@ -58,6 +58,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Updated README with new features
- Enhanced CLAUDE.md with repository status definitions
## [3.7.1] - 2025-09-14
### Fixed
- Cleanup archiving for mirror repositories now works reliably (refs #84; awaiting user confirmation).
- Gitea rejects names violating the AlphaDashDot rule; archiving a mirror now uses a sanitized rename strategy (`archived-<name>`), with a timestamped fallback on conflicts or validation errors.
- Owner resolution during cleanup no longer uses the GitHub owner by mistake. It prefers `mirroredLocation`, falls back to computed Gitea owner via configuration, and verifies location with a presence check to avoid `GetUserByName` 404s.
- Repositories UI crash resolved when cleanup marked repos as archived.
- Added `"archived"` to repository/job status enums, fixing Zod validation errors on the Repositories page.
### Changed
- Archiving logic for mirror repos is non-destructive by design: data is preserved, repo is renamed with an archive marker, and mirror interval is reduced (besteffort) to minimize sync attempts.
- Cleanup service updates DB to `status: "archived"` and `isArchived: true` on successful archive path.
### Notes
- This release addresses the scenario where a GitHub source disappears (deleted/banned), ensuring Gitea backups are preserved even when using `CLEANUP_DELETE_IF_NOT_IN_GITHUB=true` with `CLEANUP_ORPHANED_REPO_ACTION=archive`.
- No database migration required.
## [3.2.6] - 2025-08-09
### Fixed

View File

@@ -8,7 +8,7 @@
"@astrojs/mdx": "4.3.5",
"@astrojs/node": "9.4.3",
"@astrojs/react": "^4.3.1",
"@better-auth/sso": "^1.3.8",
"@better-auth/sso": "^1.3.9",
"@octokit/plugin-throttling": "^11.0.1",
"@octokit/rest": "^22.0.0",
"@radix-ui/react-accordion": "^1.2.12",
@@ -34,7 +34,7 @@
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.1.12",
"@types/react-dom": "^19.1.9",
"astro": "^5.13.6",
"astro": "^5.13.7",
"bcryptjs": "^3.0.2",
"better-auth": "^1.3.9",
"canvas-confetti": "^1.9.3",
@@ -149,7 +149,7 @@
"@babel/types": ["@babel/types@7.28.2", "", { "dependencies": { "@babel/helper-string-parser": "^7.27.1", "@babel/helper-validator-identifier": "^7.27.1" } }, "sha512-ruv7Ae4J5dUYULmeXw1gmb7rYRz57OWCPM57pHojnLq/3Z1CK2lNSLTCVjxVk1F/TZHwOZZrOWi0ur95BbLxNQ=="],
"@better-auth/sso": ["@better-auth/sso@1.3.8", "", { "dependencies": { "@better-fetch/fetch": "^1.1.18", "fast-xml-parser": "^5.2.5", "jose": "^5.10.0", "oauth2-mock-server": "^7.2.1", "samlify": "^2.10.1", "zod": "^4.1.5" }, "peerDependencies": { "better-auth": "1.3.8" } }, "sha512-ohJl4uTRwVACu8840A5Ys/z2jus/vEsCrWvOj/RannsZ6CxQAjr8utYYXXs6lVn08ynOcuT4m0OsYRbrw7a42g=="],
"@better-auth/sso": ["@better-auth/sso@1.3.9", "", { "dependencies": { "@better-fetch/fetch": "^1.1.18", "fast-xml-parser": "^5.2.5", "jose": "^5.10.0", "oauth2-mock-server": "^7.2.1", "samlify": "^2.10.1", "zod": "^4.1.5" }, "peerDependencies": { "better-auth": "1.3.9" } }, "sha512-rYbFtl/MpD6iEyKSnwTlG8jdu6xqYmDF1Bmx2P8NaXzTtkhb432Q045vqB7cWpZKpLCf2tNK5QeAyPhS0zg+Gw=="],
"@better-auth/utils": ["@better-auth/utils@0.2.6", "", { "dependencies": { "uncrypto": "^0.1.3" } }, "sha512-3y/vaL5Ox33dBwgJ6ub3OPkVqr6B5xL2kgxNHG8eHZuryLyG/4JSPGqjbdRSgjuy9kALUZYDFl+ORIAxlWMSuA=="],
@@ -183,7 +183,7 @@
"@emmetio/stream-reader-utils": ["@emmetio/stream-reader-utils@0.1.0", "", {}, "sha512-ZsZ2I9Vzso3Ho/pjZFsmmZ++FWeEd/txqybHTm4OgaZzdS8V9V/YYWQwg5TC38Z7uLWUV1vavpLLbjJtKubR1A=="],
"@emnapi/runtime": ["@emnapi/runtime@1.4.3", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-pBPWdu6MLKROBX05wSNKcNb++m5Er+KQ9QkB+WVM+pW2Kx9hoSrVTnu3BdkI5eBLZoKu/J6mW/B6i6bJB2ytXQ=="],
"@emnapi/runtime": ["@emnapi/runtime@1.4.5", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-++LApOtY0pEEz1zrd9vy1/zXVaVJJ/EbAF3u0fXIzPJEDtnITsBGbbK0EkM72amhl/R5b+5xx0Y/QhcVOpuulg=="],
"@esbuild-kit/esm-loader": ["tsx@4.20.5", "", { "dependencies": { "esbuild": "~0.25.0", "get-tsconfig": "^4.7.5" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "bin": { "tsx": "dist/cli.mjs" } }, "sha512-+wKjMNU9w/EaQayHXb7WA7ZaHY6hN8WgfvHNQ3t1PnU91/7O8TcTnIhCDYTZwnt8JsO9IBqZ30Ln1r7pPF52Aw=="],
@@ -249,43 +249,49 @@
"@hexagon/base64": ["@hexagon/base64@1.1.28", "", {}, "sha512-lhqDEAvWixy3bZ+UOYbPwUbBkwBq5C1LAJ/xPC8Oi+lL54oyakv/npbA0aU2hgCsx/1NUd4IBvV03+aUBWxerw=="],
"@img/sharp-darwin-arm64": ["@img/sharp-darwin-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-arm64": "1.0.4" }, "os": "darwin", "cpu": "arm64" }, "sha512-UT4p+iz/2H4twwAoLCqfA9UH5pI6DggwKEGuaPy7nCVQ8ZsiY5PIcrRvD1DzuY3qYL07NtIQcWnBSY/heikIFQ=="],
"@img/sharp-darwin-arm64": ["@img/sharp-darwin-arm64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-arm64": "1.2.0" }, "os": "darwin", "cpu": "arm64" }, "sha512-ryFMfvxxpQRsgZJqBd4wsttYQbCxsJksrv9Lw/v798JcQ8+w84mBWuXwl+TT0WJ/WrYOLaYpwQXi3sA9nTIaIg=="],
"@img/sharp-darwin-x64": ["@img/sharp-darwin-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-x64": "1.0.4" }, "os": "darwin", "cpu": "x64" }, "sha512-fyHac4jIc1ANYGRDxtiqelIbdWkIuQaI84Mv45KvGRRxSAa7o7d1ZKAOBaYbnepLC1WqxfpimdeWfvqqSGwR2Q=="],
"@img/sharp-darwin-x64": ["@img/sharp-darwin-x64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-x64": "1.2.0" }, "os": "darwin", "cpu": "x64" }, "sha512-yHpJYynROAj12TA6qil58hmPmAwxKKC7reUqtGLzsOHfP7/rniNGTL8tjWX6L3CTV4+5P4ypcS7Pp+7OB+8ihA=="],
"@img/sharp-libvips-darwin-arm64": ["@img/sharp-libvips-darwin-arm64@1.0.4", "", { "os": "darwin", "cpu": "arm64" }, "sha512-XblONe153h0O2zuFfTAbQYAX2JhYmDHeWikp1LM9Hul9gVPjFY427k6dFEcOL72O01QxQsWi761svJ/ev9xEDg=="],
"@img/sharp-libvips-darwin-arm64": ["@img/sharp-libvips-darwin-arm64@1.2.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-sBZmpwmxqwlqG9ueWFXtockhsxefaV6O84BMOrhtg/YqbTaRdqDE7hxraVE3y6gVM4eExmfzW4a8el9ArLeEiQ=="],
"@img/sharp-libvips-darwin-x64": ["@img/sharp-libvips-darwin-x64@1.0.4", "", { "os": "darwin", "cpu": "x64" }, "sha512-xnGR8YuZYfJGmWPvmlunFaWJsb9T/AO2ykoP3Fz/0X5XV2aoYBPkX6xqCQvUTKKiLddarLaxpzNe+b1hjeWHAQ=="],
"@img/sharp-libvips-darwin-x64": ["@img/sharp-libvips-darwin-x64@1.2.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-M64XVuL94OgiNHa5/m2YvEQI5q2cl9d/wk0qFTDVXcYzi43lxuiFTftMR1tOnFQovVXNZJ5TURSDK2pNe9Yzqg=="],
"@img/sharp-libvips-linux-arm": ["@img/sharp-libvips-linux-arm@1.0.5", "", { "os": "linux", "cpu": "arm" }, "sha512-gvcC4ACAOPRNATg/ov8/MnbxFDJqf/pDePbBnuBDcjsI8PssmjoKMAz4LtLaVi+OnSb5FK/yIOamqDwGmXW32g=="],
"@img/sharp-libvips-linux-arm": ["@img/sharp-libvips-linux-arm@1.2.0", "", { "os": "linux", "cpu": "arm" }, "sha512-mWd2uWvDtL/nvIzThLq3fr2nnGfyr/XMXlq8ZJ9WMR6PXijHlC3ksp0IpuhK6bougvQrchUAfzRLnbsen0Cqvw=="],
"@img/sharp-libvips-linux-arm64": ["@img/sharp-libvips-linux-arm64@1.0.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-9B+taZ8DlyyqzZQnoeIvDVR/2F4EbMepXMc/NdVbkzsJbzkUjhXv/70GQJ7tdLA4YJgNP25zukcxpX2/SueNrA=="],
"@img/sharp-libvips-linux-arm64": ["@img/sharp-libvips-linux-arm64@1.2.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-RXwd0CgG+uPRX5YYrkzKyalt2OJYRiJQ8ED/fi1tq9WQW2jsQIn0tqrlR5l5dr/rjqq6AHAxURhj2DVjyQWSOA=="],
"@img/sharp-libvips-linux-s390x": ["@img/sharp-libvips-linux-s390x@1.0.4", "", { "os": "linux", "cpu": "s390x" }, "sha512-u7Wz6ntiSSgGSGcjZ55im6uvTrOxSIS8/dgoVMoiGE9I6JAfU50yH5BoDlYA1tcuGS7g/QNtetJnxA6QEsCVTA=="],
"@img/sharp-libvips-linux-ppc64": ["@img/sharp-libvips-linux-ppc64@1.2.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-Xod/7KaDDHkYu2phxxfeEPXfVXFKx70EAFZ0qyUdOjCcxbjqyJOEUpDe6RIyaunGxT34Anf9ue/wuWOqBW2WcQ=="],
"@img/sharp-libvips-linux-x64": ["@img/sharp-libvips-linux-x64@1.0.4", "", { "os": "linux", "cpu": "x64" }, "sha512-MmWmQ3iPFZr0Iev+BAgVMb3ZyC4KeFc3jFxnNbEPas60e1cIfevbtuyf9nDGIzOaW9PdnDciJm+wFFaTlj5xYw=="],
"@img/sharp-libvips-linux-s390x": ["@img/sharp-libvips-linux-s390x@1.2.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-eMKfzDxLGT8mnmPJTNMcjfO33fLiTDsrMlUVcp6b96ETbnJmd4uvZxVJSKPQfS+odwfVaGifhsB07J1LynFehw=="],
"@img/sharp-libvips-linuxmusl-arm64": ["@img/sharp-libvips-linuxmusl-arm64@1.0.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-9Ti+BbTYDcsbp4wfYib8Ctm1ilkugkA/uscUn6UXK1ldpC1JjiXbLfFZtRlBhjPZ5o1NCLiDbg8fhUPKStHoTA=="],
"@img/sharp-libvips-linux-x64": ["@img/sharp-libvips-linux-x64@1.2.0", "", { "os": "linux", "cpu": "x64" }, "sha512-ZW3FPWIc7K1sH9E3nxIGB3y3dZkpJlMnkk7z5tu1nSkBoCgw2nSRTFHI5pB/3CQaJM0pdzMF3paf9ckKMSE9Tg=="],
"@img/sharp-libvips-linuxmusl-x64": ["@img/sharp-libvips-linuxmusl-x64@1.0.4", "", { "os": "linux", "cpu": "x64" }, "sha512-viYN1KX9m+/hGkJtvYYp+CCLgnJXwiQB39damAO7WMdKWlIhmYTfHjwSbQeUK/20vY154mwezd9HflVFM1wVSw=="],
"@img/sharp-libvips-linuxmusl-arm64": ["@img/sharp-libvips-linuxmusl-arm64@1.2.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UG+LqQJbf5VJ8NWJ5Z3tdIe/HXjuIdo4JeVNADXBFuG7z9zjoegpzzGIyV5zQKi4zaJjnAd2+g2nna8TZvuW9Q=="],
"@img/sharp-linux-arm": ["@img/sharp-linux-arm@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm": "1.0.5" }, "os": "linux", "cpu": "arm" }, "sha512-JTS1eldqZbJxjvKaAkxhZmBqPRGmxgu+qFKSInv8moZ2AmT5Yib3EQ1c6gp493HvrvV8QgdOXdyaIBrhvFhBMQ=="],
"@img/sharp-libvips-linuxmusl-x64": ["@img/sharp-libvips-linuxmusl-x64@1.2.0", "", { "os": "linux", "cpu": "x64" }, "sha512-SRYOLR7CXPgNze8akZwjoGBoN1ThNZoqpOgfnOxmWsklTGVfJiGJoC/Lod7aNMGA1jSsKWM1+HRX43OP6p9+6Q=="],
"@img/sharp-linux-arm64": ["@img/sharp-linux-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm64": "1.0.4" }, "os": "linux", "cpu": "arm64" }, "sha512-JMVv+AMRyGOHtO1RFBiJy/MBsgz0x4AWrT6QoEVVTyh1E39TrCUpTRI7mx9VksGX4awWASxqCYLCV4wBZHAYxA=="],
"@img/sharp-linux-arm": ["@img/sharp-linux-arm@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm": "1.2.0" }, "os": "linux", "cpu": "arm" }, "sha512-oBK9l+h6KBN0i3dC8rYntLiVfW8D8wH+NPNT3O/WBHeW0OQWCjfWksLUaPidsrDKpJgXp3G3/hkmhptAW0I3+A=="],
"@img/sharp-linux-s390x": ["@img/sharp-linux-s390x@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-s390x": "1.0.4" }, "os": "linux", "cpu": "s390x" }, "sha512-y/5PCd+mP4CA/sPDKl2961b+C9d+vPAveS33s6Z3zfASk2j5upL6fXVPZi7ztePZ5CuH+1kW8JtvxgbuXHRa4Q=="],
"@img/sharp-linux-arm64": ["@img/sharp-linux-arm64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm64": "1.2.0" }, "os": "linux", "cpu": "arm64" }, "sha512-QdrKe3EvQrqwkDrtuTIjI0bu6YEJHTgEeqdzI3uWJOH6G1O8Nl1iEeVYRGdj1h5I21CqxSvQp1Yv7xeU3ZewbA=="],
"@img/sharp-linux-x64": ["@img/sharp-linux-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-x64": "1.0.4" }, "os": "linux", "cpu": "x64" }, "sha512-opC+Ok5pRNAzuvq1AG0ar+1owsu842/Ab+4qvU879ippJBHvyY5n2mxF1izXqkPYlGuP/M556uh53jRLJmzTWA=="],
"@img/sharp-linux-ppc64": ["@img/sharp-linux-ppc64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linux-ppc64": "1.2.0" }, "os": "linux", "cpu": "ppc64" }, "sha512-GLtbLQMCNC5nxuImPR2+RgrviwKwVql28FWZIW1zWruy6zLgA5/x2ZXk3mxj58X/tszVF69KK0Is83V8YgWhLA=="],
"@img/sharp-linuxmusl-arm64": ["@img/sharp-linuxmusl-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-arm64": "1.0.4" }, "os": "linux", "cpu": "arm64" }, "sha512-XrHMZwGQGvJg2V/oRSUfSAfjfPxO+4DkiRh6p2AFjLQztWUuY/o8Mq0eMQVIY7HJ1CDQUJlxGGZRw1a5bqmd1g=="],
"@img/sharp-linux-s390x": ["@img/sharp-linux-s390x@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linux-s390x": "1.2.0" }, "os": "linux", "cpu": "s390x" }, "sha512-3gahT+A6c4cdc2edhsLHmIOXMb17ltffJlxR0aC2VPZfwKoTGZec6u5GrFgdR7ciJSsHT27BD3TIuGcuRT0KmQ=="],
"@img/sharp-linuxmusl-x64": ["@img/sharp-linuxmusl-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-x64": "1.0.4" }, "os": "linux", "cpu": "x64" }, "sha512-WT+d/cgqKkkKySYmqoZ8y3pxx7lx9vVejxW/W4DOFMYVSkErR+w7mf2u8m/y4+xHe7yY9DAXQMWQhpnMuFfScw=="],
"@img/sharp-linux-x64": ["@img/sharp-linux-x64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linux-x64": "1.2.0" }, "os": "linux", "cpu": "x64" }, "sha512-8kYso8d806ypnSq3/Ly0QEw90V5ZoHh10yH0HnrzOCr6DKAPI6QVHvwleqMkVQ0m+fc7EH8ah0BB0QPuWY6zJQ=="],
"@img/sharp-wasm32": ["@img/sharp-wasm32@0.33.5", "", { "dependencies": { "@emnapi/runtime": "^1.2.0" }, "cpu": "none" }, "sha512-ykUW4LVGaMcU9lu9thv85CbRMAwfeadCJHRsg2GmeRa/cJxsVY9Rbd57JcMxBkKHag5U/x7TSBpScF4U8ElVzg=="],
"@img/sharp-linuxmusl-arm64": ["@img/sharp-linuxmusl-arm64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-arm64": "1.2.0" }, "os": "linux", "cpu": "arm64" }, "sha512-vAjbHDlr4izEiXM1OTggpCcPg9tn4YriK5vAjowJsHwdBIdx0fYRsURkxLG2RLm9gyBq66gwtWI8Gx0/ov+JKQ=="],
"@img/sharp-win32-ia32": ["@img/sharp-win32-ia32@0.33.5", "", { "os": "win32", "cpu": "ia32" }, "sha512-T36PblLaTwuVJ/zw/LaH0PdZkRz5rd3SmMHX8GSmR7vtNSP5Z6bQkExdSK7xGWyxLw4sUknBuugTelgw2faBbQ=="],
"@img/sharp-linuxmusl-x64": ["@img/sharp-linuxmusl-x64@0.34.3", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-x64": "1.2.0" }, "os": "linux", "cpu": "x64" }, "sha512-gCWUn9547K5bwvOn9l5XGAEjVTTRji4aPTqLzGXHvIr6bIDZKNTA34seMPgM0WmSf+RYBH411VavCejp3PkOeQ=="],
"@img/sharp-win32-x64": ["@img/sharp-win32-x64@0.33.5", "", { "os": "win32", "cpu": "x64" }, "sha512-MpY/o8/8kj+EcnxwvrP4aTJSWw/aZ7JIGR4aBeZkZw5B7/Jn+tY9/VNwtcoGmdT7GfggGIU4kygOMSbYnOrAbg=="],
"@img/sharp-wasm32": ["@img/sharp-wasm32@0.34.3", "", { "dependencies": { "@emnapi/runtime": "^1.4.4" }, "cpu": "none" }, "sha512-+CyRcpagHMGteySaWos8IbnXcHgfDn7pO2fiC2slJxvNq9gDipYBN42/RagzctVRKgxATmfqOSulgZv5e1RdMg=="],
"@img/sharp-win32-arm64": ["@img/sharp-win32-arm64@0.34.3", "", { "os": "win32", "cpu": "arm64" }, "sha512-MjnHPnbqMXNC2UgeLJtX4XqoVHHlZNd+nPt1kRPmj63wURegwBhZlApELdtxM2OIZDRv/DFtLcNhVbd1z8GYXQ=="],
"@img/sharp-win32-ia32": ["@img/sharp-win32-ia32@0.34.3", "", { "os": "win32", "cpu": "ia32" }, "sha512-xuCdhH44WxuXgOM714hn4amodJMZl3OEvf0GVTm0BEyMeA2to+8HEdRPShH0SLYptJY1uBw+SCFP9WVQi1Q/cw=="],
"@img/sharp-win32-x64": ["@img/sharp-win32-x64@0.34.3", "", { "os": "win32", "cpu": "x64" }, "sha512-OWwz05d++TxzLEv4VnsTz5CmZ6mI6S05sfQGEMrNrQcOEERbX46332IvE7pO/EUiw7jUrrS40z/M7kPyjfl04g=="],
"@isaacs/fs-minipass": ["@isaacs/fs-minipass@4.0.1", "", { "dependencies": { "minipass": "^7.0.4" } }, "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w=="],
@@ -673,7 +679,7 @@
"astring": ["astring@1.9.0", "", { "bin": { "astring": "bin/astring" } }, "sha512-LElXdjswlqjWrPpJFg1Fx4wpkOCxj1TDHlSV4PlaRxHGWko024xICaa97ZkMfs6DRKlCguiAI+rbXv5GWwXIkg=="],
"astro": ["astro@5.13.6", "", { "dependencies": { "@astrojs/compiler": "^2.12.2", "@astrojs/internal-helpers": "0.7.2", "@astrojs/markdown-remark": "6.3.6", "@astrojs/telemetry": "3.3.0", "@capsizecss/unpack": "^2.4.0", "@oslojs/encoding": "^1.1.0", "@rollup/pluginutils": "^5.2.0", "acorn": "^8.15.0", "aria-query": "^5.3.2", "axobject-query": "^4.1.0", "boxen": "8.0.1", "ci-info": "^4.3.0", "clsx": "^2.1.1", "common-ancestor-path": "^1.0.1", "cookie": "^1.0.2", "cssesc": "^3.0.0", "debug": "^4.4.1", "deterministic-object-hash": "^2.0.2", "devalue": "^5.1.1", "diff": "^5.2.0", "dlv": "^1.1.3", "dset": "^3.1.4", "es-module-lexer": "^1.7.0", "esbuild": "^0.25.0", "estree-walker": "^3.0.3", "flattie": "^1.1.1", "fontace": "~0.3.0", "github-slugger": "^2.0.0", "html-escaper": "3.0.3", "http-cache-semantics": "^4.2.0", "import-meta-resolve": "^4.2.0", "js-yaml": "^4.1.0", "kleur": "^4.1.5", "magic-string": "^0.30.18", "magicast": "^0.3.5", "mrmime": "^2.0.1", "neotraverse": "^0.6.18", "p-limit": "^6.2.0", "p-queue": "^8.1.0", "package-manager-detector": "^1.3.0", "picomatch": "^4.0.3", "prompts": "^2.4.2", "rehype": "^13.0.2", "semver": "^7.7.2", "shiki": "^3.12.0", "simple-swizzle": "0.2.2", "smol-toml": "^1.4.2", "tinyexec": "^0.3.2", "tinyglobby": "^0.2.14", "tsconfck": "^3.1.6", "ultrahtml": "^1.6.0", "unifont": "~0.5.2", "unist-util-visit": "^5.0.0", "unstorage": "^1.17.0", "vfile": "^6.0.3", "vite": "^6.3.6", "vitefu": "^1.1.1", "xxhash-wasm": "^1.1.0", "yargs-parser": "^21.1.1", "yocto-spinner": "^0.2.3", "zod": "^3.25.76", "zod-to-json-schema": "^3.24.6", "zod-to-ts": "^1.2.0" }, "optionalDependencies": { "sharp": "^0.33.3" }, "bin": { "astro": "astro.js" } }, "sha512-chy1J+AO3d4lui4MjUyqusiW1jilfkviCBDz+c2MoXxhIImF96GqoliX+79fGy6KMsnMh5lUn+qwy3yUBJqZqg=="],
"astro": ["astro@5.13.7", "", { "dependencies": { "@astrojs/compiler": "^2.12.2", "@astrojs/internal-helpers": "0.7.2", "@astrojs/markdown-remark": "6.3.6", "@astrojs/telemetry": "3.3.0", "@capsizecss/unpack": "^2.4.0", "@oslojs/encoding": "^1.1.0", "@rollup/pluginutils": "^5.2.0", "acorn": "^8.15.0", "aria-query": "^5.3.2", "axobject-query": "^4.1.0", "boxen": "8.0.1", "ci-info": "^4.3.0", "clsx": "^2.1.1", "common-ancestor-path": "^1.0.1", "cookie": "^1.0.2", "cssesc": "^3.0.0", "debug": "^4.4.1", "deterministic-object-hash": "^2.0.2", "devalue": "^5.1.1", "diff": "^5.2.0", "dlv": "^1.1.3", "dset": "^3.1.4", "es-module-lexer": "^1.7.0", "esbuild": "^0.25.0", "estree-walker": "^3.0.3", "flattie": "^1.1.1", "fontace": "~0.3.0", "github-slugger": "^2.0.0", "html-escaper": "3.0.3", "http-cache-semantics": "^4.2.0", "import-meta-resolve": "^4.2.0", "js-yaml": "^4.1.0", "kleur": "^4.1.5", "magic-string": "^0.30.18", "magicast": "^0.3.5", "mrmime": "^2.0.1", "neotraverse": "^0.6.18", "p-limit": "^6.2.0", "p-queue": "^8.1.0", "package-manager-detector": "^1.3.0", "picomatch": "^4.0.3", "prompts": "^2.4.2", "rehype": "^13.0.2", "semver": "^7.7.2", "shiki": "^3.12.0", "smol-toml": "^1.4.2", "tinyexec": "^0.3.2", "tinyglobby": "^0.2.14", "tsconfck": "^3.1.6", "ultrahtml": "^1.6.0", "unifont": "~0.5.2", "unist-util-visit": "^5.0.0", "unstorage": "^1.17.0", "vfile": "^6.0.3", "vite": "^6.3.6", "vitefu": "^1.1.1", "xxhash-wasm": "^1.1.0", "yargs-parser": "^21.1.1", "yocto-spinner": "^0.2.3", "zod": "^3.25.76", "zod-to-json-schema": "^3.24.6", "zod-to-ts": "^1.2.0" }, "optionalDependencies": { "sharp": "^0.34.0" }, "bin": { "astro": "astro.js" } }, "sha512-Of2tST7ErbE4y1dVb4aWDXaQSIRBAfraJ4jDqaA3PzPRJOn6Ina36+tQ+8BezjYqiWwRRJdOEE07PRAJXnsddw=="],
"axobject-query": ["axobject-query@4.1.0", "", {}, "sha512-qIj0G9wZbMGNLjLmg1PT6v2mE9AH2zlnADJD/2tC6E00hgmhUOfEB6greHPAfLRSufHqROIUTkw6E+M3lH0PTQ=="],
@@ -1491,7 +1497,7 @@
"setprototypeof": ["setprototypeof@1.2.0", "", {}, "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw=="],
"sharp": ["sharp@0.33.5", "", { "dependencies": { "color": "^4.2.3", "detect-libc": "^2.0.3", "semver": "^7.6.3" }, "optionalDependencies": { "@img/sharp-darwin-arm64": "0.33.5", "@img/sharp-darwin-x64": "0.33.5", "@img/sharp-libvips-darwin-arm64": "1.0.4", "@img/sharp-libvips-darwin-x64": "1.0.4", "@img/sharp-libvips-linux-arm": "1.0.5", "@img/sharp-libvips-linux-arm64": "1.0.4", "@img/sharp-libvips-linux-s390x": "1.0.4", "@img/sharp-libvips-linux-x64": "1.0.4", "@img/sharp-libvips-linuxmusl-arm64": "1.0.4", "@img/sharp-libvips-linuxmusl-x64": "1.0.4", "@img/sharp-linux-arm": "0.33.5", "@img/sharp-linux-arm64": "0.33.5", "@img/sharp-linux-s390x": "0.33.5", "@img/sharp-linux-x64": "0.33.5", "@img/sharp-linuxmusl-arm64": "0.33.5", "@img/sharp-linuxmusl-x64": "0.33.5", "@img/sharp-wasm32": "0.33.5", "@img/sharp-win32-ia32": "0.33.5", "@img/sharp-win32-x64": "0.33.5" } }, "sha512-haPVm1EkS9pgvHrQ/F3Xy+hgcuMV0Wm9vfIBSiwZ05k+xgb0PkBQpGsAA/oWdDobNaZTH5ppvHtzCFbnSEwHVw=="],
"sharp": ["sharp@0.34.3", "", { "dependencies": { "color": "^4.2.3", "detect-libc": "^2.0.4", "semver": "^7.7.2" }, "optionalDependencies": { "@img/sharp-darwin-arm64": "0.34.3", "@img/sharp-darwin-x64": "0.34.3", "@img/sharp-libvips-darwin-arm64": "1.2.0", "@img/sharp-libvips-darwin-x64": "1.2.0", "@img/sharp-libvips-linux-arm": "1.2.0", "@img/sharp-libvips-linux-arm64": "1.2.0", "@img/sharp-libvips-linux-ppc64": "1.2.0", "@img/sharp-libvips-linux-s390x": "1.2.0", "@img/sharp-libvips-linux-x64": "1.2.0", "@img/sharp-libvips-linuxmusl-arm64": "1.2.0", "@img/sharp-libvips-linuxmusl-x64": "1.2.0", "@img/sharp-linux-arm": "0.34.3", "@img/sharp-linux-arm64": "0.34.3", "@img/sharp-linux-ppc64": "0.34.3", "@img/sharp-linux-s390x": "0.34.3", "@img/sharp-linux-x64": "0.34.3", "@img/sharp-linuxmusl-arm64": "0.34.3", "@img/sharp-linuxmusl-x64": "0.34.3", "@img/sharp-wasm32": "0.34.3", "@img/sharp-win32-arm64": "0.34.3", "@img/sharp-win32-ia32": "0.34.3", "@img/sharp-win32-x64": "0.34.3" } }, "sha512-eX2IQ6nFohW4DbvHIOLRB3MHFpYqaqvXd3Tp5e/T/dSH83fxaNJQRvDMhASmkNTsNTVF2/OOopzRCt7xokgPfg=="],
"shiki": ["shiki@3.12.2", "", { "dependencies": { "@shikijs/core": "3.12.2", "@shikijs/engine-javascript": "3.12.2", "@shikijs/engine-oniguruma": "3.12.2", "@shikijs/langs": "3.12.2", "@shikijs/themes": "3.12.2", "@shikijs/types": "3.12.2", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4" } }, "sha512-uIrKI+f9IPz1zDT+GMz+0RjzKJiijVr6WDWm9Pe3NNY6QigKCfifCEv9v9R2mDASKKjzjQ2QpFLcxaR3iHSnMA=="],

View File

@@ -57,6 +57,7 @@ services:
- SCHEDULE_ENABLED=${SCHEDULE_ENABLED:-false}
- GITEA_MIRROR_INTERVAL=${GITEA_MIRROR_INTERVAL:-8h}
- AUTO_IMPORT_REPOS=${AUTO_IMPORT_REPOS:-true}
- AUTO_MIRROR_REPOS=${AUTO_MIRROR_REPOS:-false}
# Repository Cleanup Configuration
- CLEANUP_DELETE_IF_NOT_IN_GITHUB=${CLEANUP_DELETE_IF_NOT_IN_GITHUB:-false}
- CLEANUP_ORPHANED_REPO_ACTION=${CLEANUP_ORPHANED_REPO_ACTION:-archive}

View File

@@ -172,6 +172,7 @@ if [ ! -f "/app/data/gitea-mirror.db" ]; then
owner TEXT NOT NULL,
organization TEXT,
mirrored_location TEXT DEFAULT '',
destination_org TEXT,
is_private INTEGER NOT NULL DEFAULT 0,
is_fork INTEGER NOT NULL DEFAULT 0,
forked_from TEXT,
@@ -181,6 +182,8 @@ if [ ! -f "/app/data/gitea-mirror.db" ]; then
size INTEGER NOT NULL DEFAULT 0,
has_lfs INTEGER NOT NULL DEFAULT 0,
has_submodules INTEGER NOT NULL DEFAULT 0,
language TEXT,
description TEXT,
default_branch TEXT NOT NULL,
visibility TEXT NOT NULL DEFAULT 'public',
status TEXT NOT NULL DEFAULT 'imported',
@@ -192,6 +195,8 @@ if [ ! -f "/app/data/gitea-mirror.db" ]; then
FOREIGN KEY (config_id) REFERENCES configs(id)
);
-- Uniqueness of (user_id, full_name) for repositories is enforced via drizzle migrations
CREATE TABLE IF NOT EXISTS organizations (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,

View File

@@ -195,6 +195,7 @@ Configure automatic scheduled mirroring.
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `AUTO_IMPORT_REPOS` | Automatically discover and import new GitHub repositories during scheduled syncs | `true` | `true`, `false` |
| `AUTO_MIRROR_REPOS` | Automatically mirror newly imported repositories during scheduled syncs (no manual “Mirror All” required) | `false` | `true`, `false` |
| `SCHEDULE_ONLY_MIRROR_UPDATED` | Only mirror repos with updates | `false` | `true`, `false` |
| `SCHEDULE_UPDATE_INTERVAL` | Check for updates interval (milliseconds) | `86400000` | Number |
| `SCHEDULE_SKIP_RECENTLY_MIRRORED` | Skip recently mirrored repos | `true` | `true`, `false` |
@@ -407,4 +408,4 @@ BETTER_AUTH_TRUSTED_ORIGINS=http://localhost:3000,http://192.168.1.100:3000
- `admin:org` (read organization data)
- Additional scopes may be required for specific features
For more examples and detailed configuration, see the `.env.example` file in the repository.
For more examples and detailed configuration, see the `.env.example` file in the repository.

View File

@@ -29,6 +29,8 @@ This guide explains how to test SSO authentication locally with Gitea Mirror.
- Client Secret: (from Google Console)
- Save the provider
Note: Provider creation uses Better Auth's SSO registration under the hood. Do not call the legacy `POST /api/sso/providers` endpoint directly; it is deprecated and reserved for internal mirroring. Use the UI or Better Auth client/server registration APIs instead.
## Option 2: Using Keycloak (Local Identity Provider)
### Setup with Docker:
@@ -113,8 +115,8 @@ npm start
2. **Provider not showing in login**
- Check browser console for errors
- Verify provider was saved successfully
- Check `/api/sso/providers` returns your providers
- Verify provider was saved successfully (via UI)
- Check `/api/sso/providers` (or `/api/sso/providers/public`) returns your providers. This list mirrors what was registered with Better Auth.
3. **Redirect URI mismatch**
- Ensure the redirect URI in your OAuth app matches exactly:
@@ -190,4 +192,4 @@ After successful SSO setup:
1. Test user attribute mapping
2. Configure role-based access
3. Set up SAML if needed
4. Test with your organization's actual IdP
4. Test with your organization's actual IdP

31756
docs/better-auth-docs.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1 @@
CREATE UNIQUE INDEX `uniq_repositories_user_full_name` ON `repositories` (`user_id`,`full_name`);

View File

@@ -0,0 +1 @@
ALTER TABLE `sso_providers` ADD `saml_config` text;

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -36,6 +36,20 @@
"when": 1757392620734,
"tag": "0004_grey_butterfly",
"breakpoints": true
},
{
"idx": 5,
"version": "6",
"when": 1757786449446,
"tag": "0005_polite_preak",
"breakpoints": true
},
{
"idx": 6,
"version": "6",
"when": 1757825311459,
"tag": "0006_illegal_spyke",
"breakpoints": true
}
]
}

View File

@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.6.0",
"version": "3.7.1",
"engines": {
"bun": ">=1.2.9"
},
@@ -46,7 +46,7 @@
"@astrojs/mdx": "4.3.5",
"@astrojs/node": "9.4.3",
"@astrojs/react": "^4.3.1",
"@better-auth/sso": "^1.3.8",
"@better-auth/sso": "^1.3.9",
"@octokit/plugin-throttling": "^11.0.1",
"@octokit/rest": "^22.0.0",
"@radix-ui/react-accordion": "^1.2.12",
@@ -72,7 +72,7 @@
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.1.12",
"@types/react-dom": "^19.1.9",
"astro": "^5.13.6",
"astro": "^5.13.7",
"bcryptjs": "^3.0.2",
"better-auth": "^1.3.9",
"canvas-confetti": "^1.9.3",

View File

@@ -70,6 +70,8 @@ export function LoginForm() {
domain: domain,
providerId: providerId,
callbackURL: `${baseURL}/`,
errorCallbackURL: `${baseURL}/auth-error`,
newUserCallbackURL: `${baseURL}/`,
scopes: ['openid', 'email', 'profile'], // TODO: This is not being respected by the SSO plugin.
});
} catch (error) {

View File

@@ -15,11 +15,11 @@ import {
PopoverContent,
PopoverTrigger,
} from "@/components/ui/popover";
import {
Info,
GitBranch,
Star,
Lock,
import {
Info,
GitBranch,
Star,
Lock,
Archive,
GitPullRequest,
Tag,
@@ -30,9 +30,17 @@ import {
GitFork,
ChevronDown,
Funnel,
HardDrive
HardDrive,
FileCode2
} from "lucide-react";
import type { GitHubConfig, MirrorOptions, AdvancedOptions } from "@/types/config";
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy } from "@/types/config";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { cn } from "@/lib/utils";
interface GitHubMirrorSettingsProps {
@@ -53,7 +61,7 @@ export function GitHubMirrorSettings({
onAdvancedOptionsChange,
}: GitHubMirrorSettingsProps) {
const handleGitHubChange = (field: keyof GitHubConfig, value: boolean) => {
const handleGitHubChange = (field: keyof GitHubConfig, value: boolean | string) => {
onGitHubConfigChange({ ...githubConfig, [field]: value });
};
@@ -278,6 +286,40 @@ export function GitHubMirrorSettings({
</Popover>
</div>
</div>
{/* Duplicate name handling for starred repos */}
{githubConfig.mirrorStarred && (
<div className="mt-4 space-y-2">
<Label className="text-xs font-medium text-muted-foreground">
Duplicate name handling
</Label>
<div className="flex items-center gap-3">
<FileCode2 className="h-4 w-4 text-muted-foreground" />
<div className="flex-1">
<p className="text-sm">Name collision strategy</p>
<p className="text-xs text-muted-foreground">
How to handle repos with the same name from different owners
</p>
</div>
<Select
value={githubConfig.starredDuplicateStrategy || "suffix"}
onValueChange={(value) => handleGitHubChange('starredDuplicateStrategy', value as DuplicateNameStrategy)}
>
<SelectTrigger className="w-[180px] h-8 text-xs">
<SelectValue placeholder="Select strategy" />
</SelectTrigger>
<SelectContent align="end">
<SelectItem value="suffix" className="text-xs">
<span className="font-mono">repo-owner</span>
</SelectItem>
<SelectItem value="prefix" className="text-xs">
<span className="font-mono">owner-repo</span>
</SelectItem>
</SelectContent>
</Select>
</div>
</div>
)}
</div>
</div>
@@ -596,4 +638,4 @@ export function GitHubMirrorSettings({
</div>
</div>
);
}
}

View File

@@ -14,6 +14,7 @@ import { Badge } from '../ui/badge';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Textarea } from '@/components/ui/textarea';
import { MultiSelect } from '@/components/ui/multi-select';
import { authClient } from '@/lib/auth-client';
function isTrustedIssuer(issuer: string, allowedHosts: string[]): boolean {
try {
@@ -158,50 +159,146 @@ export function SSOSettings() {
const createProvider = async () => {
setAddingProvider(true);
try {
const requestData: any = {
providerId: providerForm.providerId,
issuer: providerForm.issuer,
domain: providerForm.domain,
organizationId: providerForm.organizationId || undefined,
providerType,
};
if (providerType === 'oidc') {
requestData.clientId = providerForm.clientId;
requestData.clientSecret = providerForm.clientSecret;
requestData.authorizationEndpoint = providerForm.authorizationEndpoint;
requestData.tokenEndpoint = providerForm.tokenEndpoint;
requestData.jwksEndpoint = providerForm.jwksEndpoint;
requestData.userInfoEndpoint = providerForm.userInfoEndpoint;
requestData.discoveryEndpoint = providerForm.discoveryEndpoint;
requestData.scopes = providerForm.scopes;
requestData.pkce = providerForm.pkce;
} else {
requestData.entryPoint = providerForm.entryPoint;
requestData.cert = providerForm.cert;
requestData.callbackUrl = providerForm.callbackUrl || `${window.location.origin}/api/auth/sso/saml2/callback/${providerForm.providerId}`;
requestData.audience = providerForm.audience || window.location.origin;
requestData.wantAssertionsSigned = providerForm.wantAssertionsSigned;
requestData.signatureAlgorithm = providerForm.signatureAlgorithm;
requestData.digestAlgorithm = providerForm.digestAlgorithm;
requestData.identifierFormat = providerForm.identifierFormat;
}
if (editingProvider) {
// Update existing provider
const updatedProvider = await apiRequest<SSOProvider>(`/sso/providers?id=${editingProvider.id}`, {
method: 'PUT',
data: requestData,
});
setProviders(providers.map(p => p.id === editingProvider.id ? updatedProvider : p));
toast.success('SSO provider updated successfully');
// Delete and recreate to align with Better Auth docs
try {
await apiRequest(`/sso/providers?id=${editingProvider.id}`, { method: 'DELETE' });
} catch (e) {
// Continue even if local delete fails; registration will mirror latest
console.warn('Failed to delete local provider before recreate', e);
}
// Recreate via Better Auth registration
try {
if (providerType === 'oidc') {
await authClient.sso.register({
providerId: providerForm.providerId,
issuer: providerForm.issuer,
domain: providerForm.domain,
organizationId: providerForm.organizationId || undefined,
oidcConfig: {
clientId: providerForm.clientId || undefined,
clientSecret: providerForm.clientSecret || undefined,
authorizationEndpoint: providerForm.authorizationEndpoint || undefined,
tokenEndpoint: providerForm.tokenEndpoint || undefined,
jwksEndpoint: providerForm.jwksEndpoint || undefined,
userInfoEndpoint: providerForm.userInfoEndpoint || undefined,
discoveryEndpoint: providerForm.discoveryEndpoint || undefined,
scopes: providerForm.scopes,
pkce: providerForm.pkce,
},
mapping: {
id: 'sub',
email: 'email',
emailVerified: 'email_verified',
name: 'name',
image: 'picture',
},
} as any);
} else {
await authClient.sso.register({
providerId: providerForm.providerId,
issuer: providerForm.issuer,
domain: providerForm.domain,
organizationId: providerForm.organizationId || undefined,
samlConfig: {
entryPoint: providerForm.entryPoint,
cert: providerForm.cert,
callbackUrl:
providerForm.callbackUrl ||
`${window.location.origin}/api/auth/sso/saml2/callback/${providerForm.providerId}`,
audience: providerForm.audience || window.location.origin,
wantAssertionsSigned: providerForm.wantAssertionsSigned,
signatureAlgorithm: providerForm.signatureAlgorithm,
digestAlgorithm: providerForm.digestAlgorithm,
identifierFormat: providerForm.identifierFormat,
},
mapping: {
id: 'nameID',
email: 'email',
name: 'displayName',
firstName: 'givenName',
lastName: 'surname',
},
} as any);
}
toast.success('SSO provider recreated');
} catch (e: any) {
console.error('Recreate failed', e);
const msg = typeof e?.message === 'string' ? e.message : String(e);
// Common case: providerId already exists in Better Auth
if (msg.toLowerCase().includes('already exists')) {
toast.error('Provider ID already exists in auth server. Choose a new Provider ID and try again.');
} else {
showErrorToast(e, toast);
}
}
// Refresh providers from our API after registration mirrors into DB
const refreshed = await apiRequest<SSOProvider[] | { providers: SSOProvider[] }>(
'/sso/providers'
);
setProviders(Array.isArray(refreshed) ? refreshed : refreshed?.providers || []);
} else {
// Create new provider
const newProvider = await apiRequest<SSOProvider>('/sso/providers', {
method: 'POST',
data: requestData,
});
setProviders([...providers, newProvider]);
// Create new provider - follow Better Auth docs using the SSO client
if (providerType === 'oidc') {
await authClient.sso.register({
providerId: providerForm.providerId,
issuer: providerForm.issuer,
domain: providerForm.domain,
organizationId: providerForm.organizationId || undefined,
oidcConfig: {
clientId: providerForm.clientId || undefined,
clientSecret: providerForm.clientSecret || undefined,
authorizationEndpoint: providerForm.authorizationEndpoint || undefined,
tokenEndpoint: providerForm.tokenEndpoint || undefined,
jwksEndpoint: providerForm.jwksEndpoint || undefined,
userInfoEndpoint: providerForm.userInfoEndpoint || undefined,
discoveryEndpoint: providerForm.discoveryEndpoint || undefined,
scopes: providerForm.scopes,
pkce: providerForm.pkce,
},
mapping: {
id: 'sub',
email: 'email',
emailVerified: 'email_verified',
name: 'name',
image: 'picture',
},
} as any);
} else {
await authClient.sso.register({
providerId: providerForm.providerId,
issuer: providerForm.issuer,
domain: providerForm.domain,
organizationId: providerForm.organizationId || undefined,
samlConfig: {
entryPoint: providerForm.entryPoint,
cert: providerForm.cert,
callbackUrl:
providerForm.callbackUrl ||
`${window.location.origin}/api/auth/sso/saml2/callback/${providerForm.providerId}`,
audience: providerForm.audience || window.location.origin,
wantAssertionsSigned: providerForm.wantAssertionsSigned,
signatureAlgorithm: providerForm.signatureAlgorithm,
digestAlgorithm: providerForm.digestAlgorithm,
identifierFormat: providerForm.identifierFormat,
},
mapping: {
id: 'nameID',
email: 'email',
name: 'displayName',
firstName: 'givenName',
lastName: 'surname',
},
} as any);
}
// Refresh providers from our API after registration mirrors into DB
const refreshed = await apiRequest<SSOProvider[] | { providers: SSOProvider[] }>(
'/sso/providers'
);
setProviders(Array.isArray(refreshed) ? refreshed : refreshed?.providers || []);
toast.success('SSO provider created successfully');
}
@@ -724,4 +821,4 @@ export function SSOSettings() {
</Card>
</div>
);
}
}

View File

@@ -1,5 +1,5 @@
import { z } from "zod";
import { sqliteTable, text, integer, index } from "drizzle-orm/sqlite-core";
import { sqliteTable, text, integer, index, uniqueIndex } from "drizzle-orm/sqlite-core";
import { sql } from "drizzle-orm";
// ===== Zod Validation Schemas =====
@@ -28,6 +28,7 @@ export const githubConfigSchema = z.object({
mirrorStrategy: z.enum(["preserve", "single-org", "flat-user", "mixed"]).default("preserve"),
defaultOrg: z.string().optional(),
skipStarredIssues: z.boolean().default(false),
starredDuplicateStrategy: z.enum(["suffix", "prefix", "owner-org"]).default("suffix").optional(),
});
export const giteaConfigSchema = z.object({
@@ -80,6 +81,8 @@ export const scheduleConfigSchema = z.object({
updateInterval: z.number().default(86400000),
skipRecentlyMirrored: z.boolean().default(true),
recentThreshold: z.number().default(3600000),
autoImport: z.boolean().default(true),
autoMirror: z.boolean().default(false),
lastRun: z.coerce.date().optional(),
nextRun: z.coerce.date().optional(),
});
@@ -151,6 +154,7 @@ export const repositorySchema = z.object({
"deleted",
"syncing",
"synced",
"archived",
])
.default("imported"),
lastMirrored: z.coerce.date().optional().nullable(),
@@ -180,6 +184,7 @@ export const mirrorJobSchema = z.object({
"deleted",
"syncing",
"synced",
"archived",
])
.default("imported"),
message: z.string(),
@@ -379,6 +384,7 @@ export const repositories = sqliteTable("repositories", {
index("idx_repositories_organization").on(table.organization),
index("idx_repositories_is_fork").on(table.isForked),
index("idx_repositories_is_starred").on(table.isStarred),
uniqueIndex("uniq_repositories_user_full_name").on(table.userId, table.fullName),
]);
export const mirrorJobs = sqliteTable("mirror_jobs", {
@@ -611,6 +617,7 @@ export const ssoProviders = sqliteTable("sso_providers", {
issuer: text("issuer").notNull(),
domain: text("domain").notNull(),
oidcConfig: text("oidc_config").notNull(), // JSON string with OIDC configuration
samlConfig: text("saml_config"), // JSON string with SAML configuration (optional)
userId: text("user_id").notNull(), // Admin who created this provider
providerId: text("provider_id").notNull().unique(), // Unique identifier for the provider
organizationId: text("organization_id"), // Optional - if provider is linked to an organization
@@ -674,4 +681,4 @@ export type Repository = z.infer<typeof repositorySchema>;
export type MirrorJob = z.infer<typeof mirrorJobSchema>;
export type Organization = z.infer<typeof organizationSchema>;
export type Event = z.infer<typeof eventSchema>;
export type RateLimit = z.infer<typeof rateLimitSchema>;
export type RateLimit = z.infer<typeof rateLimitSchema>;

View File

@@ -69,6 +69,8 @@ interface EnvConfig {
updateInterval?: number;
skipRecentlyMirrored?: boolean;
recentThreshold?: number;
autoImport?: boolean;
autoMirror?: boolean;
};
cleanup: {
enabled?: boolean;
@@ -157,6 +159,8 @@ function parseEnvConfig(): EnvConfig {
updateInterval: process.env.SCHEDULE_UPDATE_INTERVAL ? parseInt(process.env.SCHEDULE_UPDATE_INTERVAL, 10) : undefined,
skipRecentlyMirrored: process.env.SCHEDULE_SKIP_RECENTLY_MIRRORED === 'true',
recentThreshold: process.env.SCHEDULE_RECENT_THRESHOLD ? parseInt(process.env.SCHEDULE_RECENT_THRESHOLD, 10) : undefined,
autoImport: process.env.AUTO_IMPORT_REPOS !== 'false',
autoMirror: process.env.AUTO_MIRROR_REPOS === 'true',
},
cleanup: {
enabled: process.env.CLEANUP_ENABLED === 'true' ||
@@ -301,7 +305,8 @@ export async function initializeConfigFromEnv(): Promise<void> {
updateInterval: envConfig.schedule.updateInterval ?? existingConfig?.[0]?.scheduleConfig?.updateInterval ?? 86400000,
skipRecentlyMirrored: envConfig.schedule.skipRecentlyMirrored ?? existingConfig?.[0]?.scheduleConfig?.skipRecentlyMirrored ?? true,
recentThreshold: envConfig.schedule.recentThreshold ?? existingConfig?.[0]?.scheduleConfig?.recentThreshold ?? 3600000,
autoImport: process.env.AUTO_IMPORT_REPOS !== 'false', // New field for auto-importing new repositories
autoImport: envConfig.schedule.autoImport ?? existingConfig?.[0]?.scheduleConfig?.autoImport ?? true,
autoMirror: envConfig.schedule.autoMirror ?? existingConfig?.[0]?.scheduleConfig?.autoMirror ?? false,
lastRun: existingConfig?.[0]?.scheduleConfig?.lastRun || undefined,
nextRun: existingConfig?.[0]?.scheduleConfig?.nextRun || undefined,
};
@@ -359,4 +364,4 @@ export async function initializeConfigFromEnv(): Promise<void> {
console.error('[ENV Config Loader] Failed to initialize configuration from environment:', error);
// Don't throw - this is a non-critical initialization
}
}
}

View File

@@ -274,15 +274,37 @@ export const mirrorGithubRepoToGitea = async ({
// Get the correct owner based on the strategy (with organization overrides)
let repoOwner = await getGiteaRepoOwnerAsync({ config, repository });
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
if (repository.isStarred && config.githubConfig) {
// Extract GitHub owner from full_name (format: owner/repo)
const githubOwner = repository.fullName.split('/')[0];
targetRepoName = await generateUniqueRepoName({
config,
orgName: repoOwner,
baseName: repository.name,
githubOwner,
strategy: config.githubConfig.starredDuplicateStrategy,
});
if (targetRepoName !== repository.name) {
console.log(
`Starred repo ${repository.fullName} will be mirrored as ${repoOwner}/${targetRepoName} to avoid naming conflict`
);
}
}
const isExisting = await isRepoPresentInGitea({
config,
owner: repoOwner,
repoName: repository.name,
repoName: targetRepoName,
});
if (isExisting) {
console.log(
`Repository ${repository.name} already exists in Gitea under ${repoOwner}. Updating database status.`
`Repository ${targetRepoName} already exists in Gitea under ${repoOwner}. Updating database status.`
);
// Update database to reflect that the repository is already mirrored
@@ -293,7 +315,7 @@ export const mirrorGithubRepoToGitea = async ({
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${repository.name}`,
mirroredLocation: `${repoOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
@@ -393,11 +415,11 @@ export const mirrorGithubRepoToGitea = async ({
const existingRepo = await getGiteaRepoInfo({
config,
owner: repoOwner,
repoName: repository.name,
repoName: targetRepoName,
});
if (existingRepo && !existingRepo.mirror) {
console.log(`Repository ${repository.name} exists but is not a mirror. Handling...`);
console.log(`Repository ${targetRepoName} exists but is not a mirror. Handling...`);
// Handle the existing non-mirror repository
await handleExistingNonMirrorRepo({
@@ -408,14 +430,14 @@ export const mirrorGithubRepoToGitea = async ({
});
// After handling, proceed with mirror creation
console.log(`Proceeding with mirror creation for ${repository.name}`);
console.log(`Proceeding with mirror creation for ${targetRepoName}`);
}
const response = await httpPost(
apiUrl,
{
clone_addr: cloneAddress,
repo_name: repository.name,
repo_name: targetRepoName,
mirror: true,
mirror_interval: config.giteaConfig?.mirrorInterval || "8h", // Set mirror interval
wiki: config.giteaConfig?.wiki || false, // will mirror wiki if it exists
@@ -438,6 +460,8 @@ export const mirrorGithubRepoToGitea = async ({
config,
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored releases for ${repository.name}`);
} catch (error) {
@@ -460,6 +484,7 @@ export const mirrorGithubRepoToGitea = async ({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored issues for ${repository.name}`);
} catch (error) {
@@ -477,6 +502,7 @@ export const mirrorGithubRepoToGitea = async ({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored pull requests for ${repository.name}`);
} catch (error) {
@@ -494,6 +520,7 @@ export const mirrorGithubRepoToGitea = async ({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored labels for ${repository.name}`);
} catch (error) {
@@ -511,6 +538,7 @@ export const mirrorGithubRepoToGitea = async ({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored milestones for ${repository.name}`);
} catch (error) {
@@ -519,7 +547,7 @@ export const mirrorGithubRepoToGitea = async ({
}
}
console.log(`Repository ${repository.name} mirrored successfully`);
console.log(`Repository ${repository.name} mirrored successfully as ${targetRepoName}`);
// Mark repos as "mirrored" in DB
await db
@@ -529,7 +557,7 @@ export const mirrorGithubRepoToGitea = async ({
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${repository.name}`,
mirroredLocation: `${repoOwner}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
@@ -538,8 +566,8 @@ export const mirrorGithubRepoToGitea = async ({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Successfully mirrored repository: ${repository.name}`,
details: `Repository ${repository.name} was mirrored to Gitea.`,
message: `Successfully mirrored repository: ${repository.name}${targetRepoName !== repository.name ? ` as ${targetRepoName}` : ''}`,
details: `Repository ${repository.fullName} was mirrored to Gitea at ${repoOwner}/${targetRepoName}.`,
status: "mirrored",
});
@@ -608,6 +636,80 @@ export async function getOrCreateGiteaOrg({
}
}
/**
* Generate a unique repository name for starred repos with duplicate names
*/
async function generateUniqueRepoName({
config,
orgName,
baseName,
githubOwner,
strategy,
}: {
config: Partial<Config>;
orgName: string;
baseName: string;
githubOwner: string;
strategy?: string;
}): Promise<string> {
const duplicateStrategy = strategy || "suffix";
// First check if base name is available
const baseExists = await isRepoPresentInGitea({
config,
owner: orgName,
repoName: baseName,
});
if (!baseExists) {
return baseName;
}
// Generate name based on strategy
let candidateName: string;
let attempt = 0;
const maxAttempts = 10;
while (attempt < maxAttempts) {
switch (duplicateStrategy) {
case "prefix":
// Prefix with owner: owner-reponame
candidateName = attempt === 0
? `${githubOwner}-${baseName}`
: `${githubOwner}-${baseName}-${attempt}`;
break;
case "owner-org":
// This would require creating sub-organizations, not supported in this PR
// Fall back to suffix strategy
case "suffix":
default:
// Suffix with owner: reponame-owner
candidateName = attempt === 0
? `${baseName}-${githubOwner}`
: `${baseName}-${githubOwner}-${attempt}`;
break;
}
const exists = await isRepoPresentInGitea({
config,
owner: orgName,
repoName: candidateName,
});
if (!exists) {
console.log(`Found unique name for duplicate starred repo: ${candidateName}`);
return candidateName;
}
attempt++;
}
// If all attempts failed, use timestamp as last resort
const timestamp = Date.now();
return `${baseName}-${githubOwner}-${timestamp}`;
}
export async function mirrorGitHubRepoToGiteaOrg({
octokit,
config,
@@ -633,15 +735,37 @@ export async function mirrorGitHubRepoToGiteaOrg({
// Decrypt config tokens for API usage
const decryptedConfig = decryptConfigTokens(config as Config);
// Determine the actual repository name to use (handle duplicates for starred repos)
let targetRepoName = repository.name;
if (repository.isStarred && config.githubConfig) {
// Extract GitHub owner from full_name (format: owner/repo)
const githubOwner = repository.fullName.split('/')[0];
targetRepoName = await generateUniqueRepoName({
config,
orgName,
baseName: repository.name,
githubOwner,
strategy: config.githubConfig.starredDuplicateStrategy,
});
if (targetRepoName !== repository.name) {
console.log(
`Starred repo ${repository.fullName} will be mirrored as ${orgName}/${targetRepoName} to avoid naming conflict`
);
}
}
const isExisting = await isRepoPresentInGitea({
config,
owner: orgName,
repoName: repository.name,
repoName: targetRepoName,
});
if (isExisting) {
console.log(
`Repository ${repository.name} already exists in Gitea organization ${orgName}. Updating database status.`
`Repository ${targetRepoName} already exists in Gitea organization ${orgName}. Updating database status.`
);
// Update database to reflect that the repository is already mirrored
@@ -652,7 +776,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${orgName}/${repository.name}`,
mirroredLocation: `${orgName}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
@@ -661,19 +785,19 @@ export async function mirrorGitHubRepoToGiteaOrg({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea organization ${orgName}`,
details: `Repository ${repository.name} was found to already exist in Gitea organization ${orgName} and database status was updated.`,
message: `Repository ${targetRepoName} already exists in Gitea organization ${orgName}`,
details: `Repository ${targetRepoName} was found to already exist in Gitea organization ${orgName} and database status was updated.`,
status: "mirrored",
});
console.log(
`Repository ${repository.name} database status updated to mirrored in organization ${orgName}`
`Repository ${targetRepoName} database status updated to mirrored in organization ${orgName}`
);
return;
}
console.log(
`Mirroring repository ${repository.name} to organization ${orgName}`
`Mirroring repository ${repository.fullName} to organization ${orgName} as ${targetRepoName}`
);
let cloneAddress = repository.cloneUrl;
@@ -710,7 +834,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
{
clone_addr: cloneAddress,
uid: giteaOrgId,
repo_name: repository.name,
repo_name: targetRepoName,
mirror: true,
mirror_interval: config.giteaConfig?.mirrorInterval || "8h", // Set mirror interval
wiki: config.giteaConfig?.wiki || false, // will mirror wiki if it exists
@@ -730,6 +854,8 @@ export async function mirrorGitHubRepoToGiteaOrg({
config,
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored releases for ${repository.name}`);
} catch (error) {
@@ -752,10 +878,11 @@ export async function mirrorGitHubRepoToGiteaOrg({
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored issues for ${repository.name} to org ${orgName}`);
console.log(`[Metadata] Successfully mirrored issues for ${repository.name} to org ${orgName}/${targetRepoName}`);
} catch (error) {
console.error(`[Metadata] Failed to mirror issues for ${repository.name} to org ${orgName}: ${error instanceof Error ? error.message : String(error)}`);
console.error(`[Metadata] Failed to mirror issues for ${repository.name} to org ${orgName}/${targetRepoName}: ${error instanceof Error ? error.message : String(error)}`);
// Continue with other metadata operations even if issues fail
}
}
@@ -769,10 +896,11 @@ export async function mirrorGitHubRepoToGiteaOrg({
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored pull requests for ${repository.name} to org ${orgName}`);
console.log(`[Metadata] Successfully mirrored pull requests for ${repository.name} to org ${orgName}/${targetRepoName}`);
} catch (error) {
console.error(`[Metadata] Failed to mirror pull requests for ${repository.name} to org ${orgName}: ${error instanceof Error ? error.message : String(error)}`);
console.error(`[Metadata] Failed to mirror pull requests for ${repository.name} to org ${orgName}/${targetRepoName}: ${error instanceof Error ? error.message : String(error)}`);
// Continue with other metadata operations even if PRs fail
}
}
@@ -786,10 +914,11 @@ export async function mirrorGitHubRepoToGiteaOrg({
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored labels for ${repository.name} to org ${orgName}`);
console.log(`[Metadata] Successfully mirrored labels for ${repository.name} to org ${orgName}/${targetRepoName}`);
} catch (error) {
console.error(`[Metadata] Failed to mirror labels for ${repository.name} to org ${orgName}: ${error instanceof Error ? error.message : String(error)}`);
console.error(`[Metadata] Failed to mirror labels for ${repository.name} to org ${orgName}/${targetRepoName}: ${error instanceof Error ? error.message : String(error)}`);
// Continue with other metadata operations even if labels fail
}
}
@@ -803,16 +932,17 @@ export async function mirrorGitHubRepoToGiteaOrg({
octokit,
repository,
giteaOwner: orgName,
giteaRepoName: targetRepoName,
});
console.log(`[Metadata] Successfully mirrored milestones for ${repository.name} to org ${orgName}`);
console.log(`[Metadata] Successfully mirrored milestones for ${repository.name} to org ${orgName}/${targetRepoName}`);
} catch (error) {
console.error(`[Metadata] Failed to mirror milestones for ${repository.name} to org ${orgName}: ${error instanceof Error ? error.message : String(error)}`);
console.error(`[Metadata] Failed to mirror milestones for ${repository.name} to org ${orgName}/${targetRepoName}: ${error instanceof Error ? error.message : String(error)}`);
// Continue with other metadata operations even if milestones fail
}
}
console.log(
`Repository ${repository.name} mirrored successfully to organization ${orgName}`
`Repository ${repository.name} mirrored successfully to organization ${orgName} as ${targetRepoName}`
);
// Mark repos as "mirrored" in DB
@@ -823,7 +953,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${orgName}/${repository.name}`,
mirroredLocation: `${orgName}/${targetRepoName}`,
})
.where(eq(repositories.id, repository.id!));
@@ -832,8 +962,8 @@ export async function mirrorGitHubRepoToGiteaOrg({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} mirrored successfully`,
details: `Repository ${repository.name} was mirrored to Gitea`,
message: `Repository ${repository.name} mirrored successfully${targetRepoName !== repository.name ? ` as ${targetRepoName}` : ''}`,
details: `Repository ${repository.fullName} was mirrored to Gitea at ${orgName}/${targetRepoName}`,
status: "mirrored",
});
@@ -1149,11 +1279,13 @@ export const mirrorGitRepoIssuesToGitea = async ({
octokit,
repository,
giteaOwner,
giteaRepoName,
}: {
config: Partial<Config>;
octokit: Octokit;
repository: Repository;
giteaOwner: string;
giteaRepoName?: string;
}) => {
//things covered here are- issue, title, body, labels, comments and assignees
if (
@@ -1168,23 +1300,26 @@ export const mirrorGitRepoIssuesToGitea = async ({
// Decrypt config tokens for API usage
const decryptedConfig = decryptConfigTokens(config as Config);
// Use provided giteaRepoName or fall back to repository.name
const repoName = giteaRepoName || repository.name;
// Log configuration details for debugging
console.log(`[Issues] Starting issue mirroring for repository ${repository.name}`);
console.log(`[Issues] Starting issue mirroring for repository ${repository.name} as ${repoName}`);
console.log(`[Issues] Gitea URL: ${config.giteaConfig!.url}`);
console.log(`[Issues] Gitea Owner: ${giteaOwner}`);
console.log(`[Issues] Gitea Default Owner: ${config.giteaConfig!.defaultOwner}`);
// Verify the repository exists in Gitea before attempting to mirror metadata
console.log(`[Issues] Verifying repository ${repository.name} exists at ${giteaOwner}`);
console.log(`[Issues] Verifying repository ${repoName} exists at ${giteaOwner}`);
const repoExists = await isRepoPresentInGitea({
config,
owner: giteaOwner,
repoName: repository.name,
repoName: repoName,
});
if (!repoExists) {
console.error(`[Issues] Repository ${repository.name} not found at ${giteaOwner}. Cannot mirror issues.`);
throw new Error(`Repository ${repository.name} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
console.error(`[Issues] Repository ${repoName} not found at ${giteaOwner}. Cannot mirror issues.`);
throw new Error(`Repository ${repoName} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
}
const [owner, repo] = repository.fullName.split("/");
@@ -1215,7 +1350,7 @@ export const mirrorGitRepoIssuesToGitea = async ({
// Get existing labels from Gitea
const giteaLabelsRes = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/labels`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/labels`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
@@ -1247,9 +1382,7 @@ export const mirrorGitRepoIssuesToGitea = async ({
} else {
try {
const created = await httpPost(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${
repository.name
}/labels`,
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/labels`,
{ name, color: "#ededed" }, // Default color
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
@@ -1284,9 +1417,7 @@ export const mirrorGitRepoIssuesToGitea = async ({
// Create the issue in Gitea
const createdIssue = await httpPost(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${
repository.name
}/issues`,
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues`,
issuePayload,
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
@@ -1311,9 +1442,7 @@ export const mirrorGitRepoIssuesToGitea = async ({
comments,
async (comment) => {
await httpPost(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${
repository.name
}/issues/${createdIssue.data.number}/comments`,
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues/${createdIssue.data.number}/comments`,
{
body: `@${comment.user?.login} commented on GitHub:\n\n${comment.body}`,
},
@@ -1367,10 +1496,14 @@ export async function mirrorGitHubReleasesToGitea({
octokit,
repository,
config,
giteaOwner,
giteaRepoName,
}: {
octokit: Octokit;
repository: Repository;
config: Partial<Config>;
giteaOwner?: string;
giteaRepoName?: string;
}) {
if (
!config.giteaConfig?.defaultOwner ||
@@ -1383,17 +1516,16 @@ export async function mirrorGitHubReleasesToGitea({
// Decrypt config tokens for API usage
const decryptedConfig = decryptConfigTokens(config as Config);
const repoOwner = await getGiteaRepoOwnerAsync({
config,
repository,
});
// Determine target owner/repo in Gitea (supports renamed repos)
const repoOwner = giteaOwner || (await getGiteaRepoOwnerAsync({ config, repository }));
const repoName = giteaRepoName || repository.name;
// Verify the repository exists in Gitea before attempting to mirror releases
console.log(`[Releases] Verifying repository ${repository.name} exists at ${repoOwner}`);
console.log(`[Releases] Verifying repository ${repoName} exists at ${repoOwner}`);
const repoExists = await isRepoPresentInGitea({
config,
owner: repoOwner,
repoName: repository.name,
repoName: repoName,
});
if (!repoExists) {
@@ -1429,7 +1561,7 @@ export async function mirrorGitHubReleasesToGitea({
try {
// Check if release already exists
const existingReleasesResponse = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repository.name}/releases/tags/${release.tag_name}`,
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/tags/${release.tag_name}`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
@@ -1446,7 +1578,7 @@ export async function mirrorGitHubReleasesToGitea({
console.log(`[Releases] Updating existing release ${release.tag_name} with new changelog/title`);
await httpPut(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repository.name}/releases/${existingRelease.id}`,
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${existingRelease.id}`,
{
tag_name: release.tag_name,
target: release.target_commitish,
@@ -1477,7 +1609,7 @@ export async function mirrorGitHubReleasesToGitea({
}
const createReleaseResponse = await httpPost(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repository.name}/releases`,
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases`,
{
tag_name: release.tag_name,
target: release.target_commitish,
@@ -1518,7 +1650,7 @@ export async function mirrorGitHubReleasesToGitea({
formData.append('attachment', new Blob([assetData]), asset.name);
const uploadResponse = await fetch(
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repository.name}/releases/${createReleaseResponse.data.id}/assets?name=${encodeURIComponent(asset.name)}`,
`${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/releases/${createReleaseResponse.data.id}/assets?name=${encodeURIComponent(asset.name)}`,
{
method: 'POST',
headers: {
@@ -1556,11 +1688,13 @@ export async function mirrorGitRepoPullRequestsToGitea({
octokit,
repository,
giteaOwner,
giteaRepoName,
}: {
config: Partial<Config>;
octokit: Octokit;
repository: Repository;
giteaOwner: string;
giteaRepoName?: string;
}) {
if (
!config.githubConfig?.token ||
@@ -1574,23 +1708,26 @@ export async function mirrorGitRepoPullRequestsToGitea({
// Decrypt config tokens for API usage
const decryptedConfig = decryptConfigTokens(config as Config);
// Use provided giteaRepoName or fall back to repository.name
const repoName = giteaRepoName || repository.name;
// Log configuration details for debugging
console.log(`[Pull Requests] Starting PR mirroring for repository ${repository.name}`);
console.log(`[Pull Requests] Starting PR mirroring for repository ${repository.name} as ${repoName}`);
console.log(`[Pull Requests] Gitea URL: ${config.giteaConfig!.url}`);
console.log(`[Pull Requests] Gitea Owner: ${giteaOwner}`);
console.log(`[Pull Requests] Gitea Default Owner: ${config.giteaConfig!.defaultOwner}`);
// Verify the repository exists in Gitea before attempting to mirror metadata
console.log(`[Pull Requests] Verifying repository ${repository.name} exists at ${giteaOwner}`);
console.log(`[Pull Requests] Verifying repository ${repoName} exists at ${giteaOwner}`);
const repoExists = await isRepoPresentInGitea({
config,
owner: giteaOwner,
repoName: repository.name,
repoName: repoName,
});
if (!repoExists) {
console.error(`[Pull Requests] Repository ${repository.name} not found at ${giteaOwner}. Cannot mirror PRs.`);
throw new Error(`Repository ${repository.name} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
console.error(`[Pull Requests] Repository ${repoName} not found at ${giteaOwner}. Cannot mirror PRs.`);
throw new Error(`Repository ${repoName} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
}
const [owner, repo] = repository.fullName.split("/");
@@ -1622,7 +1759,7 @@ export async function mirrorGitRepoPullRequestsToGitea({
// Get existing labels from Gitea and ensure "pull-request" label exists
const giteaLabelsRes = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/labels`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/labels`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
@@ -1640,7 +1777,7 @@ export async function mirrorGitRepoPullRequestsToGitea({
} else {
try {
const created = await httpPost(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/labels`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/labels`,
{
name: "pull-request",
color: "#0366d6",
@@ -1744,7 +1881,7 @@ export async function mirrorGitRepoPullRequestsToGitea({
console.log(`[Pull Requests] Creating enriched issue for PR #${pr.number}: ${pr.title}`);
await httpPost(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repository.name}/issues`,
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues`,
issueData,
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
@@ -1764,7 +1901,7 @@ export async function mirrorGitRepoPullRequestsToGitea({
try {
await httpPost(
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repository.name}/issues`,
`${config.giteaConfig!.url}/api/v1/repos/${giteaOwner}/${repoName}/issues`,
basicIssueData,
{
Authorization: `token ${decryptedConfig.giteaConfig!.token}`,
@@ -1795,11 +1932,13 @@ export async function mirrorGitRepoLabelsToGitea({
octokit,
repository,
giteaOwner,
giteaRepoName,
}: {
config: Partial<Config>;
octokit: Octokit;
repository: Repository;
giteaOwner: string;
giteaRepoName?: string;
}) {
if (
!config.githubConfig?.token ||
@@ -1812,17 +1951,20 @@ export async function mirrorGitRepoLabelsToGitea({
// Decrypt config tokens for API usage
const decryptedConfig = decryptConfigTokens(config as Config);
// Use provided giteaRepoName or fall back to repository.name
const repoName = giteaRepoName || repository.name;
// Verify the repository exists in Gitea before attempting to mirror metadata
console.log(`[Labels] Verifying repository ${repository.name} exists at ${giteaOwner}`);
console.log(`[Labels] Verifying repository ${repoName} exists at ${giteaOwner}`);
const repoExists = await isRepoPresentInGitea({
config,
owner: giteaOwner,
repoName: repository.name,
repoName: repoName,
});
if (!repoExists) {
console.error(`[Labels] Repository ${repository.name} not found at ${giteaOwner}. Cannot mirror labels.`);
throw new Error(`Repository ${repository.name} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
console.error(`[Labels] Repository ${repoName} not found at ${giteaOwner}. Cannot mirror labels.`);
throw new Error(`Repository ${repoName} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
}
const [owner, repo] = repository.fullName.split("/");
@@ -1847,7 +1989,7 @@ export async function mirrorGitRepoLabelsToGitea({
// Get existing labels from Gitea
const giteaLabelsRes = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/labels`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/labels`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
@@ -1862,7 +2004,7 @@ export async function mirrorGitRepoLabelsToGitea({
if (!existingLabels.has(label.name)) {
try {
await httpPost(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/labels`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/labels`,
{
name: label.name,
color: `#${label.color}`,
@@ -1889,11 +2031,13 @@ export async function mirrorGitRepoMilestonesToGitea({
octokit,
repository,
giteaOwner,
giteaRepoName,
}: {
config: Partial<Config>;
octokit: Octokit;
repository: Repository;
giteaOwner: string;
giteaRepoName?: string;
}) {
if (
!config.githubConfig?.token ||
@@ -1906,17 +2050,20 @@ export async function mirrorGitRepoMilestonesToGitea({
// Decrypt config tokens for API usage
const decryptedConfig = decryptConfigTokens(config as Config);
// Use provided giteaRepoName or fall back to repository.name
const repoName = giteaRepoName || repository.name;
// Verify the repository exists in Gitea before attempting to mirror metadata
console.log(`[Milestones] Verifying repository ${repository.name} exists at ${giteaOwner}`);
console.log(`[Milestones] Verifying repository ${repoName} exists at ${giteaOwner}`);
const repoExists = await isRepoPresentInGitea({
config,
owner: giteaOwner,
repoName: repository.name,
repoName: repoName,
});
if (!repoExists) {
console.error(`[Milestones] Repository ${repository.name} not found at ${giteaOwner}. Cannot mirror milestones.`);
throw new Error(`Repository ${repository.name} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
console.error(`[Milestones] Repository ${repoName} not found at ${giteaOwner}. Cannot mirror milestones.`);
throw new Error(`Repository ${repoName} does not exist in Gitea at ${giteaOwner}. Please ensure the repository is mirrored first.`);
}
const [owner, repo] = repository.fullName.split("/");
@@ -1942,7 +2089,7 @@ export async function mirrorGitRepoMilestonesToGitea({
// Get existing milestones from Gitea
const giteaMilestonesRes = await httpGet(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/milestones`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/milestones`,
{
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
}
@@ -1957,7 +2104,7 @@ export async function mirrorGitRepoMilestonesToGitea({
if (!existingMilestones.has(milestone.title)) {
try {
await httpPost(
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repository.name}/milestones`,
`${config.giteaConfig.url}/api/v1/repos/${giteaOwner}/${repoName}/milestones`,
{
title: milestone.title,
description: milestone.description || "",
@@ -2029,6 +2176,14 @@ export async function archiveGiteaRepo(
repo: string
): Promise<void> {
try {
// Helper: sanitize to Gitea's AlphaDashDot rule
const sanitizeRepoNameAlphaDashDot = (name: string): string => {
// Replace anything that's not [A-Za-z0-9.-] with '-'
const base = name.replace(/[^A-Za-z0-9.-]+/g, "-").replace(/-+/g, "-");
// Trim leading/trailing separators and dots for safety
return base.replace(/^[.-]+/, "").replace(/[.-]+$/, "");
};
// First, check if this is a mirror repository
const repoResponse = await httpGet(
`${client.url}/api/v1/repos/${owner}/${repo}`,
@@ -2060,7 +2215,8 @@ export async function archiveGiteaRepo(
return;
}
const archivedName = `[ARCHIVED] ${currentName}`;
// Use a safe prefix and sanitize the name to satisfy AlphaDashDot rule
let archivedName = `archived-${sanitizeRepoNameAlphaDashDot(currentName)}`;
const currentDesc = repoResponse.data.description || '';
const archiveNotice = `\n\n⚠ ARCHIVED: Original GitHub repository no longer exists. Preserved as backup on ${new Date().toISOString()}`;
@@ -2069,23 +2225,40 @@ export async function archiveGiteaRepo(
? currentDesc
: currentDesc + archiveNotice;
const renameResponse = await httpPatch(
`${client.url}/api/v1/repos/${owner}/${repo}`,
{
name: archivedName,
description: newDescription,
},
{
Authorization: `token ${client.token}`,
'Content-Type': 'application/json',
try {
await httpPatch(
`${client.url}/api/v1/repos/${owner}/${repo}`,
{
name: archivedName,
description: newDescription,
},
{
Authorization: `token ${client.token}`,
'Content-Type': 'application/json',
}
);
} catch (e: any) {
// If rename fails (e.g., 422 AlphaDashDot or name conflict), attempt a timestamped fallback
const ts = new Date().toISOString().replace(/[-:T.]/g, "").slice(0, 14);
archivedName = `archived-${ts}-${sanitizeRepoNameAlphaDashDot(currentName)}`;
try {
await httpPatch(
`${client.url}/api/v1/repos/${owner}/${repo}`,
{
name: archivedName,
description: newDescription,
},
{
Authorization: `token ${client.token}`,
'Content-Type': 'application/json',
}
);
} catch (e2) {
// If this also fails, log but don't throw - data remains preserved
console.error(`[Archive] Failed to rename mirror repository ${owner}/${repo}:`, e2);
console.log(`[Archive] Repository ${owner}/${repo} remains accessible but not marked as archived`);
return;
}
);
if (renameResponse.status >= 400) {
// If rename fails, log but don't throw - data is still preserved
console.error(`[Archive] Failed to rename mirror repository ${owner}/${repo}: ${renameResponse.status}`);
console.log(`[Archive] Repository ${owner}/${repo} remains accessible but not marked as archived`);
return;
}
console.log(`[Archive] Successfully marked mirror repository ${owner}/${repo} as archived (renamed to ${archivedName})`);

View File

@@ -18,8 +18,11 @@ if (process.env.NODE_ENV !== "test") {
}
}
// Extend Octokit with throttling plugin
const MyOctokit = Octokit.plugin(throttling);
// Extend Octokit with throttling plugin when available (tests may stub Octokit)
// Fallback to base Octokit if .plugin is not present
const MyOctokit: any = (Octokit as any)?.plugin?.call
? (Octokit as any).plugin(throttling)
: Octokit as any;
/**
* Creates an authenticated Octokit instance with rate limit tracking and throttling

View File

@@ -0,0 +1,75 @@
import { describe, it, expect } from 'bun:test';
import { mergeGitReposPreferStarred, normalizeGitRepoToInsert, calcBatchSizeForInsert } from '@/lib/repo-utils';
import type { GitRepo } from '@/types/Repository';
function sampleRepo(overrides: Partial<GitRepo> = {}): GitRepo {
const base: GitRepo = {
name: 'repo',
fullName: 'owner/repo',
url: 'https://github.com/owner/repo',
cloneUrl: 'https://github.com/owner/repo.git',
owner: 'owner',
organization: undefined,
mirroredLocation: '',
destinationOrg: null,
isPrivate: false,
isForked: false,
forkedFrom: undefined,
hasIssues: true,
isStarred: false,
isArchived: false,
size: 1,
hasLFS: false,
hasSubmodules: false,
language: null,
description: null,
defaultBranch: 'main',
visibility: 'public',
status: 'imported',
lastMirrored: undefined,
errorMessage: undefined,
createdAt: new Date(),
updatedAt: new Date(),
};
return { ...base, ...overrides };
}
describe('mergeGitReposPreferStarred', () => {
it('keeps unique repos', () => {
const basic = [sampleRepo({ fullName: 'a/x', name: 'x' })];
const starred: GitRepo[] = [];
const merged = mergeGitReposPreferStarred(basic, starred);
expect(merged).toHaveLength(1);
expect(merged[0].fullName).toBe('a/x');
});
it('prefers starred when duplicate exists', () => {
const basic = [sampleRepo({ fullName: 'a/x', name: 'x', isStarred: false })];
const starred = [sampleRepo({ fullName: 'a/x', name: 'x', isStarred: true })];
const merged = mergeGitReposPreferStarred(basic, starred);
expect(merged).toHaveLength(1);
expect(merged[0].isStarred).toBe(true);
});
});
describe('normalizeGitRepoToInsert', () => {
it('sets undefined optional fields to null', () => {
const repo = sampleRepo({ organization: undefined, forkedFrom: undefined, language: undefined, description: undefined, lastMirrored: undefined, errorMessage: undefined });
const insert = normalizeGitRepoToInsert(repo, { userId: 'u', configId: 'c' });
expect(insert.organization).toBeNull();
expect(insert.forkedFrom).toBeNull();
expect(insert.language).toBeNull();
expect(insert.description).toBeNull();
expect(insert.lastMirrored).toBeNull();
expect(insert.errorMessage).toBeNull();
});
});
describe('calcBatchSizeForInsert', () => {
it('respects 999 parameter limit', () => {
const batch = calcBatchSizeForInsert(29);
expect(batch).toBeGreaterThan(0);
expect(batch * 29).toBeLessThanOrEqual(999);
});
});

71
src/lib/repo-utils.ts Normal file
View File

@@ -0,0 +1,71 @@
import { v4 as uuidv4 } from 'uuid';
import type { GitRepo } from '@/types/Repository';
import { repositories } from '@/lib/db/schema';
export type RepoInsert = typeof repositories.$inferInsert;
// Merge lists and de-duplicate by fullName, preferring starred variant when present
export function mergeGitReposPreferStarred(
basicAndForked: GitRepo[],
starred: GitRepo[]
): GitRepo[] {
const map = new Map<string, GitRepo>();
for (const r of [...basicAndForked, ...starred]) {
const existing = map.get(r.fullName);
if (!existing || (!existing.isStarred && r.isStarred)) {
map.set(r.fullName, r);
}
}
return Array.from(map.values());
}
// Convert a GitRepo to a normalized DB insert object with all nullable fields set
export function normalizeGitRepoToInsert(
repo: GitRepo,
{
userId,
configId,
}: { userId: string; configId: string }
): RepoInsert {
return {
id: uuidv4(),
userId,
configId,
name: repo.name,
fullName: repo.fullName,
url: repo.url,
cloneUrl: repo.cloneUrl,
owner: repo.owner,
organization: repo.organization ?? null,
mirroredLocation: repo.mirroredLocation || '',
destinationOrg: repo.destinationOrg || null,
isPrivate: repo.isPrivate,
isForked: repo.isForked,
forkedFrom: repo.forkedFrom ?? null,
hasIssues: repo.hasIssues,
isStarred: repo.isStarred,
isArchived: repo.isArchived,
size: repo.size,
hasLFS: repo.hasLFS,
hasSubmodules: repo.hasSubmodules,
language: repo.language ?? null,
description: repo.description ?? null,
defaultBranch: repo.defaultBranch,
visibility: repo.visibility,
status: 'imported',
lastMirrored: repo.lastMirrored ?? null,
errorMessage: repo.errorMessage ?? null,
createdAt: repo.createdAt || new Date(),
updatedAt: repo.updatedAt || new Date(),
};
}
// Compute a safe batch size based on SQLite 999-parameter limit
export function calcBatchSizeForInsert(columnCount: number, maxParams = 999): number {
if (columnCount <= 0) return 1;
// Reserve a little headroom in case column count drifts
const safety = 0;
const effectiveMax = Math.max(1, maxParams - safety);
return Math.max(1, Math.floor(effectiveMax / columnCount));
}

View File

@@ -7,7 +7,7 @@
import { db, configs, repositories } from '@/lib/db';
import { eq, and, or, sql, not, inArray } from 'drizzle-orm';
import { createGitHubClient, getGithubRepositories, getGithubStarredRepositories } from '@/lib/github';
import { createGiteaClient, deleteGiteaRepo, archiveGiteaRepo } from '@/lib/gitea';
import { createGiteaClient, deleteGiteaRepo, archiveGiteaRepo, getGiteaRepoOwnerAsync, checkRepoLocation } from '@/lib/gitea';
import { getDecryptedGitHubToken, getDecryptedGiteaToken } from '@/lib/utils/config-encryption';
import { publishEvent } from '@/lib/events';
@@ -109,26 +109,46 @@ async function handleOrphanedRepository(
const giteaToken = getDecryptedGiteaToken(config);
const giteaClient = createGiteaClient(config.giteaConfig.url, giteaToken);
// Determine the Gitea owner and repo name
const mirroredLocation = repo.mirroredLocation || '';
let giteaOwner = repo.owner;
let giteaRepoName = repo.name;
if (mirroredLocation) {
const parts = mirroredLocation.split('/');
if (parts.length >= 2) {
giteaOwner = parts[parts.length - 2];
giteaRepoName = parts[parts.length - 1];
}
// Determine the Gitea owner and repo name more robustly
const mirroredLocation = (repo.mirroredLocation || '').trim();
let giteaOwner: string;
let giteaRepoName: string;
if (mirroredLocation && mirroredLocation.includes('/')) {
const [ownerPart, namePart] = mirroredLocation.split('/');
giteaOwner = ownerPart;
giteaRepoName = namePart;
} else {
// Fall back to expected owner based on config and repo flags (starred/org overrides)
giteaOwner = await getGiteaRepoOwnerAsync({ config, repository: repo });
giteaRepoName = repo.name;
}
// Normalize owner casing to avoid GetUserByName issues on some Gitea setups
giteaOwner = giteaOwner.trim();
if (action === 'archive') {
console.log(`[Repository Cleanup] Archiving orphaned repository ${repoFullName} in Gitea`);
// Best-effort check to validate actual location; falls back gracefully
try {
const { present, actualOwner } = await checkRepoLocation({
config,
repository: repo,
expectedOwner: giteaOwner,
});
if (present) {
giteaOwner = actualOwner;
}
} catch {
// Non-fatal; continue with best guess
}
await archiveGiteaRepo(giteaClient, giteaOwner, giteaRepoName);
// Update database status
await db.update(repositories).set({
status: 'archived',
isArchived: true,
errorMessage: 'Repository archived - no longer in GitHub',
updatedAt: new Date(),
}).where(eq(repositories.id, repo.id));
@@ -402,4 +422,4 @@ export async function triggerRepositoryCleanup(userId: string): Promise<{
}
return runRepositoryCleanup(config);
}
}

View File

@@ -11,6 +11,7 @@ import { getDecryptedGitHubToken } from '@/lib/utils/config-encryption';
import { parseInterval, formatDuration } from '@/lib/utils/duration-parser';
import type { Repository } from '@/lib/db/schema';
import { repoStatusEnum, repositoryVisibilityEnum } from '@/types/Repository';
import { mergeGitReposPreferStarred, normalizeGitRepoToInsert, calcBatchSizeForInsert } from '@/lib/repo-utils';
let schedulerInterval: NodeJS.Timeout | null = null;
let isSchedulerRunning = false;
@@ -94,8 +95,7 @@ async function runScheduledSync(config: any): Promise<void> {
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
]);
const allGithubRepos = [...basicAndForkedRepos, ...starredRepos];
const allGithubRepos = mergeGitReposPreferStarred(basicAndForkedRepos, starredRepos);
// Check for new repositories
const existingRepos = await db
@@ -110,37 +110,21 @@ async function runScheduledSync(config: any): Promise<void> {
console.log(`[Scheduler] Found ${newRepos.length} new repositories for user ${userId}`);
// Insert new repositories
const reposToInsert = newRepos.map(repo => ({
id: uuidv4(),
userId,
configId: config.id,
name: repo.name,
fullName: repo.fullName,
url: repo.url,
cloneUrl: repo.cloneUrl,
owner: repo.owner,
organization: repo.organization,
mirroredLocation: repo.mirroredLocation || "",
destinationOrg: repo.destinationOrg || null,
isPrivate: repo.isPrivate,
isForked: repo.isForked,
forkedFrom: repo.forkedFrom,
hasIssues: repo.hasIssues,
isStarred: repo.isStarred,
isArchived: repo.isArchived,
size: repo.size,
hasLFS: repo.hasLFS,
hasSubmodules: repo.hasSubmodules,
language: repo.language || null,
description: repo.description || null,
defaultBranch: repo.defaultBranch,
visibility: repo.visibility,
status: 'imported',
createdAt: new Date(),
updatedAt: new Date(),
}));
const reposToInsert = newRepos.map(repo =>
normalizeGitRepoToInsert(repo, { userId, configId: config.id })
);
await db.insert(repositories).values(reposToInsert);
// Batch insert to avoid SQLite parameter limit
const sample = reposToInsert[0];
const columnCount = Object.keys(sample ?? {}).length || 1;
const BATCH_SIZE = calcBatchSizeForInsert(columnCount);
for (let i = 0; i < reposToInsert.length; i += BATCH_SIZE) {
const batch = reposToInsert.slice(i, i + BATCH_SIZE);
await db
.insert(repositories)
.values(batch)
.onConflictDoNothing({ target: [repositories.userId, repositories.fullName] });
}
console.log(`[Scheduler] Successfully imported ${newRepos.length} new repositories for user ${userId}`);
} else {
console.log(`[Scheduler] No new repositories found for user ${userId}`);
@@ -182,6 +166,75 @@ async function runScheduledSync(config: any): Promise<void> {
}
}
// Auto-mirror: Mirror imported/pending/failed repositories if enabled
if (scheduleConfig.autoMirror) {
try {
console.log(`[Scheduler] Auto-mirror enabled - checking for repositories to mirror for user ${userId}...`);
const reposNeedingMirror = await db
.select()
.from(repositories)
.where(
and(
eq(repositories.userId, userId),
or(
eq(repositories.status, 'imported'),
eq(repositories.status, 'pending'),
eq(repositories.status, 'failed')
)
)
);
if (reposNeedingMirror.length > 0) {
console.log(`[Scheduler] Found ${reposNeedingMirror.length} repositories that need initial mirroring`);
// Prepare Octokit client
const decryptedToken = getDecryptedGitHubToken(config);
const { Octokit } = await import('@octokit/rest');
const octokit = new Octokit({ auth: decryptedToken });
// Process repositories in batches
const batchSize = scheduleConfig.batchSize || 10;
const pauseBetweenBatches = scheduleConfig.pauseBetweenBatches || 2000;
for (let i = 0; i < reposNeedingMirror.length; i += batchSize) {
const batch = reposNeedingMirror.slice(i, Math.min(i + batchSize, reposNeedingMirror.length));
console.log(`[Scheduler] Auto-mirror batch ${Math.floor(i / batchSize) + 1} of ${Math.ceil(reposNeedingMirror.length / batchSize)} (${batch.length} repos)`);
await Promise.all(
batch.map(async (repo) => {
try {
const repository: Repository = {
...repo,
status: repoStatusEnum.parse(repo.status),
organization: repo.organization ?? undefined,
lastMirrored: repo.lastMirrored ?? undefined,
errorMessage: repo.errorMessage ?? undefined,
mirroredLocation: repo.mirroredLocation || '',
forkedFrom: repo.forkedFrom ?? undefined,
visibility: repositoryVisibilityEnum.parse(repo.visibility),
};
await mirrorGithubRepoToGitea({ octokit, repository, config });
console.log(`[Scheduler] Auto-mirrored repository: ${repo.fullName}`);
} catch (error) {
console.error(`[Scheduler] Failed to auto-mirror repository ${repo.fullName}:`, error);
}
})
);
// Pause between batches if configured
if (i + batchSize < reposNeedingMirror.length) {
console.log(`[Scheduler] Pausing for ${pauseBetweenBatches}ms before next auto-mirror batch...`);
await new Promise(resolve => setTimeout(resolve, pauseBetweenBatches));
}
}
} else {
console.log(`[Scheduler] No repositories need initial mirroring`);
}
} catch (mirrorError) {
console.error(`[Scheduler] Error during auto-mirror phase for user ${userId}:`, mirrorError);
}
}
// Get repositories to sync
let reposToSync = await db
.select()
@@ -375,8 +428,7 @@ async function performInitialAutoStart(): Promise<void> {
? getGithubStarredRepositories({ octokit, config })
: Promise.resolve([]),
]);
const allGithubRepos = [...basicAndForkedRepos, ...starredRepos];
const allGithubRepos = mergeGitReposPreferStarred(basicAndForkedRepos, starredRepos);
// Check for new repositories
const existingRepos = await db
@@ -391,37 +443,21 @@ async function performInitialAutoStart(): Promise<void> {
console.log(`[Scheduler] Importing ${reposToImport.length} repositories for user ${config.userId}...`);
// Insert new repositories
const reposToInsert = reposToImport.map(repo => ({
id: uuidv4(),
userId: config.userId,
configId: config.id,
name: repo.name,
fullName: repo.fullName,
url: repo.url,
cloneUrl: repo.cloneUrl,
owner: repo.owner,
organization: repo.organization,
mirroredLocation: repo.mirroredLocation || "",
destinationOrg: repo.destinationOrg || null,
isPrivate: repo.isPrivate,
isForked: repo.isForked,
forkedFrom: repo.forkedFrom,
hasIssues: repo.hasIssues,
isStarred: repo.isStarred,
isArchived: repo.isArchived,
size: repo.size,
hasLFS: repo.hasLFS,
hasSubmodules: repo.hasSubmodules,
language: repo.language || null,
description: repo.description || null,
defaultBranch: repo.defaultBranch,
visibility: repo.visibility,
status: 'imported',
createdAt: new Date(),
updatedAt: new Date(),
}));
const reposToInsert = reposToImport.map(repo =>
normalizeGitRepoToInsert(repo, { userId: config.userId, configId: config.id })
);
await db.insert(repositories).values(reposToInsert);
// Batch insert to avoid SQLite parameter limit
const sample = reposToInsert[0];
const columnCount = Object.keys(sample ?? {}).length || 1;
const BATCH_SIZE = calcBatchSizeForInsert(columnCount);
for (let i = 0; i < reposToInsert.length; i += BATCH_SIZE) {
const batch = reposToInsert.slice(i, i + BATCH_SIZE);
await db
.insert(repositories)
.values(batch)
.onConflictDoNothing({ target: [repositories.userId, repositories.fullName] });
}
console.log(`[Scheduler] Successfully imported ${reposToImport.length} repositories`);
} else {
console.log(`[Scheduler] No new repositories to import for user ${config.userId}`);
@@ -697,4 +733,4 @@ export function stopSchedulerService(): void {
*/
export function isSchedulerServiceRunning(): boolean {
return schedulerInterval !== null;
}
}

View File

@@ -11,6 +11,7 @@ import type {
} from "@/types/config";
import { z } from "zod";
import { githubConfigSchema, giteaConfigSchema, scheduleConfigSchema, cleanupConfigSchema } from "@/lib/db/schema";
import { parseInterval } from "@/lib/utils/duration-parser";
// Use the actual database schema types
type DbGitHubConfig = z.infer<typeof githubConfigSchema>;
@@ -165,27 +166,22 @@ export function mapDbToUiConfig(dbConfig: any): {
/**
* Maps UI schedule config to database schema
*/
export function mapUiScheduleToDb(uiSchedule: any): DbScheduleConfig {
export function mapUiScheduleToDb(uiSchedule: any, existing?: DbScheduleConfig): DbScheduleConfig {
// Preserve existing schedule config and only update fields controlled by the UI
const base: DbScheduleConfig = existing
? { ...(existing as unknown as DbScheduleConfig) }
: (scheduleConfigSchema.parse({}) as unknown as DbScheduleConfig);
// Store interval as seconds string to avoid lossy cron conversion
const intervalSeconds = typeof uiSchedule.interval === 'number' && uiSchedule.interval > 0
? String(uiSchedule.interval)
: (typeof base.interval === 'string' ? base.interval : String(86400));
return {
enabled: uiSchedule.enabled || false,
interval: uiSchedule.interval ? `0 */${Math.floor(uiSchedule.interval / 3600)} * * *` : "0 2 * * *", // Convert seconds to cron expression
concurrent: false,
batchSize: 10,
pauseBetweenBatches: 5000,
retryAttempts: 3,
retryDelay: 60000,
timeout: 3600000,
autoRetry: true,
cleanupBeforeMirror: false,
notifyOnFailure: true,
notifyOnSuccess: false,
logLevel: "info",
timezone: "UTC",
onlyMirrorUpdated: false,
updateInterval: 86400000,
skipRecentlyMirrored: true,
recentThreshold: 3600000,
};
...base,
enabled: !!uiSchedule.enabled,
interval: intervalSeconds,
} as DbScheduleConfig;
}
/**
@@ -202,23 +198,18 @@ export function mapDbScheduleToUi(dbSchedule: DbScheduleConfig): any {
};
}
// Extract hours from cron expression if possible
// Parse interval supporting numbers (seconds), duration strings, and cron
let intervalSeconds = 86400; // Default to daily (24 hours)
if (dbSchedule.interval) {
// Check if it's already a number (seconds), use it directly
if (typeof dbSchedule.interval === 'number') {
intervalSeconds = dbSchedule.interval;
} else if (typeof dbSchedule.interval === 'string') {
// Check if it's a cron expression
const cronMatch = dbSchedule.interval.match(/0 \*\/(\d+) \* \* \*/);
if (cronMatch) {
intervalSeconds = parseInt(cronMatch[1]) * 3600;
} else if (dbSchedule.interval === "0 2 * * *") {
// Daily at 2 AM
intervalSeconds = 86400;
}
}
try {
const ms = parseInterval(
typeof dbSchedule.interval === 'number'
? dbSchedule.interval
: (dbSchedule.interval as unknown as string)
);
intervalSeconds = Math.max(1, Math.floor(ms / 1000));
} catch (_e) {
// Fallback to default if unparsable
intervalSeconds = 86400;
}
return {
@@ -266,4 +257,4 @@ export function mapDbCleanupToUi(dbCleanup: DbCleanupConfig): any {
lastRun: dbCleanup.lastRun || null,
nextRun: dbCleanup.nextRun || null,
};
}
}

View File

@@ -2,6 +2,9 @@ import type { APIContext } from "astro";
import { createSecureErrorResponse } from "@/lib/utils";
import { requireAuth } from "@/lib/utils/auth-helpers";
import { auth } from "@/lib/auth";
import { db, ssoProviders } from "@/lib/db";
import { eq } from "drizzle-orm";
import { nanoid } from "nanoid";
// POST /api/auth/sso/register - Register a new SSO provider using Better Auth
export async function POST(context: APIContext) {
@@ -168,7 +171,47 @@ export async function POST(context: APIContext) {
}
const result = await response.json();
// Mirror provider into our local sso_providers table for UI listing
try {
const existing = await db
.select()
.from(ssoProviders)
.where(eq(ssoProviders.providerId, providerId))
.limit(1);
const values: any = {
issuer: registrationBody.issuer,
domain: registrationBody.domain,
organizationId: registrationBody.organizationId,
updatedAt: new Date(),
};
if (registrationBody.oidcConfig) {
values.oidcConfig = JSON.stringify(registrationBody.oidcConfig);
}
if (registrationBody.samlConfig) {
values.samlConfig = JSON.stringify(registrationBody.samlConfig);
}
if (existing.length > 0) {
await db.update(ssoProviders).set(values).where(eq(ssoProviders.id, existing[0].id));
} else {
await db.insert(ssoProviders).values({
id: nanoid(),
issuer: registrationBody.issuer,
domain: registrationBody.domain,
oidcConfig: JSON.stringify(registrationBody.oidcConfig || {}),
samlConfig: registrationBody.samlConfig ? JSON.stringify(registrationBody.samlConfig) : undefined,
userId: user.id,
providerId: registrationBody.providerId,
organizationId: registrationBody.organizationId,
});
}
} catch (e) {
// Do not fail the main request if mirroring to local table fails
console.warn("Failed to mirror SSO provider to local DB:", e);
}
return new Response(JSON.stringify(result), {
status: 201,
headers: { "Content-Type": "application/json" },
@@ -199,4 +242,4 @@ export async function GET(context: APIContext) {
} catch (error) {
return createSecureErrorResponse(error, "SSO provider listing");
}
}
}

View File

@@ -87,7 +87,10 @@ export const POST: APIRoute = async ({ request }) => {
}
// Map schedule and cleanup configs to database schema
const processedScheduleConfig = mapUiScheduleToDb(scheduleConfig);
const processedScheduleConfig = mapUiScheduleToDb(
scheduleConfig,
existingConfig ? existingConfig.scheduleConfig : undefined
);
const processedCleanupConfig = mapUiCleanupToDb(cleanupConfig);
if (existingConfig) {

View File

@@ -8,6 +8,7 @@ import type {
ScheduleSyncRepoResponse,
} from "@/types/sync";
import { createSecureErrorResponse } from "@/lib/utils";
import { parseInterval } from "@/lib/utils/duration-parser";
export const POST: APIRoute = async ({ request }) => {
try {
@@ -72,8 +73,17 @@ export const POST: APIRoute = async ({ request }) => {
// Calculate nextRun and update lastRun and nextRun in the config
const currentTime = new Date();
const interval = config.scheduleConfig?.interval ?? 3600;
const nextRun = new Date(currentTime.getTime() + interval * 1000);
let intervalMs = 3600 * 1000;
try {
intervalMs = parseInterval(
typeof config.scheduleConfig?.interval === 'number'
? config.scheduleConfig.interval
: (config.scheduleConfig?.interval as unknown as string) || '3600'
);
} catch {
intervalMs = 3600 * 1000;
}
const nextRun = new Date(currentTime.getTime() + intervalMs);
// Update the full giteaConfig object
await db

View File

@@ -4,6 +4,7 @@ import { requireAuth } from "@/lib/utils/auth-helpers";
import { db, ssoProviders } from "@/lib/db";
import { nanoid } from "nanoid";
import { eq } from "drizzle-orm";
import { auth } from "@/lib/auth";
// GET /api/sso/providers - List all SSO providers
export async function GET(context: APIContext) {
@@ -29,7 +30,11 @@ export async function GET(context: APIContext) {
}
}
// POST /api/sso/providers - Create a new SSO provider
// POST /api/sso/providers - DEPRECATED legacy create (use Better Auth registration)
// This route remains for backward-compatibility only. Preferred flow:
// - Client/UI calls authClient.sso.register(...) to register with Better Auth
// - Server mirrors provider into local DB for listing
// Creation via this route is discouraged and may be removed in a future version.
export async function POST(context: APIContext) {
try {
const { user, response } = await requireAuth(context);
@@ -45,10 +50,12 @@ export async function POST(context: APIContext) {
tokenEndpoint,
jwksEndpoint,
userInfoEndpoint,
discoveryEndpoint,
mapping,
providerId,
organizationId,
scopes,
pkce,
} = body;
// Validate required fields
@@ -62,6 +69,32 @@ export async function POST(context: APIContext) {
);
}
// Clean issuer URL (remove trailing slash); validate URL format
let cleanIssuer = issuer;
try {
const issuerUrl = new URL(issuer.toString().trim());
cleanIssuer = issuerUrl.toString().replace(/\/$/, "");
} catch {
return new Response(
JSON.stringify({ error: `Invalid issuer URL format: ${issuer}` }),
{ status: 400, headers: { "Content-Type": "application/json" } }
);
}
// Validate OIDC endpoints: require discoveryEndpoint or at least authorization+token
const hasDiscovery = typeof discoveryEndpoint === 'string' && discoveryEndpoint.trim() !== '';
const hasCoreEndpoints = typeof authorizationEndpoint === 'string' && authorizationEndpoint.trim() !== ''
&& typeof tokenEndpoint === 'string' && tokenEndpoint.trim() !== '';
if (!hasDiscovery && !hasCoreEndpoints) {
return new Response(
JSON.stringify({
error: "Invalid OIDC configuration",
details: "Provide discoveryEndpoint, or both authorizationEndpoint and tokenEndpoint."
}),
{ status: 400, headers: { "Content-Type": "application/json" } }
);
}
// Check if provider ID already exists
const existing = await db
.select()
@@ -79,15 +112,27 @@ export async function POST(context: APIContext) {
);
}
// Create OIDC config object
// Helper to validate and normalize URL strings (optional fields allowed)
const validateUrl = (value?: string) => {
if (!value || typeof value !== 'string' || value.trim() === '') return undefined;
try {
return new URL(value.trim()).toString();
} catch {
return undefined;
}
};
// Create OIDC config object (store as-is for UI and for Better Auth registration)
const oidcConfig = {
clientId,
clientSecret,
authorizationEndpoint,
tokenEndpoint,
jwksEndpoint,
userInfoEndpoint,
authorizationEndpoint: validateUrl(authorizationEndpoint),
tokenEndpoint: validateUrl(tokenEndpoint),
jwksEndpoint: validateUrl(jwksEndpoint),
userInfoEndpoint: validateUrl(userInfoEndpoint),
discoveryEndpoint: validateUrl(discoveryEndpoint),
scopes: scopes || ["openid", "email", "profile"],
pkce: pkce !== false,
mapping: mapping || {
id: "sub",
email: "email",
@@ -97,12 +142,55 @@ export async function POST(context: APIContext) {
},
};
// First, register with Better Auth so the SSO plugin has the provider
try {
const headers = new Headers();
const cookieHeader = context.request.headers.get("cookie");
if (cookieHeader) headers.set("cookie", cookieHeader);
const res = await auth.api.registerSSOProvider({
body: {
providerId,
issuer: cleanIssuer,
domain,
organizationId,
oidcConfig: {
clientId: oidcConfig.clientId,
clientSecret: oidcConfig.clientSecret,
authorizationEndpoint: oidcConfig.authorizationEndpoint,
tokenEndpoint: oidcConfig.tokenEndpoint,
jwksEndpoint: oidcConfig.jwksEndpoint,
discoveryEndpoint: oidcConfig.discoveryEndpoint,
userInfoEndpoint: oidcConfig.userInfoEndpoint,
scopes: oidcConfig.scopes,
pkce: oidcConfig.pkce,
},
mapping: oidcConfig.mapping,
},
headers,
});
if (!res.ok) {
const errText = await res.text();
return new Response(
JSON.stringify({ error: `Failed to register SSO provider: ${errText}` }),
{ status: res.status || 500, headers: { "Content-Type": "application/json" } }
);
}
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
return new Response(
JSON.stringify({ error: `Better Auth registration failed: ${message}` }),
{ status: 500, headers: { "Content-Type": "application/json" } }
);
}
// Insert new provider
const [newProvider] = await db
.insert(ssoProviders)
.values({
id: nanoid(),
issuer,
issuer: cleanIssuer,
domain,
oidcConfig: JSON.stringify(oidcConfig),
userId: user.id,
@@ -259,4 +347,4 @@ export async function DELETE(context: APIContext) {
} catch (error) {
return createSecureErrorResponse(error, "SSO providers API");
}
}
}

View File

@@ -10,6 +10,7 @@ import {
getGithubStarredRepositories,
} from "@/lib/github";
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
import { mergeGitReposPreferStarred, calcBatchSizeForInsert } from "@/lib/repo-utils";
import { getDecryptedGitHubToken } from "@/lib/utils/config-encryption";
export const POST: APIRoute = async ({ request }) => {
@@ -55,7 +56,8 @@ export const POST: APIRoute = async ({ request }) => {
getGithubOrganizations({ octokit, config }),
]);
const allGithubRepos = [...basicAndForkedRepos, ...starredRepos];
// Merge and de-duplicate by fullName, preferring starred variant when duplicated
const allGithubRepos = mergeGitReposPreferStarred(basicAndForkedRepos, starredRepos);
// Prepare full list of repos and orgs
const newRepos = allGithubRepos.map((repo) => ({
@@ -67,25 +69,25 @@ export const POST: APIRoute = async ({ request }) => {
url: repo.url,
cloneUrl: repo.cloneUrl,
owner: repo.owner,
organization: repo.organization,
organization: repo.organization ?? null,
mirroredLocation: repo.mirroredLocation || "",
destinationOrg: repo.destinationOrg || null,
isPrivate: repo.isPrivate,
isForked: repo.isForked,
forkedFrom: repo.forkedFrom,
forkedFrom: repo.forkedFrom ?? null,
hasIssues: repo.hasIssues,
isStarred: repo.isStarred,
isArchived: repo.isArchived,
size: repo.size,
hasLFS: repo.hasLFS,
hasSubmodules: repo.hasSubmodules,
language: repo.language || null,
description: repo.description || null,
language: repo.language ?? null,
description: repo.description ?? null,
defaultBranch: repo.defaultBranch,
visibility: repo.visibility,
status: repo.status,
lastMirrored: repo.lastMirrored,
errorMessage: repo.errorMessage,
lastMirrored: repo.lastMirrored ?? null,
errorMessage: repo.errorMessage ?? null,
createdAt: repo.createdAt,
updatedAt: repo.updatedAt,
}));
@@ -128,12 +130,27 @@ export const POST: APIRoute = async ({ request }) => {
);
insertedOrgs = newOrgs.filter((o) => !existingOrgNames.has(o.name));
// Batch insert repositories to avoid SQLite parameter limit (dynamic by column count)
const sample = newRepos[0];
const columnCount = Object.keys(sample ?? {}).length || 1;
const REPO_BATCH_SIZE = calcBatchSizeForInsert(columnCount);
if (insertedRepos.length > 0) {
await tx.insert(repositories).values(insertedRepos);
for (let i = 0; i < insertedRepos.length; i += REPO_BATCH_SIZE) {
const batch = insertedRepos.slice(i, i + REPO_BATCH_SIZE);
await tx
.insert(repositories)
.values(batch)
.onConflictDoNothing({ target: [repositories.userId, repositories.fullName] });
}
}
// Batch insert organizations (they have fewer fields, so we can use larger batches)
const ORG_BATCH_SIZE = 100;
if (insertedOrgs.length > 0) {
await tx.insert(organizations).values(insertedOrgs);
for (let i = 0; i < insertedOrgs.length; i += ORG_BATCH_SIZE) {
const batch = insertedOrgs.slice(i, i + ORG_BATCH_SIZE);
await tx.insert(organizations).values(batch);
}
}
});

View File

@@ -122,25 +122,36 @@ export const POST: APIRoute = async ({ request }) => {
destinationOrg: null,
isPrivate: repo.private,
isForked: repo.fork,
forkedFrom: undefined,
forkedFrom: null,
hasIssues: repo.has_issues,
isStarred: false,
isArchived: repo.archived,
size: repo.size,
hasLFS: false,
hasSubmodules: false,
language: repo.language || null,
description: repo.description || null,
language: repo.language ?? null,
description: repo.description ?? null,
defaultBranch: repo.default_branch ?? "main",
visibility: (repo.visibility ?? "public") as RepositoryVisibility,
status: "imported" as RepoStatus,
lastMirrored: undefined,
errorMessage: undefined,
lastMirrored: null,
errorMessage: null,
createdAt: repo.created_at ? new Date(repo.created_at) : new Date(),
updatedAt: repo.updated_at ? new Date(repo.updated_at) : new Date(),
}));
await db.insert(repositories).values(repoRecords);
// Batch insert repositories to avoid SQLite parameter limit
// Compute batch size based on column count
const sample = repoRecords[0];
const columnCount = Object.keys(sample ?? {}).length || 1;
const BATCH_SIZE = Math.max(1, Math.floor(999 / columnCount));
for (let i = 0; i < repoRecords.length; i += BATCH_SIZE) {
const batch = repoRecords.slice(i, i + BATCH_SIZE);
await db
.insert(repositories)
.values(batch)
.onConflictDoNothing({ target: [repositories.userId, repositories.fullName] });
}
// Insert organization metadata
const organizationRecord = {

View File

@@ -80,25 +80,23 @@ export const POST: APIRoute = async ({ request }) => {
cloneUrl: repoData.clone_url,
owner: repoData.owner.login,
organization:
repoData.owner.type === "Organization"
? repoData.owner.login
: undefined,
repoData.owner.type === "Organization" ? repoData.owner.login : null,
isPrivate: repoData.private,
isForked: repoData.fork,
forkedFrom: undefined,
forkedFrom: null,
hasIssues: repoData.has_issues,
isStarred: false,
isArchived: repoData.archived,
size: repoData.size,
hasLFS: false,
hasSubmodules: false,
language: repoData.language || null,
description: repoData.description || null,
language: repoData.language ?? null,
description: repoData.description ?? null,
defaultBranch: repoData.default_branch,
visibility: (repoData.visibility ?? "public") as RepositoryVisibility,
status: "imported" as Repository["status"],
lastMirrored: undefined,
errorMessage: undefined,
lastMirrored: null,
errorMessage: null,
mirroredLocation: "",
destinationOrg: null,
createdAt: repoData.created_at
@@ -109,7 +107,10 @@ export const POST: APIRoute = async ({ request }) => {
: new Date(),
};
await db.insert(repositories).values(metadata);
await db
.insert(repositories)
.values(metadata)
.onConflictDoNothing({ target: [repositories.userId, repositories.fullName] });
createMirrorJob({
userId,

View File

@@ -52,7 +52,9 @@ import MainLayout from '../../layouts/main.astro';
{ var: 'PORT', desc: 'Server port', default: '4321' },
{ var: 'HOST', desc: 'Server host', default: '0.0.0.0' },
{ var: 'BETTER_AUTH_SECRET', desc: 'Authentication secret key', default: 'Auto-generated' },
{ var: 'BETTER_AUTH_URL', desc: 'Authentication base URL', default: 'http://localhost:4321' },
{ var: 'BETTER_AUTH_URL', desc: 'Authentication base URL (public origin)', default: 'http://localhost:4321' },
{ var: 'PUBLIC_BETTER_AUTH_URL', desc: 'Optional: public URL used by the client', default: 'Unset' },
{ var: 'BETTER_AUTH_TRUSTED_ORIGINS', desc: 'Comma-separated list of additional trusted origins', default: 'Unset' },
{ var: 'NODE_EXTRA_CA_CERTS', desc: 'Path to CA certificate file', default: 'None' },
{ var: 'DATABASE_URL', desc: 'SQLite database path', default: './data/gitea-mirror.db' },
].map((item, i) => (
@@ -464,4 +466,4 @@ ls -t "$BACKUP_DIR"/backup_*.tar.gz | tail -n +8 | xargs rm -f`}</code></pre>
</section>
</article>
</main>
</MainLayout>
</MainLayout>

View File

@@ -216,7 +216,7 @@ import MainLayout from '../../layouts/main.astro';
<div class="space-y-3">
{[
'User accounts and authentication data (Better Auth)',
'OAuth applications and SSO provider configurations',
'OAuth applications and SSO provider configurations (providers registered via Better Auth; mirrored locally for UI)',
'GitHub and Gitea configuration',
'Repository and organization information',
'Mirroring job history and status',
@@ -336,4 +336,4 @@ import MainLayout from '../../layouts/main.astro';
</section>
</article>
</main>
</MainLayout>
</MainLayout>

View File

@@ -107,6 +107,23 @@ import MainLayout from '../../layouts/main.astro';
</p>
<h3 class="text-xl font-semibold mb-4">Adding an SSO Provider</h3>
<div class="bg-blue-500/10 border border-blue-500/20 rounded-lg p-4 mb-6">
<div class="flex gap-3">
<div class="text-blue-600 dark:text-blue-500">
<svg class="w-5 h-5 mt-0.5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z"/>
</svg>
</div>
<div>
<p class="font-semibold text-blue-600 dark:text-blue-500 mb-1">Better Auth Registration</p>
<p class="text-sm">
Provider creation uses Better Auth's SSO registration under the hood. The legacy API route
<code class="bg-muted px-1 rounded">POST /api/sso/providers</code> is deprecated and not required for setup.
To change Provider ID or endpoints, delete and recreate the provider from the UI.
</p>
</div>
</div>
</div>
<div class="bg-card rounded-lg border border-border p-6 mb-6">
<h4 class="font-semibold mb-4">Required Information</h4>
@@ -147,11 +164,11 @@ import MainLayout from '../../layouts/main.astro';
<h3 class="text-xl font-semibold mb-4">Redirect URL Configuration</h3>
<div class="bg-muted/30 rounded-lg p-4">
<p class="text-sm mb-2">When configuring your SSO provider, use this redirect URL:</p>
<code class="bg-muted rounded px-3 py-2 block">https://your-domain.com/api/auth/sso/callback/{`{provider-id}`}</code>
<p class="text-xs text-muted-foreground mt-2">Replace <code>{`{provider-id}`}</code> with your chosen Provider ID (e.g., google-sso)</p>
</div>
<div class="bg-muted/30 rounded-lg p-4">
<p class="text-sm mb-2">When configuring your SSO provider, use this redirect URL:</p>
<code class="bg-muted rounded px-3 py-2 block">https://your-domain.com/api/auth/sso/callback/{`{provider-id}`}</code>
<p class="text-xs text-muted-foreground mt-2">Replace <code>{`{provider-id}`}</code> with your chosen Provider ID (e.g., google-sso)</p>
</div>
</section>
<div class="my-12 h-px bg-border/50"></div>
@@ -532,4 +549,4 @@ import MainLayout from '../../layouts/main.astro';
</section>
</article>
</main>
</MainLayout>
</MainLayout>

View File

@@ -4,7 +4,10 @@ import MainLayout from '../../layouts/main.astro';
const envVars = [
{ name: 'NODE_ENV', desc: 'Runtime environment', default: 'development', example: 'production' },
{ name: 'DATABASE_URL', desc: 'SQLite database URL', default: 'file:data/gitea-mirror.db', example: 'file:path/to/database.db' },
{ name: 'JWT_SECRET', desc: 'Secret key for JWT auth', default: 'Auto-generated', example: 'your-secure-string' },
{ name: 'BETTER_AUTH_SECRET', desc: 'Authentication secret key', default: 'Auto-generated', example: 'generate a strong random string' },
{ name: 'BETTER_AUTH_URL', desc: 'Authentication base URL (public origin)', default: 'http://localhost:4321', example: 'https://gitea-mirror.example.com' },
{ name: 'PUBLIC_BETTER_AUTH_URL', desc: 'Optional: public URL used by the client', default: 'Unset', example: 'https://gitea-mirror.example.com' },
{ name: 'BETTER_AUTH_TRUSTED_ORIGINS', desc: 'Comma-separated list of additional trusted origins', default: 'Unset', example: 'https://gitea-mirror.example.com,https://alt.example.com' },
{ name: 'HOST', desc: 'Server host', default: 'localhost', example: '0.0.0.0' },
{ name: 'PORT', desc: 'Server port', default: '4321', example: '8080' }
];
@@ -509,4 +512,4 @@ curl http://your-server:port/api/health`}</code></pre>
</section>
</article>
</main>
</MainLayout>
</MainLayout>

View File

@@ -260,6 +260,16 @@ bun run start</code></pre>
},
{
num: '3',
title: 'Optional: Configure SSO',
items: [
'Open Configuration → Authentication',
'Click “Add Provider” and enter your OIDC details',
'Use redirect URL: https://<your-domain>/api/auth/sso/callback/{provider-id}',
'Edits are handled as delete & recreate (Better Auth registration)'
]
},
{
num: '4',
title: 'Configure Gitea Connection',
items: [
'Enter your Gitea server URL',
@@ -269,7 +279,7 @@ bun run start</code></pre>
]
},
{
num: '4',
num: '5',
title: 'Set Up Scheduling',
items: [
'Enable automatic mirroring',
@@ -434,4 +444,4 @@ bun run start</code></pre>
</section>
</article>
</main>
</MainLayout>
</MainLayout>

View File

@@ -12,6 +12,7 @@ export const repoStatusEnum = z.enum([
"deleted",
"syncing",
"synced",
"archived",
]);
export type RepoStatus = z.infer<typeof repoStatusEnum>;

View File

@@ -29,11 +29,14 @@ export interface DatabaseCleanupConfig {
nextRun?: Date;
}
export type DuplicateNameStrategy = "suffix" | "prefix" | "owner-org";
export interface GitHubConfig {
username: string;
token: string;
privateRepositories: boolean;
mirrorStarred: boolean;
starredDuplicateStrategy?: DuplicateNameStrategy;
}
export interface MirrorOptions {

View File

@@ -0,0 +1,10 @@
import { describe, expect, it } from "bun:test";
import { repoStatusEnum } from "@/types/Repository";
describe("repoStatusEnum", () => {
it("includes archived status", () => {
const res = repoStatusEnum.safeParse("archived");
expect(res.success).toBe(true);
});
});

28
www/pnpm-lock.yaml generated
View File

@@ -28,7 +28,7 @@ importers:
version: 1.10.52
'@tailwindcss/vite':
specifier: ^4.1.12
version: 4.1.12(vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))
version: 4.1.12(vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))
'@types/canvas-confetti':
specifier: ^1.9.0
version: 1.9.0
@@ -2017,8 +2017,8 @@ packages:
vfile@6.0.3:
resolution: {integrity: sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q==}
vite@6.3.5:
resolution: {integrity: sha512-cZn6NDFE7wdTpINgs++ZJ4N49W2vRp8LCKrn3Ob1kYNtOo21vfDoaV5GzBfLU4MovSAB8uNRm4jgzVQZ+mBzPQ==}
vite@6.3.6:
resolution: {integrity: sha512-0msEVHJEScQbhkbVTb/4iHZdJ6SXp/AvxL2sjwYQFfBqleHtnCqv1J3sa9zbWz/6kW1m9Tfzn92vW+kZ1WV6QA==}
engines: {node: ^18.0.0 || ^20.0.0 || >=22.0.0}
hasBin: true
peerDependencies:
@@ -2193,11 +2193,11 @@ snapshots:
dependencies:
'@types/react': 19.1.12
'@types/react-dom': 19.1.9(@types/react@19.1.12)
'@vitejs/plugin-react': 4.6.0(vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))
'@vitejs/plugin-react': 4.6.0(vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))
react: 19.1.1
react-dom: 19.1.1(react@19.1.1)
ultrahtml: 1.6.0
vite: 6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
vite: 6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
transitivePeerDependencies:
- '@types/node'
- jiti
@@ -2760,12 +2760,12 @@ snapshots:
'@tailwindcss/oxide-win32-arm64-msvc': 4.1.12
'@tailwindcss/oxide-win32-x64-msvc': 4.1.12
'@tailwindcss/vite@4.1.12(vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))':
'@tailwindcss/vite@4.1.12(vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))':
dependencies:
'@tailwindcss/node': 4.1.12
'@tailwindcss/oxide': 4.1.12
tailwindcss: 4.1.12
vite: 6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
vite: 6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
'@types/babel__core@7.20.5':
dependencies:
@@ -2838,7 +2838,7 @@ snapshots:
'@ungap/structured-clone@1.3.0': {}
'@vitejs/plugin-react@4.6.0(vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))':
'@vitejs/plugin-react@4.6.0(vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))':
dependencies:
'@babel/core': 7.28.0
'@babel/plugin-transform-react-jsx-self': 7.27.1(@babel/core@7.28.0)
@@ -2846,7 +2846,7 @@ snapshots:
'@rolldown/pluginutils': 1.0.0-beta.19
'@types/babel__core': 7.20.5
react-refresh: 0.17.0
vite: 6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
vite: 6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
transitivePeerDependencies:
- supports-color
@@ -2935,8 +2935,8 @@ snapshots:
unist-util-visit: 5.0.0
unstorage: 1.16.0
vfile: 6.0.3
vite: 6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
vitefu: 1.1.1(vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))
vite: 6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
vitefu: 1.1.1(vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1))
xxhash-wasm: 1.1.0
yargs-parser: 21.1.1
yocto-spinner: 0.2.3
@@ -4526,7 +4526,7 @@ snapshots:
'@types/unist': 3.0.3
vfile-message: 4.0.2
vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1):
vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1):
dependencies:
esbuild: 0.25.9
fdir: 6.4.6(picomatch@4.0.2)
@@ -4540,9 +4540,9 @@ snapshots:
jiti: 2.5.1
lightningcss: 1.30.1
vitefu@1.1.1(vite@6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)):
vitefu@1.1.1(vite@6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)):
optionalDependencies:
vite: 6.3.5(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
vite: 6.3.6(@types/node@24.0.11)(jiti@2.5.1)(lightningcss@1.30.1)
web-namespaces@2.0.1: {}

View File

@@ -67,8 +67,35 @@ export function Hero() {
</div>
</div>
{/* Product Hunt Badge */}
<div className="mt-6 sm:mt-8 flex items-center justify-center px-4 z-20">
<a
href="https://www.producthunt.com/products/gitea-mirror?embed=true&utm_source=badge-featured&utm_medium=badge&utm_source=badge-gitea-mirror"
target="_blank"
rel="noopener noreferrer"
className="inline-block transition-transform hover:scale-105"
>
<img
src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=1013721&theme=light&t=1757620787136"
alt="Gitea Mirror - Automated github to gitea repository mirroring & backup | Product Hunt"
style={{ width: '250px', height: '54px' }}
width="250"
height="54"
className="dark:hidden"
/>
<img
src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=1013721&theme=dark&t=1757620890723"
alt="Gitea Mirror - Automated github to gitea repository mirroring & backup | Product Hunt"
style={{ width: '250px', height: '54px' }}
width="250"
height="54"
className="hidden dark:block"
/>
</a>
</div>
{/* Call to action buttons */}
<div className="mt-8 sm:mt-10 flex flex-col sm:flex-row items-center justify-center gap-3 sm:gap-4 px-4 z-20">
{/* <div className="mt-8 sm:mt-10 flex flex-col sm:flex-row items-center justify-center gap-3 sm:gap-4 px-4 z-20">
<Button
size="lg"
className="relative group w-full sm:w-auto min-h-[48px] text-base bg-gradient-to-r from-primary to-accent hover:from-primary/90 hover:to-accent/90 shadow-lg shadow-primary/25 hover:shadow-xl hover:shadow-primary/30 transition-all duration-300"
@@ -91,7 +118,7 @@ export function Hero() {
>
<a href="#features">View Features</a>
</Button>
</div>
</div> */}
</div>
</section>
);