Commit Graph

451 Commits

Author SHA1 Message Date
Arunavo Ray
2ea917fdaa fix: resolve migration 0005 duplicate constraint failure (#97)
**Problem:**
- Users upgrading to v3.7.2 encountered database migration failures
- Migration 0005 tried to add unique index without handling existing duplicates
- Hybrid initialization (manual SQL + Drizzle) caused schema inconsistencies
- Error: "UNIQUE constraint failed: repositories.user_id, repositories.full_name"

**Solution:**

1. **Fixed Migration 0005:**
   - Added deduplication step before creating unique index
   - Removes duplicate (user_id, full_name) entries, keeping most recent
   - Safely creates unique constraint after cleanup

2. **Removed Hybrid Database Initialization:**
   - Eliminated 154 lines of manual SQL from docker-entrypoint.sh
   - Now uses Drizzle exclusively for schema management
   - Single source of truth prevents schema drift
   - Migrations run automatically via src/lib/db/index.ts

**Testing:**
-  Fresh database initialization works
-  Duplicate deduplication verified
-  Unique constraint properly enforced
-  All 6 migrations apply cleanly

**Changes:**
- docker-entrypoint.sh: Removed manual table creation SQL
- drizzle/0005_polite_preak.sql: Added deduplication before index creation

Fixes #97
2025-10-01 07:57:16 +05:30
Arunavo Ray
b841057f1a updated packages v3.8.0 2025-10-01 07:29:39 +05:30
ARUNAVO RAY
d588ce91b4 Merge pull request #106 from RayLabsHQ/alert-autofix-33
Potential fix for code scanning alert no. 33: Workflow does not contain permissions
2025-10-01 07:06:58 +05:30
ARUNAVO RAY
553396483e Potential fix for code scanning alert no. 33: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-10-01 07:06:06 +05:30
ARUNAVO RAY
ebeabdb4fc Merge pull request #105 from RayLabsHQ/fix/forgejo-12-private-repos
fix: Forgejo 12 compatibility - use separate auth fields for private repos (#102)
2025-10-01 07:02:30 +05:30
ARUNAVO RAY
ff209a6376 Merge pull request #101 from V-Paranoiaque/helm-chart
Helm chart to deploy on Kubernetes
2025-10-01 07:01:19 +05:30
Arunavo Ray
096e0c03ac images now tagged lowercase in comments 2025-09-30 23:43:27 +05:30
Virgil R.
63f20a7f04 Update helm-test.yml 2025-09-30 20:07:48 +02:00
Arunavo Ray
34f741beef fix: Forgejo 12 compatibility - use separate auth fields for private repos (#102)
## Problem
Forgejo 12.0+ rejects migration API calls with credentials embedded in URLs,
causing HTTP 422 errors when mirroring private GitHub repositories.

## Root Cause
Breaking security change in Forgejo 12.0 (July 2025) enforces credential
separation to prevent accidental exposure in logs/errors. Previous versions
(Forgejo 11.x, Gitea 1.x) accepted embedded credentials.

## Solution
- Use separate `auth_username` and `auth_token` fields instead of embedding
  credentials in clone URLs
- Set `auth_username` to "oauth2" for GitHub token authentication
- Pass GitHub token via `auth_token` field

## Changes
- src/lib/gitea.ts:
  - mirrorGithubRepoToGitea(): Use separate auth fields for private repos
  - mirrorGitHubRepoToGiteaOrg(): Use separate auth fields for private repos

- .github/workflows/docker-build.yml:
  - Enable PR image building and pushing to GHCR
  - Tag PR images as pr-<number> for easy testing
  - Add automated PR comment with image details and testing instructions
  - Separate load step for security scanning

## Backward Compatibility
 Works with Forgejo 12.0+
 Works with Forgejo 11.x and earlier
 Works with Gitea 1.x

## Testing
Public repos:  Working (no auth needed)
Private repos:  Fixed (separate auth fields)

Fixes #102
2025-09-30 23:12:33 +05:30
V-Paranoiaque
1f98f441f3 Fix ingress + improve testing 2025-09-27 18:28:10 +02:00
V-Paranoiaque
9c1ac76ff9 Fix annotations 2025-09-27 15:18:52 +02:00
V-Paranoiaque
cf5027bafc Fix CLEANUP_RETENTION_DAYS 2025-09-27 15:15:42 +02:00
V-Paranoiaque
6fd2774d43 Fix MIRROR_STARRED var 2025-09-27 15:13:54 +02:00
V-Paranoiaque
8f379baad4 Improve CI/CD 2025-09-27 10:34:20 +02:00
V-Paranoiaque
91fa3604b6 Add some basic CICD for testing 2025-09-27 10:24:18 +02:00
Virgil R.
c0fff30fcb Create README.md 2025-09-27 10:12:29 +02:00
Virgil R.
18de63d192 Update deployment.yaml 2025-09-27 10:04:42 +02:00
V-Paranoiaque
1fe20c3e54 GITHUB_TYPE env var 2025-09-24 22:07:15 +02:00
V-Paranoiaque
7386b54a46 Fix env vars 2025-09-24 21:12:16 +02:00
V-Paranoiaque
432a2bc54d Add missing podSecurityContext 2025-09-24 20:47:54 +02:00
V-Paranoiaque
f9d18f34ab Fix image tag 2025-09-24 20:41:24 +02:00
V-Paranoiaque
cd86a09bbd Minor fixes 2025-09-24 20:29:14 +02:00
V-Paranoiaque
1e2c1c686d Add volumes 2025-09-21 21:15:50 +02:00
V-Paranoiaque
f701574e67 Move NODE_ENV to the configmap 2025-09-21 15:15:58 +02:00
V-Paranoiaque
4528be8cc6 Too many spaces 2025-09-21 12:29:02 +02:00
V-Paranoiaque
80fd43ef42 More features 2025-09-21 12:26:26 +02:00
V-Paranoiaque
3c52fe58aa Missing space 2025-09-21 10:23:48 +02:00
V-Paranoiaque
319e7925ff First commit TBT 2025-09-20 22:59:22 +02:00
Arunavo Ray
5add8766a4 fix(scheduler,config): preserve ENV schedule; add AUTO_MIRROR_REPOS auto-mirroring
- Prevent Automation UI from overriding schedule:
      - mapDbScheduleToUi now parses intervals robustly (cron/duration/seconds) via parseInterval
      - mapUiScheduleToDb merges with existing config and stores interval as seconds (no lossy cron conversion)
      - /api/config passes existing scheduleConfig to preserve ENV-sourced values
      - schedule-sync endpoint uses parseInterval for nextRun calculation
  - Add AUTO_MIRROR_REPOS support and scheduled auto-mirror phase:
      - scheduleConfig schema includes autoImport and autoMirror
      - env-config-loader reads AUTO_MIRROR_REPOS and carries through to DB
      - scheduler auto-mirrors imported/pending/failed repos when autoMirror is enabled before regular sync
      - docker-compose and ENV docs updated with AUTO_MIRROR_REPOS
  - Tests pass and build succeeds
v3.7.2
2025-09-14 08:31:31 +05:30
Arunavo Ray
6ce70bb5bf chore(version): bump to 3.7.1\n\ncleanup: attempt fix for orphaned repo archiving (refs #84)\n- Sanitize mirror rename to satisfy AlphaDashDot; timestamped fallback\n- Resolve Gitea owner robustly via mirroredLocation/strategy; verify presence\n- Add 'archived' status to Zod enums; set isArchived on archive\n- Update CHANGELOG entry without closing keyword v3.7.1 2025-09-14 07:53:36 +05:30
Arunavo Ray
f3aae2ec94 fix for repo name collison v3.7.0 2025-09-14 00:13:13 +05:30
Arunavo Ray
46d5ec46fc Updated deisgn for 'Duplicate collision strategy' 2025-09-13 23:54:14 +05:30
Arunavo Ray
0caa53b67f v3.7.0 2025-09-13 23:39:50 +05:30
Arunavo Ray
18ecdbc252 fix(sync): batch inserts + normalize nulls to avoid SQLite param mismatch
- Batch repository inserts with dynamic sizing under SQLite 999-param limit
- Normalize undefined → null to keep multi-row insert shapes consistent
- De-duplicate owned + starred repos by fullName (prefer starred variant)
- Enforce uniqueness via (user_id, full_name) + onConflictDoNothing
- Handle starred name collisions (suffix/prefix) across mirror + metadata
- Add repo-utils helpers + tests; guard Octokit.plugin in tests
- Remove manual unique index from entrypoint; rely on drizzle-kit migrations
2025-09-13 23:38:50 +05:30
Arunavo Ray
51a6c8ca58 Added product hunt badge on website 2025-09-12 01:44:13 +05:30
Arunavo Ray
41b8806268 update packages 2025-09-10 09:49:08 +05:30
ARUNAVO RAY
ac5c7800c1 Merge pull request #93 from RayLabsHQ/dependabot/npm_and_yarn/www/npm_and_yarn-73ea615029
Bump vite from 6.3.5 to 6.3.6 in /www in the npm_and_yarn group across 1 directory
2025-09-10 09:46:02 +05:30
dependabot[bot]
13e7661f07 Bump vite in /www in the npm_and_yarn group across 1 directory
Bumps the npm_and_yarn group with 1 update in the /www directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 6.3.5 to 6.3.6
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.3.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.3.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.3.6
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-09-10 02:49:30 +00:00
Arunavo Ray
37e5b68bd5 Added Github API rate limiting
- Implemented comprehensive GitHub API rate limit handling:
    - Integrated @octokit/plugin-throttling for automatic retry with exponential backoff
    - Added RateLimitManager service to track and enforce rate limits
    - Store rate limit status in database for persistence across restarts
    - Automatic pause and resume when limits are exceeded
    - Proper user identification for 5000 req/hr authenticated limit (vs 60 unauthenticated)

  - Improved rate limit UI/UX:
    - Removed intrusive rate limit card from dashboard
    - Toast notifications only at critical thresholds (80% and 100% usage)
    - All rate limit events logged for debugging

  - Optimized for GitHub's API constraints:
    - Reduced default batch size from 10 to 5 repositories
    - Added documentation about GitHub's 100 concurrent request limit
    - Better handling of repositories with many issues/PRs
v3.6.0
2025-09-09 11:14:43 +05:30
Arunavo Ray
89ca5abe7d fix: resolve SQLite field mismatch for large starred repo imports (#90)
- Add missing database fields (language, description, mirroredLocation, destinationOrg) to repository operations
  - Add missing organization fields (publicRepositoryCount, privateRepositoryCount, forkRepositoryCount) to schema
  - Update GitRepo interface to include all required database fields
  - Fix GitHub data fetching functions to map all fields correctly
  - Update all sync endpoints (main, repository, organization, scheduler) to handle new fields

  This fixes the "SQLite query expected X values, received Y" error when importing
  large numbers (4.6k+) of starred repositories by ensuring all database fields
  are properly mapped from GitHub API responses through to database insertion.
2025-09-09 09:56:18 +05:30
Arunavo Ray
2b78a6a4a8 v3.5.4 v3.5.4 2025-09-07 19:11:50 +05:30
Arunavo Ray
c2f6e73054 Testing Authentik SSO Issues 2025-09-07 19:09:00 +05:30
Arunavo Ray
c4b353aae8 Added docs around scheduling using corn 2025-09-07 16:51:51 +05:30
Arunavo Ray
4a54cf9009 v3.5.3 v3.5.3 2025-09-07 16:29:43 +05:30
Arunavo Ray
fab4efd93a Auto-start on boot 2025-09-07 16:29:23 +05:30
Arunavo Ray
9f21cd6b1a Addressing concerns of Issue #85 and #86 2025-09-07 15:25:48 +05:30
Arunavo Ray
9ef6017a23 v3.5.2 v3.5.2 2025-09-07 13:55:43 +05:30
Arunavo Ray
502796371f Attempt to address #84 2025-09-07 13:55:20 +05:30
Arunavo Ray
b956b71c5f Fixed #87 where the Release Notes was missing 2025-09-07 13:14:41 +05:30
Arunavo Ray
26b82e0f65 Added AGENTS.md 2025-09-07 11:46:14 +05:30