Updates for starred and personal repos

This commit is contained in:
Arunavo Ray
2025-07-18 09:37:38 +05:30
parent d388f2e691
commit 3458891511
6 changed files with 117 additions and 16 deletions

102
CLAUDE.md
View File

@@ -2,6 +2,8 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
DONT HALLUCIATE THINGS. IF YOU DONT KNOW LOOK AT THE CODE OR ASK FOR DOCS
## Project Overview
Gitea Mirror is a web application that automatically mirrors repositories from GitHub to self-hosted Gitea instances. It uses Astro for SSR, React for UI, SQLite for data storage, and Bun as the JavaScript runtime.
@@ -40,7 +42,7 @@ bun run start # Start production server
- **Frontend**: Astro (SSR) + React + Tailwind CSS v4 + Shadcn UI
- **Backend**: Bun runtime + SQLite + Drizzle ORM
- **APIs**: GitHub (Octokit) and Gitea APIs
- **Auth**: JWT tokens with bcryptjs password hashing
- **Auth**: Better Auth with email/password, SSO, and OIDC provider support
### Project Structure
- `/src/pages/api/` - API endpoints (Astro API routes)
@@ -68,10 +70,15 @@ export async function POST({ request }: APIContext) {
3. **Real-time Updates**: Server-Sent Events (SSE) endpoint at `/api/events` for live dashboard updates
4. **Authentication Flow**:
4. **Authentication System**:
- Built on Better Auth library
- Three authentication methods:
- Email & Password (traditional auth)
- SSO (authenticate via external OIDC providers)
- OIDC Provider (act as OIDC provider for other apps)
- Session-based authentication with secure cookies
- First user signup creates admin account
- JWT tokens stored in cookies
- Protected routes check auth via `getUserFromCookie()`
- Protected routes use Better Auth session validation
5. **Mirror Process**:
- Discovers repos from GitHub (user/org)
@@ -79,11 +86,18 @@ export async function POST({ request }: APIContext) {
- Tracks status in database
- Supports scheduled automatic mirroring
6. **Mirror Strategies**: Three ways to organize repositories in Gitea:
6. **Mirror Strategies**: Four ways to organize repositories in Gitea:
- **preserve**: Maintains GitHub structure (default)
- Organization repos → Same organization name in Gitea
- Personal repos → Under your Gitea username
- **single-org**: All repos go to one organization
- All repos → Single configured organization
- **flat-user**: All repos go under user account
- Starred repos always go to separate organization (starredReposOrg)
- All repos → Under your Gitea username
- **mixed**: Hybrid approach
- Organization repos → Preserve structure
- Personal repos → Single configured organization
- Starred repos always go to separate organization (starredReposOrg, default: "starred")
- Routing logic in `getGiteaRepoOwner()` function
### Database Schema (SQLite)
@@ -102,11 +116,18 @@ export async function POST({ request }: APIContext) {
### Development Tips
- Environment variables in `.env` (copy from `.env.example`)
- JWT_SECRET auto-generated if not provided
- BETTER_AUTH_SECRET required for session signing
- Database auto-initializes on first run
- Use `bun run dev:clean` for fresh database start
- Tailwind CSS v4 configured with Vite plugin
### Authentication Setup
- **Better Auth** handles all authentication
- Configuration in `/src/lib/auth.ts` (server) and `/src/lib/auth-client.ts` (client)
- Auth endpoints available at `/api/auth/*`
- SSO providers configured through the web UI
- OIDC provider functionality for external applications
### Common Tasks
**Adding a new API endpoint:**
@@ -125,6 +146,73 @@ export async function POST({ request }: APIContext) {
2. Run `bun run init-db` to recreate database
3. Update related queries in `/src/lib/db/queries/`
## Configuration Options
### GitHub Configuration (UI Fields)
#### Basic Settings (`githubConfig`)
- **username**: GitHub username
- **token**: GitHub personal access token (requires repo and admin:org scopes)
- **privateRepositories**: Include private repositories
- **mirrorStarred**: Mirror starred repositories
### Gitea Configuration (UI Fields)
- **url**: Gitea instance URL
- **username**: Gitea username
- **token**: Gitea access token
- **organization**: Destination organization (for single-org/mixed strategies)
- **starredReposOrg**: Organization for starred repositories (default: "starred")
- **visibility**: Organization visibility - "public", "private", "limited"
- **mirrorStrategy**: Repository organization strategy (set via UI)
- **preserveOrgStructure**: Automatically set based on mirrorStrategy
### Schedule Configuration (`scheduleConfig`)
- **enabled**: Enable automatic mirroring (default: false)
- **interval**: Cron expression or seconds (default: "0 2 * * *" - 2 AM daily)
- **concurrent**: Allow concurrent mirror operations (default: false)
- **batchSize**: Number of repos to process in parallel (default: 10)
### Database Cleanup Configuration (`cleanupConfig`)
- **enabled**: Enable automatic cleanup (default: false)
- **retentionDays**: Days to keep events (stored as seconds internally)
### Mirror Options (UI Fields)
- **mirrorReleases**: Mirror GitHub releases to Gitea
- **mirrorMetadata**: Enable metadata mirroring (master toggle)
- **metadataComponents** (only available when mirrorMetadata is enabled):
- **issues**: Mirror issues
- **pullRequests**: Mirror pull requests
- **labels**: Mirror labels
- **milestones**: Mirror milestones
- **wiki**: Mirror wiki content
### Advanced Options (UI Fields)
- **skipForks**: Skip forked repositories (default: false)
- **skipStarredIssues**: Skip issues for starred repositories (default: false) - enables "Lightweight mode" for starred repos
### Authentication Configuration
#### SSO Provider Configuration
- **issuerUrl**: OIDC issuer URL (e.g., https://accounts.google.com)
- **domain**: Email domain for this provider
- **providerId**: Unique identifier for the provider
- **clientId**: OAuth client ID from provider
- **clientSecret**: OAuth client secret from provider
- **authorizationEndpoint**: OAuth authorization URL (auto-discovered if supported)
- **tokenEndpoint**: OAuth token exchange URL (auto-discovered if supported)
- **jwksEndpoint**: JSON Web Key Set URL (optional, auto-discovered)
- **userInfoEndpoint**: User information endpoint (optional, auto-discovered)
#### OIDC Provider Settings (for external apps)
- **allowedRedirectUris**: Comma-separated list of allowed redirect URIs
- **clientId**: Generated client ID for the application
- **clientSecret**: Generated client secret for the application
- **scopes**: Available scopes (openid, profile, email)
#### Environment Variables
- **BETTER_AUTH_SECRET**: Secret key for signing sessions (required)
- **BETTER_AUTH_URL**: Base URL for authentication (default: http://localhost:4321)
## Security Guidelines
- **Confidentiality Guidelines**:

View File

@@ -46,7 +46,7 @@ export function ConfigTabs() {
token: '',
organization: 'github-mirrors',
visibility: 'public',
starredReposOrg: 'github',
starredReposOrg: 'starred',
preserveOrgStructure: false,
},
scheduleConfig: {

View File

@@ -354,6 +354,19 @@ describe("getGiteaRepoOwner - Organization Override Tests", () => {
expect(result).toBe("starred");
});
test("starred repos default to 'starred' org when starredReposOrg is not configured", () => {
const repo = { ...baseRepo, isStarred: true };
const configWithoutStarredOrg = {
...baseConfig,
giteaConfig: {
...baseConfig.giteaConfig,
starredReposOrg: undefined
}
};
const result = getGiteaRepoOwner({ config: configWithoutStarredOrg, repository: repo });
expect(result).toBe("starred");
});
// Removed test for personalReposOrg as this field no longer exists
test("preserve strategy: personal repos fallback to username when no override", () => {

View File

@@ -73,8 +73,8 @@ export const getGiteaRepoOwnerAsync = async ({
}
// Check if repository is starred - starred repos always go to starredReposOrg (highest priority)
if (repository.isStarred && config.giteaConfig.starredReposOrg) {
return config.giteaConfig.starredReposOrg;
if (repository.isStarred) {
return config.giteaConfig.starredReposOrg || "starred";
}
// Check for repository-specific override (second highest priority)
@@ -118,8 +118,8 @@ export const getGiteaRepoOwner = ({
}
// Check if repository is starred - starred repos always go to starredReposOrg
if (repository.isStarred && config.giteaConfig.starredReposOrg) {
return config.giteaConfig.starredReposOrg;
if (repository.isStarred) {
return config.giteaConfig.starredReposOrg || "starred";
}
// Get the mirror strategy - use preserveOrgStructure for backward compatibility
@@ -352,8 +352,8 @@ export const mirrorGithubRepoToGitea = async ({
const apiUrl = `${config.giteaConfig.url}/api/v1/repos/migrate`;
// Handle organization creation if needed for single-org or preserve strategies
if (repoOwner !== config.giteaConfig.defaultOwner && !repository.isStarred) {
// Handle organization creation if needed for single-org, preserve strategies, or starred repos
if (repoOwner !== config.giteaConfig.defaultOwner) {
// Need to create the organization if it doesn't exist
await getOrCreateGiteaOrg({
orgName: repoOwner,

View File

@@ -120,7 +120,7 @@ export function mapDbToUiConfig(dbConfig: any): {
token: dbConfig.giteaConfig?.token || "",
organization: dbConfig.githubConfig?.defaultOrg || "github-mirrors", // Get from GitHub config
visibility: dbConfig.giteaConfig?.visibility === "default" ? "public" : dbConfig.giteaConfig?.visibility || "public",
starredReposOrg: dbConfig.githubConfig?.starredReposOrg || "github", // Get from GitHub config
starredReposOrg: dbConfig.githubConfig?.starredReposOrg || "starred", // Get from GitHub config
preserveOrgStructure: dbConfig.giteaConfig?.preserveVisibility || false, // Map preserveVisibility
mirrorStrategy: dbConfig.githubConfig?.mirrorStrategy || "preserve", // Get from GitHub config
personalReposOrg: undefined, // Not stored in current schema

View File

@@ -200,7 +200,7 @@ export const GET: APIRoute = async ({ request }) => {
includePrivate: false,
includePublic: true,
includeOrganizations: [],
starredReposOrg: "github",
starredReposOrg: "starred",
mirrorStrategy: "preserve",
defaultOrg: "github-mirrors",
},