Added Agents.md

2025-12-06 03:26:44 +03:00 · 2025-10-24 07:57:30 +05:30
parent 985c7e061c
commit 60913a9f4d
2 changed files with 363 additions and 0 deletions
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -0,0 +1,46 @@
+# Repository Guidelines
+
+## Project Structure & Module Organization
+- `src/` – app code
+  - `components/` (React, PascalCase files), `pages/` (Astro/API routes), `lib/` (domain + utilities, kebab-case), `hooks/`, `layouts/`, `styles/`, `tests/`, `types/`, `data/`, `content/`.
+- `scripts/` – operational TS scripts (DB init, recovery): e.g., `scripts/manage-db.ts`.
+- `drizzle/` – SQL migrations; `data/` – runtime SQLite (`gitea-mirror.db`).
+- `public/` – static assets; `dist/` – build output.
+- Key config: `astro.config.mjs`, `tsconfig.json` (alias `@/* → src/*`), `bunfig.toml` (test preload), `.env(.example)`.
+
+## Build, Test, and Development Commands
+- Prereq: Bun `>= 1.2.9` (see `package.json`).
+- Setup: `bun run setup` – install deps and init DB.
+- Dev: `bun run dev` – start Astro dev server.
+- Build: `bun run build` – produce `dist/`.
+- Preview/Start: `bun run preview` (static preview) or `bun run start` (SSR entry).
+- Database: `bun run db:generate|migrate|push|studio` and `bun run manage-db init|check|fix|reset-users`.
+- Tests: `bun test` | `bun run test:watch` | `bun run test:coverage`.
+- Docker: see `docker-compose.yml` and variants in repo root.
+
+## Coding Style & Naming Conventions
+- Language: TypeScript, Astro, React.
+- Indentation: 2 spaces; keep existing semicolon/quote style in touched files.
+- Components: PascalCase `.tsx` in `src/components/` (e.g., `MainLayout.tsx`).
+- Modules/utils: kebab-case in `src/lib/` (e.g., `gitea-enhanced.ts`).
+- Imports: prefer alias `@/…` (configured in `tsconfig.json`).
+- Do not introduce new lint/format configs; follow current patterns.
+
+## Testing Guidelines
+- Runner: Bun test (`bun:test`) with preload `src/tests/setup.bun.ts` (see `bunfig.toml`).
+- Location/Names: `**/*.test.ts(x)` under `src/**` (examples in `src/lib/**`).
+- Scope: add unit tests for new logic and API route tests for handlers.
+- Aim for meaningful coverage on DB, auth, and mirroring paths.
+
+## Commit & Pull Request Guidelines
+- Commits: short, imperative, scoped when helpful (e.g., `lib: fix token parsing`, `ui: align buttons`).
+- PRs must include:
+  - Summary, rationale, and testing steps/commands.
+  - Linked issues (e.g., `Closes #123`).
+  - Screenshots/gifs for UI changes.
+  - Notes on DB/migration or .env impacts; update `docs/`/CHANGELOG if applicable.
+
+## Security & Configuration Tips
+- Never commit secrets. Copy `.env.example` → `.env` and fill values; prefer `bun run startup-env-config` to validate.
+- SQLite files live in `data/`; avoid committing generated DBs.
+- Certificates (if used) reside in `certs/`; manage locally or via Docker secrets.
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -0,0 +1,317 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Gitea Mirror is a self-hosted web application that automatically mirrors repositories from GitHub to Gitea instances. It's built with Astro (SSR mode), React, and runs on the Bun runtime with SQLite for data persistence.
+
+**Key capabilities:**
+- Mirrors public, private, and starred GitHub repos to Gitea
+- Supports metadata mirroring (issues, PRs as issues, labels, milestones, releases, wiki)
+- Git LFS support
+- Multiple authentication methods (email/password, OIDC/SSO, header auth)
+- Scheduled automatic syncing with configurable intervals
+- Auto-discovery of new repos and cleanup of deleted repos
+- Multi-user support with encrypted token storage (AES-256-GCM)
+
+## Development Commands
+
+### Setup and Installation
+```bash
+# Install dependencies
+bun install
+
+# Initialize database (first time setup)
+bun run setup
+
+# Clean start (reset database)
+bun run dev:clean
+```
+
+### Development
+```bash
+# Start development server (http://localhost:4321)
+bun run dev
+
+# Build for production
+bun run build
+
+# Preview production build
+bun run preview
+
+# Start production server
+bun run start
+```
+
+### Testing
+```bash
+# Run all tests
+bun test
+
+# Run tests in watch mode
+bun test:watch
+
+# Run tests with coverage
+bun test:coverage
+```
+
+**Test configuration:**
+- Test runner: Bun's built-in test runner (configured in `bunfig.toml`)
+- Setup file: `src/tests/setup.bun.ts` (auto-loaded via bunfig.toml)
+- Timeout: 5000ms default
+- Tests are colocated with source files using `*.test.ts` pattern
+
+### Database Management
+```bash
+# Database operations via Drizzle
+bun run db:generate    # Generate migrations from schema
+bun run db:migrate     # Run migrations
+bun run db:push        # Push schema changes directly
+bun run db:studio      # Open Drizzle Studio (database GUI)
+bun run db:check       # Check schema consistency
+
+# Database utilities via custom scripts
+bun run manage-db init       # Initialize database
+bun run manage-db check      # Check database health
+bun run manage-db fix        # Fix database issues
+bun run manage-db reset-users # Reset all users
+bun run cleanup-db           # Delete database file
+```
+
+### Utility Scripts
+```bash
+# Recovery and diagnostic scripts
+bun run startup-recovery       # Recover from crashes
+bun run startup-recovery-force # Force recovery
+bun run test-recovery          # Test recovery mechanism
+bun run test-shutdown          # Test graceful shutdown
+
+# Environment configuration
+bun run startup-env-config     # Load config from env vars
+```
+
+## Architecture
+
+### Tech Stack
+- **Frontend:** Astro v5 (SSR mode) + React v19 + Shadcn UI + Tailwind CSS v4
+- **Backend:** Astro API routes (Node adapter, standalone mode)
+- **Runtime:** Bun (>=1.2.9)
+- **Database:** SQLite via Drizzle ORM
+- **Authentication:** Better Auth (session-based)
+- **APIs:** GitHub (Octokit with throttling plugin), Gitea REST API
+
+### Directory Structure
+
+```
+src/
+├── components/          # React components (UI, features)
+│   ├── ui/             # Shadcn UI components
+│   ├── repositories/   # Repository management components
+│   ├── organizations/  # Organization management components
+│   └── ...
+├── pages/              # Astro pages and API routes
+│   ├── api/            # API endpoints (Better Auth integration)
+│   │   ├── auth/       # Authentication endpoints
+│   │   ├── github/     # GitHub operations
+│   │   ├── gitea/      # Gitea operations
+│   │   ├── sync/       # Mirror sync operations
+│   │   ├── job/        # Job management
+│   │   └── ...
+│   └── *.astro         # Page components
+├── lib/                # Core business logic
+│   ├── db/             # Database (Drizzle ORM)
+│   │   ├── schema.ts   # Database schema with Zod validation
+│   │   ├── index.ts    # Database instance and table exports
+│   │   └── adapter.ts  # Better Auth SQLite adapter
+│   ├── github.ts       # GitHub API client (Octokit)
+│   ├── gitea.ts        # Gitea API client
+│   ├── gitea-enhanced.ts # Enhanced Gitea operations (metadata)
+│   ├── scheduler-service.ts # Automatic mirroring scheduler
+│   ├── cleanup-service.ts   # Activity log cleanup
+│   ├── repository-cleanup-service.ts # Orphaned repo cleanup
+│   ├── auth.ts         # Better Auth configuration
+│   ├── config.ts       # Configuration management
+│   ├── helpers.ts      # Mirror job creation
+│   ├── utils/          # Utility functions
+│   │   ├── encryption.ts        # AES-256-GCM token encryption
+│   │   ├── config-encryption.ts # Config token encryption
+│   │   ├── duration-parser.ts   # Parse intervals (e.g., "8h", "30m")
+│   │   ├── concurrency.ts       # Concurrency control utilities
+│   │   └── mirror-strategies.ts # Mirror strategy logic
+│   └── ...
+├── types/              # TypeScript type definitions
+├── tests/              # Test utilities and setup
+└── middleware.ts       # Astro middleware (auth, session)
+
+scripts/                # Utility scripts
+├── manage-db.ts        # Database management CLI
+├── startup-recovery.ts # Crash recovery
+└── ...
+```
+
+### Key Architectural Patterns
+
+#### 1. Database Schema and Validation
+- **Location:** `src/lib/db/schema.ts`
+- **Pattern:** Drizzle ORM tables + Zod schemas for validation
+- **Key tables:**
+  - `configs` - User configuration (GitHub/Gitea settings, mirror options)
+  - `repositories` - Tracked repositories with metadata
+  - `organizations` - GitHub organizations with destination overrides
+  - `mirrorJobs` - Mirror job queue and history
+  - `activities` - Activity log for dashboard
+  - `user`, `session`, `account` - Better Auth tables
+
+**Important:** All config tokens (GitHub/Gitea) are encrypted at rest using AES-256-GCM. Use helper functions from `src/lib/utils/config-encryption.ts` to decrypt.
+
+#### 2. Mirror Job System
+- **Location:** `src/lib/helpers.ts` (createMirrorJob)
+- **Flow:**
+  1. User triggers mirror via API endpoint
+  2. `createMirrorJob()` creates job record with status "pending"
+  3. Job processor (in API routes) performs GitHub → Gitea operations
+  4. Job status updated throughout: "mirroring" → "success"/"failed"
+  5. Events published via SSE for real-time UI updates
+
+#### 3. GitHub ↔ Gitea Mirroring
+- **GitHub Client:** `src/lib/github.ts` - Octokit with rate limit tracking
+- **Gitea Client:** `src/lib/gitea.ts` - Basic repo operations
+- **Enhanced Gitea:** `src/lib/gitea-enhanced.ts` - Metadata mirroring (issues, PRs, releases)
+
+**Mirror strategies (configured per user):**
+- `preserve` - Maintain GitHub org structure in Gitea
+- `single-org` - All repos into one Gitea org
+- `flat-user` - All repos under user account
+- `mixed` - Personal repos in one org, org repos preserve structure
+
+**Metadata mirroring:**
+- Issues transferred with comments, labels, assignees
+- PRs converted to issues (Gitea API limitation - cannot create PRs)
+  - Tagged with "pull-request" label
+  - Title prefixed with `[PR #number] [STATUS]`
+  - Body includes commit history, file changes, merge status
+- Releases mirrored with assets
+- Labels and milestones preserved
+- Wiki content cloned if enabled
+- **Sequential processing:** Issues/PRs mirrored one at a time to prevent out-of-order creation (see `src/lib/gitea-enhanced.ts`)
+
+#### 4. Scheduler Service
+- **Location:** `src/lib/scheduler-service.ts`
+- **Features:**
+  - Cron-based or interval-based scheduling (uses `duration-parser.ts`)
+  - Auto-start on boot when `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` is set
+  - Auto-import new GitHub repos
+  - Auto-cleanup orphaned repos (archive or delete)
+  - Respects per-repo mirror intervals (not Gitea's default 24h)
+- **Concurrency control:** Uses `src/lib/utils/concurrency.ts` for batch processing
+
+#### 5. Authentication System
+- **Location:** `src/lib/auth.ts`, `src/lib/auth-client.ts`
+- **Better Auth integration:**
+  - Email/password (always enabled)
+  - OIDC/SSO providers (configurable via UI)
+  - Header authentication for reverse proxies (Authentik, Authelia)
+- **Session management:** Cookie-based, validated in Astro middleware
+- **User helpers:** `src/lib/utils/auth-helpers.ts`
+
+#### 6. Environment Configuration
+- **Startup:** `src/lib/env-config-loader.ts` + `scripts/startup-env-config.ts`
+- **Pattern:** Environment variables can pre-configure settings, but users can override via web UI
+- **Encryption:** `ENCRYPTION_SECRET` for tokens, `BETTER_AUTH_SECRET` for sessions
+
+#### 7. Real-time Updates
+- **Events:** `src/lib/events.ts` + `src/lib/events/realtime.ts`
+- **Pattern:** Server-Sent Events (SSE) for live dashboard updates
+- **Endpoints:** `/api/sse` - client subscribes to job/repo events
+
+### Testing Patterns
+
+**Unit tests:**
+- Colocated with source: `filename.test.ts` alongside `filename.ts`
+- Use Bun's built-in assertions and mocking
+- Mock external APIs (GitHub, Gitea) using `src/tests/mock-fetch.ts`
+
+**Integration tests:**
+- Located in `src/tests/`
+- Test database operations with in-memory SQLite
+- Example: `src/lib/db/index.test.ts`
+
+**Test utilities:**
+- `src/tests/setup.bun.ts` - Global test setup (loaded via bunfig.toml)
+- `src/tests/mock-fetch.ts` - Fetch mocking utilities
+
+### Important Development Notes
+
+1. **Path Aliases:** Use `@/` for imports (configured in `tsconfig.json`)
+   ```typescript
+   import { db } from '@/lib/db';
+   ```
+
+2. **Token Encryption:** Always use encryption helpers when dealing with tokens:
+   ```typescript
+   import { getDecryptedGitHubToken, getDecryptedGiteaToken } from '@/lib/utils/config-encryption';
+   ```
+
+3. **API Route Pattern:** Astro API routes in `src/pages/api/` should:
+   - Check authentication via Better Auth
+   - Validate input with Zod schemas
+   - Handle errors gracefully
+   - Return JSON responses
+
+4. **Database Migrations:**
+   - Schema changes: Update `src/lib/db/schema.ts`
+   - Generate migration: `bun run db:generate`
+   - Review generated SQL in `drizzle/` directory
+   - Apply: `bun run db:migrate` (or `db:push` for dev)
+
+5. **Concurrency Control:**
+   - Use utilities from `src/lib/utils/concurrency.ts` for batch operations
+   - Respect rate limits (GitHub: 5000 req/hr authenticated, Gitea: varies)
+   - Issue/PR mirroring is sequential to maintain chronological order
+
+6. **Duration Parsing:**
+   - Use `parseInterval()` from `src/lib/utils/duration-parser.ts`
+   - Supports: "30m", "8h", "24h", "7d", cron expressions, or milliseconds
+
+7. **Graceful Shutdown:**
+   - Services implement cleanup handlers (see `src/lib/shutdown-manager.ts`)
+   - Recovery system in `src/lib/recovery.ts` handles interrupted jobs
+
+## Common Development Workflows
+
+### Adding a new mirror option
+1. Update Zod schema in `src/lib/db/schema.ts` (e.g., `giteaConfigSchema`)
+2. Update TypeScript types in `src/types/config.ts`
+3. Add UI control in settings page component
+4. Update API handler in `src/pages/api/config/`
+5. Implement logic in `src/lib/gitea.ts` or `src/lib/gitea-enhanced.ts`
+
+### Debugging mirror failures
+1. Check mirror jobs: `bun run db:studio` → `mirrorJobs` table
+2. Review activity logs: Dashboard → Activity tab
+3. Check console logs for API errors (GitHub/Gitea rate limits, auth issues)
+4. Use diagnostic scripts: `bun run test-recovery`
+
+### Adding authentication provider
+1. Update Better Auth config in `src/lib/auth.ts`
+2. Add provider configuration UI in settings
+3. Test with `src/tests/test-gitea-auth.ts` patterns
+4. Update documentation in `docs/SSO-OIDC-SETUP.md`
+
+## Docker Deployment
+
+- **Dockerfile:** Multi-stage build (bun base → build → production)
+- **Entrypoint:** `docker-entrypoint.sh` - handles CA certs, user permissions, database init
+- **Compose files:**
+  - `docker-compose.alt.yml` - Quick start (pre-built image, minimal config)
+  - `docker-compose.yml` - Full setup (build from source, all env vars)
+  - `docker-compose.dev.yml` - Development with hot reload
+
+## Additional Resources
+
+- **Environment Variables:** See `docs/ENVIRONMENT_VARIABLES.md` for complete list
+- **Development Workflow:** See `docs/DEVELOPMENT_WORKFLOW.md`
+- **SSO Setup:** See `docs/SSO-OIDC-SETUP.md`
+- **Contributing:** See `CONTRIBUTING.md` for code guidelines and scope
+- **Graceful Shutdown:** See `docs/GRACEFUL_SHUTDOWN.md` for crash recovery details