Compare commits
23 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e24b856416 | ||
|
|
612805f030 | ||
|
|
7705dffee0 | ||
|
|
3dceb34174 | ||
|
|
6b747ba891 | ||
|
|
ddd67faeab | ||
|
|
832b57538d | ||
|
|
415bff8e41 | ||
|
|
13c3ddea04 | ||
|
|
b917b30830 | ||
|
|
b34ed5595b | ||
|
|
cbc11155ef | ||
|
|
941f61830f | ||
|
|
5b60cffaae | ||
|
|
ede5b4dbe8 | ||
|
|
99336e2607 | ||
|
|
cba421d606 | ||
|
|
c4b9a82806 | ||
|
|
38e0fb33b9 | ||
|
|
22a4b71653 | ||
|
|
52568eda36 | ||
|
|
a84191f0a5 | ||
|
|
33829eda20 |
BIN
.github/assets/activity.png
vendored
|
Before Width: | Height: | Size: 140 KiB After Width: | Height: | Size: 816 KiB |
BIN
.github/assets/configuration.png
vendored
|
Before Width: | Height: | Size: 164 KiB After Width: | Height: | Size: 945 KiB |
BIN
.github/assets/dashboard.png
vendored
|
Before Width: | Height: | Size: 194 KiB After Width: | Height: | Size: 943 KiB |
BIN
.github/assets/organisations.png
vendored
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 784 KiB |
BIN
.github/assets/repositories.png
vendored
|
Before Width: | Height: | Size: 170 KiB After Width: | Height: | Size: 970 KiB |
12
CHANGELOG.md
@@ -5,6 +5,18 @@ All notable changes to the Gitea Mirror project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [2.12.0] - 2025-01-27
|
||||
|
||||
### Fixed
|
||||
- Fixed SQLite "no such table: mirror_jobs" error during application startup
|
||||
- Implemented automatic database table creation during database initialization
|
||||
- Resolved database schema inconsistencies between development and production environments
|
||||
|
||||
### Improved
|
||||
- Enhanced database initialization process with automatic table creation and indexing
|
||||
- Added comprehensive error handling for database table creation
|
||||
- Integrated database repair functionality into application startup for better reliability
|
||||
|
||||
## [2.5.3] - 2025-05-22
|
||||
|
||||
### Added
|
||||
|
||||
74
README.md
@@ -30,7 +30,7 @@ sudo LOCAL_REPO_DIR=~/Development/gitea-mirror ./scripts/gitea-mirror-lxc-local.
|
||||
See the [LXC Container Deployment Guide](scripts/README-lxc.md).
|
||||
|
||||
<p align="center">
|
||||
<img src=".github/assets/dashboard.png" alt="Dashboard" width="80%"/>
|
||||
<img src=".github/assets/dashboard.png" alt="Dashboard" width="full"/>
|
||||
</p>
|
||||
|
||||
## ✨ Features
|
||||
@@ -50,12 +50,12 @@ See the [LXC Container Deployment Guide](scripts/README-lxc.md).
|
||||
## 📸 Screenshots
|
||||
|
||||
<p align="center">
|
||||
<img src=".github/assets/repositories.png" width="45%"/>
|
||||
<img src=".github/assets/organisations.png" width="45%"/>
|
||||
<img src=".github/assets/repositories.png" width="49%"/>
|
||||
<img src=".github/assets/organisations.png" width="49%"/>
|
||||
</p>
|
||||
<p align="center">
|
||||
<img src=".github/assets/configuration.png" width="45%"/>
|
||||
<img src=".github/assets/activity.png" width="45%"/>
|
||||
<img src=".github/assets/configuration.png" width="49%"/>
|
||||
<img src=".github/assets/activity.png" width="49%"/>
|
||||
</p>
|
||||
|
||||
### Dashboard
|
||||
@@ -69,7 +69,7 @@ Easily configure your GitHub and Gitea connections, set up automatic mirroring s
|
||||
|
||||
## Getting Started
|
||||
|
||||
See the [Quick Start Guide](docs/quickstart.md) for detailed instructions on getting up and running quickly.
|
||||
See the [Quick Start Guide](src/content/docs/quickstart.md) for detailed instructions on getting up and running quickly.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
@@ -282,9 +282,39 @@ bun run reset-users
|
||||
bun run check-db
|
||||
```
|
||||
|
||||
##### Database Permissions for Direct Installation
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **If you're running the application directly** (not using Docker), you may encounter SQLite permission errors. This is because SQLite requires both read/write access to the database file and write access to the directory containing the database.
|
||||
|
||||
**Common Error:**
|
||||
```
|
||||
Error: [ERROR] SQLiteError: attempt to write a readonly database
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Ensure the data directory exists and has proper permissions
|
||||
mkdir -p data
|
||||
chmod 755 data
|
||||
|
||||
# If the database file already exists, ensure it's writable
|
||||
chmod 644 data/gitea-mirror.db
|
||||
|
||||
# Make sure the user running the application owns the data directory
|
||||
chown -R $(whoami) data/
|
||||
```
|
||||
|
||||
**Why Docker doesn't have this issue:**
|
||||
- Docker containers run with a dedicated user (`gitea-mirror`) that owns the `/app/data` directory
|
||||
- The container setup ensures proper permissions are set during image build
|
||||
- Volume mounts are handled by Docker with appropriate permissions
|
||||
|
||||
**Recommended approach:** Use Docker or Docker Compose for deployment to avoid permission issues entirely.
|
||||
|
||||
### Configuration
|
||||
|
||||
Gitea Mirror can be configured through environment variables or through the web UI. See the [Configuration Guide](docs/configuration.md) for more details.
|
||||
Gitea Mirror can be configured through environment variables or through the web UI. See the [Configuration Guide](src/content/docs/configuration.md) for more details.
|
||||
|
||||
Key configuration options include:
|
||||
|
||||
@@ -483,6 +513,36 @@ Try the following steps:
|
||||
>
|
||||
> This setup provides a complete containerized deployment for the Gitea Mirror application.
|
||||
|
||||
#### Docker Volume Types and Permissions
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **Named Volumes vs Bind Mounts**: If you encounter SQLite permission errors even when using Docker, check your volume configuration:
|
||||
|
||||
**✅ Named Volumes (Recommended):**
|
||||
```yaml
|
||||
volumes:
|
||||
- gitea-mirror-data:/app/data # Docker manages permissions automatically
|
||||
```
|
||||
|
||||
**⚠️ Bind Mounts (Requires Manual Permission Setup):**
|
||||
```yaml
|
||||
volumes:
|
||||
- /host/path/to/data:/app/data # Host filesystem permissions apply
|
||||
```
|
||||
|
||||
**If using bind mounts**, ensure the host directory is owned by UID 1001 (the `gitea-mirror` user):
|
||||
```bash
|
||||
# Set correct ownership for bind mount
|
||||
sudo chown -R 1001:1001 /host/path/to/data
|
||||
sudo chmod -R 755 /host/path/to/data
|
||||
```
|
||||
|
||||
**Why named volumes work better:**
|
||||
- Docker automatically handles permissions
|
||||
- Better portability across different hosts
|
||||
- No manual permission setup required
|
||||
- Used by our official docker-compose.yml
|
||||
|
||||
|
||||
#### Database Maintenance
|
||||
|
||||
|
||||
@@ -232,6 +232,28 @@ else
|
||||
echo "❌ Startup recovery failed with exit code $RECOVERY_EXIT_CODE"
|
||||
fi
|
||||
|
||||
# Run repository status repair to fix any inconsistent mirroring states
|
||||
echo "Running repository status repair..."
|
||||
if [ -f "dist/scripts/repair-mirrored-repos.js" ]; then
|
||||
echo "Running repository repair using compiled script..."
|
||||
bun dist/scripts/repair-mirrored-repos.js --startup
|
||||
REPAIR_EXIT_CODE=$?
|
||||
elif [ -f "scripts/repair-mirrored-repos.ts" ]; then
|
||||
echo "Running repository repair using TypeScript script..."
|
||||
bun scripts/repair-mirrored-repos.ts --startup
|
||||
REPAIR_EXIT_CODE=$?
|
||||
else
|
||||
echo "Warning: Repository repair script not found. Skipping repair."
|
||||
REPAIR_EXIT_CODE=0
|
||||
fi
|
||||
|
||||
# Log repair result
|
||||
if [ $REPAIR_EXIT_CODE -eq 0 ]; then
|
||||
echo "✅ Repository status repair completed successfully"
|
||||
else
|
||||
echo "⚠️ Repository status repair completed with warnings (exit code $REPAIR_EXIT_CODE)"
|
||||
fi
|
||||
|
||||
# Function to handle shutdown signals
|
||||
shutdown_handler() {
|
||||
echo "🛑 Received shutdown signal, forwarding to application..."
|
||||
|
||||
12
package.json
@@ -1,19 +1,18 @@
|
||||
{
|
||||
"name": "gitea-mirror",
|
||||
"type": "module",
|
||||
"version": "2.9.0",
|
||||
"version": "2.12.0",
|
||||
"engines": {
|
||||
"bun": ">=1.2.9"
|
||||
},
|
||||
"scripts": {
|
||||
"setup": "bun install && bun run manage-db init && bun run update-db",
|
||||
"setup": "bun install && bun run manage-db init",
|
||||
"dev": "bunx --bun astro dev",
|
||||
"dev:clean": "bun run cleanup-db && bun run manage-db init && bun run update-db && bunx --bun astro dev",
|
||||
"dev:clean": "bun run cleanup-db && bun run manage-db init && bunx --bun astro dev",
|
||||
"build": "bunx --bun astro build",
|
||||
"cleanup-db": "rm -f gitea-mirror.db data/gitea-mirror.db",
|
||||
"manage-db": "bun scripts/manage-db.ts",
|
||||
"init-db": "bun scripts/manage-db.ts init",
|
||||
"update-db": "bun scripts/update-mirror-jobs-table.ts",
|
||||
"check-db": "bun scripts/manage-db.ts check",
|
||||
"fix-db": "bun scripts/manage-db.ts fix",
|
||||
"reset-users": "bun scripts/manage-db.ts reset-users",
|
||||
@@ -26,7 +25,7 @@
|
||||
"test-shutdown-cleanup": "bun scripts/test-graceful-shutdown.ts --cleanup",
|
||||
"preview": "bunx --bun astro preview",
|
||||
"start": "bun dist/server/entry.mjs",
|
||||
"start:fresh": "bun run cleanup-db && bun run manage-db init && bun run update-db && bun dist/server/entry.mjs",
|
||||
"start:fresh": "bun run cleanup-db && bun run manage-db init && bun dist/server/entry.mjs",
|
||||
"test": "bun test",
|
||||
"test:watch": "bun test --watch",
|
||||
"test:coverage": "bun test --coverage",
|
||||
@@ -55,7 +54,6 @@
|
||||
"@types/react": "^19.1.4",
|
||||
"@types/react-dom": "^19.1.5",
|
||||
"astro": "^5.7.13",
|
||||
"axios": "^1.9.0",
|
||||
"bcryptjs": "^3.0.2",
|
||||
"canvas-confetti": "^1.9.3",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
@@ -70,7 +68,6 @@
|
||||
"react-dom": "^19.1.0",
|
||||
"react-icons": "^5.5.0",
|
||||
"sonner": "^2.0.3",
|
||||
"superagent": "^10.2.1",
|
||||
"tailwind-merge": "^3.3.0",
|
||||
"tailwindcss": "^4.1.7",
|
||||
"tw-animate-css": "^1.3.0",
|
||||
@@ -82,7 +79,6 @@
|
||||
"@testing-library/react": "^16.3.0",
|
||||
"@types/bcryptjs": "^3.0.0",
|
||||
"@types/jsonwebtoken": "^9.0.9",
|
||||
"@types/superagent": "^8.1.9",
|
||||
"@types/uuid": "^10.0.0",
|
||||
"@vitejs/plugin-react": "^4.4.1",
|
||||
"jsdom": "^26.1.0",
|
||||
|
||||
|
Before Width: | Height: | Size: 749 B After Width: | Height: | Size: 21 KiB |
200
public/logo.svg
Normal file
|
After Width: | Height: | Size: 21 KiB |
129
scripts/cleanup-duplicate-repos.ts
Normal file
@@ -0,0 +1,129 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Script to find and clean up duplicate repositories in the database
|
||||
* Keeps the most recent entry and removes older duplicates
|
||||
*
|
||||
* Usage: bun scripts/cleanup-duplicate-repos.ts [--dry-run] [--repo-name=<name>]
|
||||
*/
|
||||
|
||||
import { db, repositories, mirrorJobs } from "@/lib/db";
|
||||
import { eq, and, desc } from "drizzle-orm";
|
||||
|
||||
const isDryRun = process.argv.includes("--dry-run");
|
||||
const specificRepo = process.argv.find(arg => arg.startsWith("--repo-name="))?.split("=")[1];
|
||||
|
||||
async function findDuplicateRepositories() {
|
||||
console.log("🔍 Finding duplicate repositories");
|
||||
console.log("=" .repeat(40));
|
||||
|
||||
if (isDryRun) {
|
||||
console.log("🔍 DRY RUN MODE - No changes will be made");
|
||||
console.log("");
|
||||
}
|
||||
|
||||
if (specificRepo) {
|
||||
console.log(`🎯 Targeting specific repository: ${specificRepo}`);
|
||||
console.log("");
|
||||
}
|
||||
|
||||
try {
|
||||
// Find all repositories, grouped by name and fullName
|
||||
let allRepos = await db.select().from(repositories);
|
||||
|
||||
if (specificRepo) {
|
||||
allRepos = allRepos.filter(repo => repo.name === specificRepo);
|
||||
}
|
||||
|
||||
// Group repositories by name and fullName
|
||||
const repoGroups = new Map<string, typeof allRepos>();
|
||||
|
||||
for (const repo of allRepos) {
|
||||
const key = `${repo.name}|${repo.fullName}`;
|
||||
if (!repoGroups.has(key)) {
|
||||
repoGroups.set(key, []);
|
||||
}
|
||||
repoGroups.get(key)!.push(repo);
|
||||
}
|
||||
|
||||
// Find groups with duplicates
|
||||
const duplicateGroups = Array.from(repoGroups.entries())
|
||||
.filter(([_, repos]) => repos.length > 1);
|
||||
|
||||
if (duplicateGroups.length === 0) {
|
||||
console.log("✅ No duplicate repositories found");
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`📋 Found ${duplicateGroups.length} sets of duplicate repositories:`);
|
||||
console.log("");
|
||||
|
||||
let totalDuplicates = 0;
|
||||
let totalRemoved = 0;
|
||||
|
||||
for (const [key, repos] of duplicateGroups) {
|
||||
const [name, fullName] = key.split("|");
|
||||
console.log(`🔄 Processing duplicates for: ${name} (${fullName})`);
|
||||
console.log(` Found ${repos.length} entries:`);
|
||||
|
||||
// Sort by updatedAt descending to keep the most recent
|
||||
repos.sort((a, b) => new Date(b.updatedAt).getTime() - new Date(a.updatedAt).getTime());
|
||||
|
||||
const keepRepo = repos[0];
|
||||
const removeRepos = repos.slice(1);
|
||||
|
||||
console.log(` ✅ Keeping: ID ${keepRepo.id} (Status: ${keepRepo.status}, Updated: ${new Date(keepRepo.updatedAt).toISOString()})`);
|
||||
|
||||
for (const repo of removeRepos) {
|
||||
console.log(` ❌ Removing: ID ${repo.id} (Status: ${repo.status}, Updated: ${new Date(repo.updatedAt).toISOString()})`);
|
||||
|
||||
if (!isDryRun) {
|
||||
try {
|
||||
// First, delete related mirror jobs
|
||||
await db
|
||||
.delete(mirrorJobs)
|
||||
.where(eq(mirrorJobs.repositoryId, repo.id!));
|
||||
|
||||
// Then delete the repository
|
||||
await db
|
||||
.delete(repositories)
|
||||
.where(eq(repositories.id, repo.id!));
|
||||
|
||||
console.log(` 🗑️ Deleted repository and related mirror jobs`);
|
||||
totalRemoved++;
|
||||
} catch (error) {
|
||||
console.log(` ❌ Error deleting repository: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
} else {
|
||||
console.log(` 🗑️ Would delete repository and related mirror jobs`);
|
||||
totalRemoved++;
|
||||
}
|
||||
}
|
||||
|
||||
totalDuplicates += removeRepos.length;
|
||||
console.log("");
|
||||
}
|
||||
|
||||
console.log("📊 Cleanup Summary:");
|
||||
console.log(` Duplicate sets found: ${duplicateGroups.length}`);
|
||||
console.log(` Total duplicates: ${totalDuplicates}`);
|
||||
console.log(` ${isDryRun ? 'Would remove' : 'Removed'}: ${totalRemoved}`);
|
||||
|
||||
if (isDryRun && totalRemoved > 0) {
|
||||
console.log("");
|
||||
console.log("💡 To apply these changes, run the script without --dry-run");
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error("❌ Error during cleanup process:", error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the cleanup
|
||||
findDuplicateRepositories().then(() => {
|
||||
console.log("Cleanup process complete.");
|
||||
process.exit(0);
|
||||
}).catch((error) => {
|
||||
console.error("Fatal error:", error);
|
||||
process.exit(1);
|
||||
});
|
||||
178
scripts/investigate-repo.ts
Normal file
@@ -0,0 +1,178 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Script to investigate a specific repository's mirroring status
|
||||
* Usage: bun scripts/investigate-repo.ts [repository-name]
|
||||
*/
|
||||
|
||||
import { db, repositories, mirrorJobs, configs } from "@/lib/db";
|
||||
import { eq, desc, and } from "drizzle-orm";
|
||||
|
||||
const repoName = process.argv[2] || "EruditionPaper";
|
||||
|
||||
async function investigateRepository() {
|
||||
console.log(`🔍 Investigating repository: ${repoName}`);
|
||||
console.log("=" .repeat(50));
|
||||
|
||||
try {
|
||||
// Find the repository in the database
|
||||
const repos = await db
|
||||
.select()
|
||||
.from(repositories)
|
||||
.where(eq(repositories.name, repoName));
|
||||
|
||||
if (repos.length === 0) {
|
||||
console.log(`❌ Repository "${repoName}" not found in database`);
|
||||
return;
|
||||
}
|
||||
|
||||
const repo = repos[0];
|
||||
console.log(`✅ Found repository: ${repo.name}`);
|
||||
console.log(` ID: ${repo.id}`);
|
||||
console.log(` Full Name: ${repo.fullName}`);
|
||||
console.log(` Owner: ${repo.owner}`);
|
||||
console.log(` Organization: ${repo.organization || "None"}`);
|
||||
console.log(` Status: ${repo.status}`);
|
||||
console.log(` Is Private: ${repo.isPrivate}`);
|
||||
console.log(` Is Forked: ${repo.isForked}`);
|
||||
console.log(` Mirrored Location: ${repo.mirroredLocation || "Not set"}`);
|
||||
console.log(` Last Mirrored: ${repo.lastMirrored ? new Date(repo.lastMirrored).toISOString() : "Never"}`);
|
||||
console.log(` Error Message: ${repo.errorMessage || "None"}`);
|
||||
console.log(` Created At: ${new Date(repo.createdAt).toISOString()}`);
|
||||
console.log(` Updated At: ${new Date(repo.updatedAt).toISOString()}`);
|
||||
|
||||
console.log("\n📋 Recent Mirror Jobs:");
|
||||
console.log("-".repeat(30));
|
||||
|
||||
// Find recent mirror jobs for this repository
|
||||
const jobs = await db
|
||||
.select()
|
||||
.from(mirrorJobs)
|
||||
.where(eq(mirrorJobs.repositoryId, repo.id))
|
||||
.orderBy(desc(mirrorJobs.timestamp))
|
||||
.limit(10);
|
||||
|
||||
if (jobs.length === 0) {
|
||||
console.log(" No mirror jobs found for this repository");
|
||||
} else {
|
||||
jobs.forEach((job, index) => {
|
||||
console.log(` ${index + 1}. ${new Date(job.timestamp).toISOString()}`);
|
||||
console.log(` Status: ${job.status}`);
|
||||
console.log(` Message: ${job.message}`);
|
||||
if (job.details) {
|
||||
console.log(` Details: ${job.details}`);
|
||||
}
|
||||
console.log("");
|
||||
});
|
||||
}
|
||||
|
||||
// Get user configuration
|
||||
console.log("⚙️ User Configuration:");
|
||||
console.log("-".repeat(20));
|
||||
|
||||
const config = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.id, repo.configId))
|
||||
.limit(1);
|
||||
|
||||
if (config.length > 0) {
|
||||
const userConfig = config[0];
|
||||
console.log(` User ID: ${userConfig.userId}`);
|
||||
console.log(` GitHub Username: ${userConfig.githubConfig?.username || "Not set"}`);
|
||||
console.log(` Gitea URL: ${userConfig.giteaConfig?.url || "Not set"}`);
|
||||
console.log(` Gitea Username: ${userConfig.giteaConfig?.username || "Not set"}`);
|
||||
console.log(` Preserve Org Structure: ${userConfig.githubConfig?.preserveOrgStructure || false}`);
|
||||
console.log(` Mirror Issues: ${userConfig.githubConfig?.mirrorIssues || false}`);
|
||||
}
|
||||
|
||||
// Check for any active jobs
|
||||
console.log("\n🔄 Active Jobs:");
|
||||
console.log("-".repeat(15));
|
||||
|
||||
const activeJobs = await db
|
||||
.select()
|
||||
.from(mirrorJobs)
|
||||
.where(
|
||||
and(
|
||||
eq(mirrorJobs.repositoryId, repo.id),
|
||||
eq(mirrorJobs.inProgress, true)
|
||||
)
|
||||
);
|
||||
|
||||
if (activeJobs.length === 0) {
|
||||
console.log(" No active jobs found");
|
||||
} else {
|
||||
activeJobs.forEach((job, index) => {
|
||||
console.log(` ${index + 1}. Job ID: ${job.id}`);
|
||||
console.log(` Type: ${job.jobType || "mirror"}`);
|
||||
console.log(` Batch ID: ${job.batchId || "None"}`);
|
||||
console.log(` Started: ${job.startedAt ? new Date(job.startedAt).toISOString() : "Unknown"}`);
|
||||
console.log(` Last Checkpoint: ${job.lastCheckpoint ? new Date(job.lastCheckpoint).toISOString() : "None"}`);
|
||||
console.log(` Progress: ${job.completedItems || 0}/${job.totalItems || 0}`);
|
||||
console.log("");
|
||||
});
|
||||
}
|
||||
|
||||
// Check if repository exists in Gitea
|
||||
if (config.length > 0) {
|
||||
const userConfig = config[0];
|
||||
console.log("\n🔗 Gitea Repository Check:");
|
||||
console.log("-".repeat(25));
|
||||
|
||||
try {
|
||||
const giteaUrl = userConfig.giteaConfig?.url;
|
||||
const giteaToken = userConfig.giteaConfig?.token;
|
||||
const giteaUsername = userConfig.giteaConfig?.username;
|
||||
|
||||
if (giteaUrl && giteaToken && giteaUsername) {
|
||||
const checkUrl = `${giteaUrl}/api/v1/repos/${giteaUsername}/${repo.name}`;
|
||||
console.log(` Checking: ${checkUrl}`);
|
||||
|
||||
const response = await fetch(checkUrl, {
|
||||
headers: {
|
||||
Authorization: `token ${giteaToken}`,
|
||||
},
|
||||
});
|
||||
|
||||
console.log(` Response Status: ${response.status} ${response.statusText}`);
|
||||
|
||||
if (response.ok) {
|
||||
const repoData = await response.json();
|
||||
console.log(` ✅ Repository exists in Gitea`);
|
||||
console.log(` Name: ${repoData.name}`);
|
||||
console.log(` Full Name: ${repoData.full_name}`);
|
||||
console.log(` Private: ${repoData.private}`);
|
||||
console.log(` Mirror: ${repoData.mirror}`);
|
||||
console.log(` Clone URL: ${repoData.clone_url}`);
|
||||
console.log(` Created: ${new Date(repoData.created_at).toISOString()}`);
|
||||
console.log(` Updated: ${new Date(repoData.updated_at).toISOString()}`);
|
||||
if (repoData.mirror_updated) {
|
||||
console.log(` Mirror Updated: ${new Date(repoData.mirror_updated).toISOString()}`);
|
||||
}
|
||||
} else {
|
||||
console.log(` ❌ Repository not found in Gitea`);
|
||||
const errorText = await response.text();
|
||||
console.log(` Error: ${errorText}`);
|
||||
}
|
||||
} else {
|
||||
console.log(" ⚠️ Missing Gitea configuration");
|
||||
}
|
||||
} catch (error) {
|
||||
console.log(` ❌ Error checking Gitea: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error("❌ Error investigating repository:", error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the investigation
|
||||
investigateRepository().then(() => {
|
||||
console.log("Investigation complete.");
|
||||
process.exit(0);
|
||||
}).catch((error) => {
|
||||
console.error("Fatal error:", error);
|
||||
process.exit(1);
|
||||
});
|
||||
277
scripts/repair-mirrored-repos.ts
Normal file
@@ -0,0 +1,277 @@
|
||||
#!/usr/bin/env bun
|
||||
|
||||
/**
|
||||
* Script to repair repositories that exist in Gitea but have incorrect status in the database
|
||||
* This fixes the issue where repositories show as "imported" but are actually mirrored in Gitea
|
||||
*
|
||||
* Usage: bun scripts/repair-mirrored-repos.ts [--dry-run] [--repo-name=<name>]
|
||||
*/
|
||||
|
||||
import { db, repositories, configs } from "@/lib/db";
|
||||
import { eq, and, or } from "drizzle-orm";
|
||||
import { createMirrorJob } from "@/lib/helpers";
|
||||
import { repoStatusEnum } from "@/types/Repository";
|
||||
|
||||
const isDryRun = process.argv.includes("--dry-run");
|
||||
const specificRepo = process.argv.find(arg => arg.startsWith("--repo-name="))?.split("=")[1];
|
||||
const isStartupMode = process.argv.includes("--startup");
|
||||
|
||||
async function checkRepoInGitea(config: any, owner: string, repoName: string): Promise<boolean> {
|
||||
try {
|
||||
if (!config.giteaConfig?.url || !config.giteaConfig?.token) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const response = await fetch(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${owner}/${repoName}`,
|
||||
{
|
||||
headers: {
|
||||
Authorization: `token ${config.giteaConfig.token}`,
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
return response.ok;
|
||||
} catch (error) {
|
||||
console.error(`Error checking repo ${owner}/${repoName} in Gitea:`, error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function getRepoDetailsFromGitea(config: any, owner: string, repoName: string): Promise<any> {
|
||||
try {
|
||||
if (!config.giteaConfig?.url || !config.giteaConfig?.token) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const response = await fetch(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${owner}/${repoName}`,
|
||||
{
|
||||
headers: {
|
||||
Authorization: `token ${config.giteaConfig.token}`,
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
if (response.ok) {
|
||||
return await response.json();
|
||||
}
|
||||
return null;
|
||||
} catch (error) {
|
||||
console.error(`Error getting repo details for ${owner}/${repoName}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function repairMirroredRepositories() {
|
||||
if (!isStartupMode) {
|
||||
console.log("🔧 Repairing mirrored repositories database status");
|
||||
console.log("=" .repeat(60));
|
||||
|
||||
if (isDryRun) {
|
||||
console.log("🔍 DRY RUN MODE - No changes will be made");
|
||||
console.log("");
|
||||
}
|
||||
|
||||
if (specificRepo) {
|
||||
console.log(`🎯 Targeting specific repository: ${specificRepo}`);
|
||||
console.log("");
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
// Find repositories that might need repair
|
||||
let query = db
|
||||
.select()
|
||||
.from(repositories)
|
||||
.where(
|
||||
or(
|
||||
eq(repositories.status, "imported"),
|
||||
eq(repositories.status, "failed")
|
||||
)
|
||||
);
|
||||
|
||||
if (specificRepo) {
|
||||
query = query.where(eq(repositories.name, specificRepo));
|
||||
}
|
||||
|
||||
const repos = await query;
|
||||
|
||||
if (repos.length === 0) {
|
||||
if (!isStartupMode) {
|
||||
console.log("✅ No repositories found that need repair");
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (!isStartupMode) {
|
||||
console.log(`📋 Found ${repos.length} repositories to check:`);
|
||||
console.log("");
|
||||
}
|
||||
|
||||
let repairedCount = 0;
|
||||
let skippedCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (const repo of repos) {
|
||||
if (!isStartupMode) {
|
||||
console.log(`🔍 Checking repository: ${repo.name}`);
|
||||
console.log(` Current status: ${repo.status}`);
|
||||
console.log(` Mirrored location: ${repo.mirroredLocation || "Not set"}`);
|
||||
}
|
||||
|
||||
try {
|
||||
// Get user configuration
|
||||
const config = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(eq(configs.id, repo.configId))
|
||||
.limit(1);
|
||||
|
||||
if (config.length === 0) {
|
||||
if (!isStartupMode) {
|
||||
console.log(` ❌ No configuration found for repository`);
|
||||
}
|
||||
errorCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
const userConfig = config[0];
|
||||
const giteaUsername = userConfig.giteaConfig?.username;
|
||||
|
||||
if (!giteaUsername) {
|
||||
if (!isStartupMode) {
|
||||
console.log(` ❌ No Gitea username in configuration`);
|
||||
}
|
||||
errorCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check if repository exists in Gitea (try both user and organization)
|
||||
let existsInGitea = false;
|
||||
let actualOwner = giteaUsername;
|
||||
let giteaRepoDetails = null;
|
||||
|
||||
// First check user location
|
||||
existsInGitea = await checkRepoInGitea(userConfig, giteaUsername, repo.name);
|
||||
if (existsInGitea) {
|
||||
giteaRepoDetails = await getRepoDetailsFromGitea(userConfig, giteaUsername, repo.name);
|
||||
}
|
||||
|
||||
// If not found in user location and repo has organization, check organization
|
||||
if (!existsInGitea && repo.organization) {
|
||||
existsInGitea = await checkRepoInGitea(userConfig, repo.organization, repo.name);
|
||||
if (existsInGitea) {
|
||||
actualOwner = repo.organization;
|
||||
giteaRepoDetails = await getRepoDetailsFromGitea(userConfig, repo.organization, repo.name);
|
||||
}
|
||||
}
|
||||
|
||||
if (!existsInGitea) {
|
||||
if (!isStartupMode) {
|
||||
console.log(` ⏭️ Repository not found in Gitea - skipping`);
|
||||
}
|
||||
skippedCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!isStartupMode) {
|
||||
console.log(` ✅ Repository found in Gitea at: ${actualOwner}/${repo.name}`);
|
||||
|
||||
if (giteaRepoDetails) {
|
||||
console.log(` 📊 Gitea details:`);
|
||||
console.log(` Mirror: ${giteaRepoDetails.mirror}`);
|
||||
console.log(` Created: ${new Date(giteaRepoDetails.created_at).toISOString()}`);
|
||||
console.log(` Updated: ${new Date(giteaRepoDetails.updated_at).toISOString()}`);
|
||||
if (giteaRepoDetails.mirror_updated) {
|
||||
console.log(` Mirror Updated: ${new Date(giteaRepoDetails.mirror_updated).toISOString()}`);
|
||||
}
|
||||
}
|
||||
} else if (repairedCount === 0) {
|
||||
// In startup mode, only log the first repair to indicate activity
|
||||
console.log(`Repairing repository status inconsistencies...`);
|
||||
}
|
||||
|
||||
if (!isDryRun) {
|
||||
// Update repository status in database
|
||||
const mirrorUpdated = giteaRepoDetails?.mirror_updated
|
||||
? new Date(giteaRepoDetails.mirror_updated)
|
||||
: new Date();
|
||||
|
||||
await db
|
||||
.update(repositories)
|
||||
.set({
|
||||
status: repoStatusEnum.parse("mirrored"),
|
||||
updatedAt: new Date(),
|
||||
lastMirrored: mirrorUpdated,
|
||||
errorMessage: null,
|
||||
mirroredLocation: `${actualOwner}/${repo.name}`,
|
||||
})
|
||||
.where(eq(repositories.id, repo.id!));
|
||||
|
||||
// Create a mirror job log entry
|
||||
await createMirrorJob({
|
||||
userId: userConfig.userId || "",
|
||||
repositoryId: repo.id,
|
||||
repositoryName: repo.name,
|
||||
message: `Repository status repaired - found existing mirror in Gitea`,
|
||||
details: `Repository ${repo.name} was found to already exist in Gitea at ${actualOwner}/${repo.name} and database status was updated from ${repo.status} to mirrored.`,
|
||||
status: "mirrored",
|
||||
});
|
||||
|
||||
if (!isStartupMode) {
|
||||
console.log(` 🔧 Repaired: Updated status to 'mirrored'`);
|
||||
}
|
||||
} else {
|
||||
if (!isStartupMode) {
|
||||
console.log(` 🔧 Would repair: Update status from '${repo.status}' to 'mirrored'`);
|
||||
}
|
||||
}
|
||||
|
||||
repairedCount++;
|
||||
|
||||
} catch (error) {
|
||||
if (!isStartupMode) {
|
||||
console.log(` ❌ Error processing repository: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
errorCount++;
|
||||
}
|
||||
|
||||
if (!isStartupMode) {
|
||||
console.log("");
|
||||
}
|
||||
}
|
||||
|
||||
if (isStartupMode) {
|
||||
// In startup mode, only log if there were repairs or errors
|
||||
if (repairedCount > 0) {
|
||||
console.log(`Repaired ${repairedCount} repository status inconsistencies`);
|
||||
}
|
||||
if (errorCount > 0) {
|
||||
console.log(`Warning: ${errorCount} repositories had errors during repair`);
|
||||
}
|
||||
} else {
|
||||
console.log("📊 Repair Summary:");
|
||||
console.log(` Repaired: ${repairedCount}`);
|
||||
console.log(` Skipped: ${skippedCount}`);
|
||||
console.log(` Errors: ${errorCount}`);
|
||||
|
||||
if (isDryRun && repairedCount > 0) {
|
||||
console.log("");
|
||||
console.log("💡 To apply these changes, run the script without --dry-run");
|
||||
}
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error("❌ Error during repair process:", error);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the repair
|
||||
repairMirroredRepositories().then(() => {
|
||||
console.log("Repair process complete.");
|
||||
process.exit(0);
|
||||
}).catch((error) => {
|
||||
console.error("Fatal error:", error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -16,7 +16,7 @@ import {
|
||||
DialogTitle,
|
||||
DialogTrigger,
|
||||
} from '../ui/dialog';
|
||||
import { apiRequest, formatDate } from '@/lib/utils';
|
||||
import { apiRequest, formatDate, showErrorToast } from '@/lib/utils';
|
||||
import { useAuth } from '@/hooks/useAuth';
|
||||
import type { MirrorJob } from '@/lib/db/schema';
|
||||
import type { ActivityApiResponse } from '@/types/activities';
|
||||
@@ -155,7 +155,7 @@ export function ActivityLog() {
|
||||
if (!res.success) {
|
||||
// Only show error toast for manual refreshes to avoid spam during live updates
|
||||
if (!isLiveRefresh) {
|
||||
toast.error(res.message ?? 'Failed to fetch activities.');
|
||||
showErrorToast(res.message ?? 'Failed to fetch activities.', toast);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
@@ -184,9 +184,7 @@ export function ActivityLog() {
|
||||
if (isMountedRef.current) {
|
||||
// Only show error toast for manual refreshes to avoid spam during live updates
|
||||
if (!isLiveRefresh) {
|
||||
toast.error(
|
||||
err instanceof Error ? err.message : 'Failed to fetch activities.',
|
||||
);
|
||||
showErrorToast(err, toast);
|
||||
}
|
||||
}
|
||||
return false;
|
||||
@@ -313,7 +311,6 @@ export function ActivityLog() {
|
||||
setIsInitialLoading(true);
|
||||
setShowCleanupDialog(false);
|
||||
|
||||
// Use fetch directly to avoid potential axios issues
|
||||
const response = await fetch('/api/activities/cleanup', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
@@ -332,11 +329,11 @@ export function ActivityLog() {
|
||||
setActivities([]);
|
||||
toast.success(`All activities cleaned up successfully. Deleted ${res.result.mirrorJobsDeleted} mirror jobs and ${res.result.eventsDeleted} events.`);
|
||||
} else {
|
||||
toast.error(res.error || 'Failed to cleanup activities.');
|
||||
showErrorToast(res.error || 'Failed to cleanup activities.', toast);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error cleaning up activities:', error);
|
||||
toast.error(error instanceof Error ? error.message : 'Failed to cleanup activities.');
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsInitialLoading(false);
|
||||
}
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { useState, useEffect } from 'react';
|
||||
import { useState } from 'react';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { Card, CardContent, CardDescription, CardFooter, CardHeader, CardTitle } from '@/components/ui/card';
|
||||
import { SiGitea } from 'react-icons/si';
|
||||
|
||||
import { toast, Toaster } from 'sonner';
|
||||
import { FlipHorizontal } from 'lucide-react';
|
||||
import { showErrorToast } from '@/lib/utils';
|
||||
|
||||
|
||||
export function LoginForm() {
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
@@ -45,10 +46,10 @@ export function LoginForm() {
|
||||
window.location.href = '/';
|
||||
}, 1000);
|
||||
} else {
|
||||
toast.error(data.error || 'Login failed. Please try again.');
|
||||
showErrorToast(data.error || 'Login failed. Please try again.', toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error('An error occurred while logging in. Please try again.');
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
@@ -59,7 +60,7 @@ export function LoginForm() {
|
||||
<Card className="w-full max-w-md">
|
||||
<CardHeader className="text-center">
|
||||
<div className="flex justify-center mb-4">
|
||||
<SiGitea className="h-10 w-10" />
|
||||
<img src="/logo.svg" alt="Gitea Mirror" className="h-10 w-10" />
|
||||
</div>
|
||||
<CardTitle className="text-2xl">Gitea Mirror</CardTitle>
|
||||
<CardDescription>
|
||||
|
||||
@@ -5,6 +5,7 @@ import { Button } from '@/components/ui/button';
|
||||
import { Card, CardContent, CardDescription, CardFooter, CardHeader, CardTitle } from '@/components/ui/card';
|
||||
import { GitMerge } from 'lucide-react';
|
||||
import { toast, Toaster } from 'sonner';
|
||||
import { showErrorToast } from '@/lib/utils';
|
||||
|
||||
export function SignupForm() {
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
@@ -51,10 +52,10 @@ export function SignupForm() {
|
||||
window.location.href = '/';
|
||||
}, 1500);
|
||||
} else {
|
||||
toast.error(data.error || 'Failed to create account. Please try again.');
|
||||
showErrorToast(data.error || 'Failed to create account. Please try again.', toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error('An error occurred while creating your account. Please try again.');
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ import type {
|
||||
} from '@/types/config';
|
||||
import { Button } from '../ui/button';
|
||||
import { useAuth } from '@/hooks/useAuth';
|
||||
import { apiRequest } from '@/lib/utils';
|
||||
import { apiRequest, showErrorToast } from '@/lib/utils';
|
||||
import { RefreshCw } from 'lucide-react';
|
||||
import { toast } from 'sonner';
|
||||
import { Skeleton } from '@/components/ui/skeleton';
|
||||
@@ -56,14 +56,18 @@ export function ConfigTabs() {
|
||||
retentionDays: 604800, // 7 days in seconds
|
||||
},
|
||||
});
|
||||
const { user, refreshUser } = useAuth();
|
||||
const { user } = useAuth();
|
||||
const [isLoading, setIsLoading] = useState(true);
|
||||
const [isSyncing, setIsSyncing] = useState<boolean>(false);
|
||||
const [isConfigSaved, setIsConfigSaved] = useState<boolean>(false);
|
||||
|
||||
const [isAutoSavingSchedule, setIsAutoSavingSchedule] = useState<boolean>(false);
|
||||
const [isAutoSavingCleanup, setIsAutoSavingCleanup] = useState<boolean>(false);
|
||||
const [isAutoSavingGitHub, setIsAutoSavingGitHub] = useState<boolean>(false);
|
||||
const [isAutoSavingGitea, setIsAutoSavingGitea] = useState<boolean>(false);
|
||||
const autoSaveScheduleTimeoutRef = useRef<NodeJS.Timeout | null>(null);
|
||||
const autoSaveCleanupTimeoutRef = useRef<NodeJS.Timeout | null>(null);
|
||||
const autoSaveGitHubTimeoutRef = useRef<NodeJS.Timeout | null>(null);
|
||||
const autoSaveGiteaTimeoutRef = useRef<NodeJS.Timeout | null>(null);
|
||||
|
||||
const isConfigFormValid = (): boolean => {
|
||||
const { githubConfig, giteaConfig } = config;
|
||||
@@ -109,47 +113,9 @@ export function ConfigTabs() {
|
||||
}
|
||||
};
|
||||
|
||||
const handleSaveConfig = async () => {
|
||||
if (!user?.id) return;
|
||||
const reqPayload: SaveConfigApiRequest = {
|
||||
userId: user.id,
|
||||
githubConfig: config.githubConfig,
|
||||
giteaConfig: config.giteaConfig,
|
||||
scheduleConfig: config.scheduleConfig,
|
||||
cleanupConfig: config.cleanupConfig,
|
||||
};
|
||||
try {
|
||||
const response = await fetch('/api/config', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(reqPayload),
|
||||
});
|
||||
const result: SaveConfigApiResponse = await response.json();
|
||||
if (result.success) {
|
||||
await refreshUser();
|
||||
setIsConfigSaved(true);
|
||||
// Invalidate config cache so other components get fresh data
|
||||
invalidateConfigCache();
|
||||
toast.success(
|
||||
'Configuration saved successfully! Now import your GitHub data to begin.',
|
||||
);
|
||||
} else {
|
||||
toast.error(
|
||||
`Failed to save configuration: ${result.message || 'Unknown error'}`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
`An error occurred while saving the configuration: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
// Auto-save function specifically for schedule config changes
|
||||
const autoSaveScheduleConfig = useCallback(async (scheduleConfig: ScheduleConfig) => {
|
||||
if (!user?.id || !isConfigSaved) return; // Only auto-save if config was previously saved
|
||||
if (!user?.id) return;
|
||||
|
||||
// Clear any existing timeout
|
||||
if (autoSaveScheduleTimeoutRef.current) {
|
||||
@@ -198,27 +164,22 @@ export function ConfigTabs() {
|
||||
console.warn('Failed to fetch updated config after auto-save:', fetchError);
|
||||
}
|
||||
} else {
|
||||
toast.error(
|
||||
showErrorToast(
|
||||
`Auto-save failed: ${result.message || 'Unknown error'}`,
|
||||
{ duration: 3000 }
|
||||
toast
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
`Auto-save error: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
{ duration: 3000 }
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsAutoSavingSchedule(false);
|
||||
}
|
||||
}, 500); // 500ms debounce
|
||||
}, [user?.id, isConfigSaved, config.githubConfig, config.giteaConfig, config.cleanupConfig]);
|
||||
}, [user?.id, config.githubConfig, config.giteaConfig, config.cleanupConfig]);
|
||||
|
||||
// Auto-save function specifically for cleanup config changes
|
||||
const autoSaveCleanupConfig = useCallback(async (cleanupConfig: DatabaseCleanupConfig) => {
|
||||
if (!user?.id || !isConfigSaved) return; // Only auto-save if config was previously saved
|
||||
if (!user?.id) return;
|
||||
|
||||
// Clear any existing timeout
|
||||
if (autoSaveCleanupTimeoutRef.current) {
|
||||
@@ -266,23 +227,112 @@ export function ConfigTabs() {
|
||||
console.warn('Failed to fetch updated config after auto-save:', fetchError);
|
||||
}
|
||||
} else {
|
||||
toast.error(
|
||||
showErrorToast(
|
||||
`Auto-save failed: ${result.message || 'Unknown error'}`,
|
||||
{ duration: 3000 }
|
||||
toast
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
`Auto-save error: ${
|
||||
error instanceof Error ? error.message : String(error)
|
||||
}`,
|
||||
{ duration: 3000 }
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsAutoSavingCleanup(false);
|
||||
}
|
||||
}, 500); // 500ms debounce
|
||||
}, [user?.id, isConfigSaved, config.githubConfig, config.giteaConfig, config.scheduleConfig]);
|
||||
}, [user?.id, config.githubConfig, config.giteaConfig, config.scheduleConfig]);
|
||||
|
||||
// Auto-save function specifically for GitHub config changes
|
||||
const autoSaveGitHubConfig = useCallback(async (githubConfig: GitHubConfig) => {
|
||||
if (!user?.id) return;
|
||||
|
||||
// Clear any existing timeout
|
||||
if (autoSaveGitHubTimeoutRef.current) {
|
||||
clearTimeout(autoSaveGitHubTimeoutRef.current);
|
||||
}
|
||||
|
||||
// Debounce the auto-save to prevent excessive API calls
|
||||
autoSaveGitHubTimeoutRef.current = setTimeout(async () => {
|
||||
setIsAutoSavingGitHub(true);
|
||||
|
||||
const reqPayload: SaveConfigApiRequest = {
|
||||
userId: user.id!,
|
||||
githubConfig: githubConfig,
|
||||
giteaConfig: config.giteaConfig,
|
||||
scheduleConfig: config.scheduleConfig,
|
||||
cleanupConfig: config.cleanupConfig,
|
||||
};
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/config', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(reqPayload),
|
||||
});
|
||||
const result: SaveConfigApiResponse = await response.json();
|
||||
|
||||
if (result.success) {
|
||||
// Silent success - no toast for auto-save
|
||||
// Invalidate config cache so other components get fresh data
|
||||
invalidateConfigCache();
|
||||
} else {
|
||||
showErrorToast(
|
||||
`Auto-save failed: ${result.message || 'Unknown error'}`,
|
||||
toast
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsAutoSavingGitHub(false);
|
||||
}
|
||||
}, 500); // 500ms debounce
|
||||
}, [user?.id, config.giteaConfig, config.scheduleConfig, config.cleanupConfig]);
|
||||
|
||||
// Auto-save function specifically for Gitea config changes
|
||||
const autoSaveGiteaConfig = useCallback(async (giteaConfig: GiteaConfig) => {
|
||||
if (!user?.id) return;
|
||||
|
||||
// Clear any existing timeout
|
||||
if (autoSaveGiteaTimeoutRef.current) {
|
||||
clearTimeout(autoSaveGiteaTimeoutRef.current);
|
||||
}
|
||||
|
||||
// Debounce the auto-save to prevent excessive API calls
|
||||
autoSaveGiteaTimeoutRef.current = setTimeout(async () => {
|
||||
setIsAutoSavingGitea(true);
|
||||
|
||||
const reqPayload: SaveConfigApiRequest = {
|
||||
userId: user.id!,
|
||||
githubConfig: config.githubConfig,
|
||||
giteaConfig: giteaConfig,
|
||||
scheduleConfig: config.scheduleConfig,
|
||||
cleanupConfig: config.cleanupConfig,
|
||||
};
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/config', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(reqPayload),
|
||||
});
|
||||
const result: SaveConfigApiResponse = await response.json();
|
||||
|
||||
if (result.success) {
|
||||
// Silent success - no toast for auto-save
|
||||
// Invalidate config cache so other components get fresh data
|
||||
invalidateConfigCache();
|
||||
} else {
|
||||
showErrorToast(
|
||||
`Auto-save failed: ${result.message || 'Unknown error'}`,
|
||||
toast
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsAutoSavingGitea(false);
|
||||
}
|
||||
}, 500); // 500ms debounce
|
||||
}, [user?.id, config.githubConfig, config.scheduleConfig, config.cleanupConfig]);
|
||||
|
||||
// Cleanup timeouts on unmount
|
||||
useEffect(() => {
|
||||
@@ -293,6 +343,12 @@ export function ConfigTabs() {
|
||||
if (autoSaveCleanupTimeoutRef.current) {
|
||||
clearTimeout(autoSaveCleanupTimeoutRef.current);
|
||||
}
|
||||
if (autoSaveGitHubTimeoutRef.current) {
|
||||
clearTimeout(autoSaveGitHubTimeoutRef.current);
|
||||
}
|
||||
if (autoSaveGiteaTimeoutRef.current) {
|
||||
clearTimeout(autoSaveGiteaTimeoutRef.current);
|
||||
}
|
||||
};
|
||||
}, []);
|
||||
|
||||
@@ -317,7 +373,7 @@ export function ConfigTabs() {
|
||||
cleanupConfig:
|
||||
response.cleanupConfig || config.cleanupConfig,
|
||||
});
|
||||
if (response.id) setIsConfigSaved(true);
|
||||
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
@@ -414,10 +470,10 @@ export function ConfigTabs() {
|
||||
<div className="flex gap-x-4">
|
||||
<Button
|
||||
onClick={handleImportGitHubData}
|
||||
disabled={isSyncing || !isConfigSaved}
|
||||
disabled={isSyncing || !isConfigFormValid()}
|
||||
title={
|
||||
!isConfigSaved
|
||||
? 'Save configuration first'
|
||||
!isConfigFormValid()
|
||||
? 'Please fill all required GitHub and Gitea fields'
|
||||
: isSyncing
|
||||
? 'Import in progress'
|
||||
: 'Import GitHub Data'
|
||||
@@ -435,17 +491,6 @@ export function ConfigTabs() {
|
||||
</>
|
||||
)}
|
||||
</Button>
|
||||
<Button
|
||||
onClick={handleSaveConfig}
|
||||
disabled={!isConfigFormValid()}
|
||||
title={
|
||||
!isConfigFormValid()
|
||||
? 'Please fill all required fields'
|
||||
: 'Save Configuration'
|
||||
}
|
||||
>
|
||||
Save Configuration
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -463,6 +508,8 @@ export function ConfigTabs() {
|
||||
: update,
|
||||
}))
|
||||
}
|
||||
onAutoSave={autoSaveGitHubConfig}
|
||||
isAutoSaving={isAutoSavingGitHub}
|
||||
/>
|
||||
<GiteaConfigForm
|
||||
config={config.giteaConfig}
|
||||
@@ -475,6 +522,8 @@ export function ConfigTabs() {
|
||||
: update,
|
||||
}))
|
||||
}
|
||||
onAutoSave={autoSaveGiteaConfig}
|
||||
isAutoSaving={isAutoSavingGitea}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex gap-x-4">
|
||||
|
||||
@@ -20,9 +20,11 @@ import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip";
|
||||
interface GitHubConfigFormProps {
|
||||
config: GitHubConfig;
|
||||
setConfig: React.Dispatch<React.SetStateAction<GitHubConfig>>;
|
||||
onAutoSave?: (githubConfig: GitHubConfig) => Promise<void>;
|
||||
isAutoSaving?: boolean;
|
||||
}
|
||||
|
||||
export function GitHubConfigForm({ config, setConfig }: GitHubConfigFormProps) {
|
||||
export function GitHubConfigForm({ config, setConfig, onAutoSave, isAutoSaving }: GitHubConfigFormProps) {
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
|
||||
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
|
||||
@@ -43,10 +45,17 @@ export function GitHubConfigForm({ config, setConfig }: GitHubConfigFormProps) {
|
||||
);
|
||||
}
|
||||
|
||||
setConfig({
|
||||
const newConfig = {
|
||||
...config,
|
||||
[name]: type === "checkbox" ? checked : value,
|
||||
});
|
||||
};
|
||||
|
||||
setConfig(newConfig);
|
||||
|
||||
// Auto-save for all field changes
|
||||
if (onAutoSave) {
|
||||
onAutoSave(newConfig);
|
||||
}
|
||||
};
|
||||
|
||||
const testConnection = async () => {
|
||||
|
||||
@@ -21,19 +21,27 @@ import { toast } from "sonner";
|
||||
interface GiteaConfigFormProps {
|
||||
config: GiteaConfig;
|
||||
setConfig: React.Dispatch<React.SetStateAction<GiteaConfig>>;
|
||||
onAutoSave?: (giteaConfig: GiteaConfig) => Promise<void>;
|
||||
isAutoSaving?: boolean;
|
||||
}
|
||||
|
||||
export function GiteaConfigForm({ config, setConfig }: GiteaConfigFormProps) {
|
||||
export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving }: GiteaConfigFormProps) {
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
|
||||
const handleChange = (
|
||||
e: React.ChangeEvent<HTMLInputElement | HTMLSelectElement>
|
||||
) => {
|
||||
const { name, value } = e.target;
|
||||
setConfig({
|
||||
const newConfig = {
|
||||
...config,
|
||||
[name]: value,
|
||||
});
|
||||
};
|
||||
setConfig(newConfig);
|
||||
|
||||
// Auto-save for all field changes
|
||||
if (onAutoSave) {
|
||||
onAutoSave(newConfig);
|
||||
}
|
||||
};
|
||||
|
||||
const testConnection = async () => {
|
||||
|
||||
@@ -5,7 +5,7 @@ import { GitFork, Clock, FlipHorizontal, Building2 } from "lucide-react";
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import type { MirrorJob, Organization, Repository } from "@/lib/db/schema";
|
||||
import { useAuth } from "@/hooks/useAuth";
|
||||
import { apiRequest } from "@/lib/utils";
|
||||
import { apiRequest, showErrorToast } from "@/lib/utils";
|
||||
import type { DashboardApiResponse } from "@/types/dashboard";
|
||||
import { useSSE } from "@/hooks/useSEE";
|
||||
import { toast } from "sonner";
|
||||
@@ -103,15 +103,11 @@ export function Dashboard() {
|
||||
}
|
||||
return true;
|
||||
} else {
|
||||
toast.error(response.error || "Error fetching dashboard data");
|
||||
showErrorToast(response.error || "Error fetching dashboard data", toast);
|
||||
return false;
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Error fetching dashboard data"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
return false;
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
|
||||
@@ -81,6 +81,11 @@ export function RepositoryList({ repositories }: RepositoryListProps) {
|
||||
Private
|
||||
</span>
|
||||
)}
|
||||
{repo.isForked && (
|
||||
<span className="rounded-full bg-muted px-2 py-0.5 text-xs">
|
||||
Fork
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex items-center gap-2 mt-1">
|
||||
<span className="text-xs text-muted-foreground">
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { useAuth } from "@/hooks/useAuth";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { SiGitea } from "react-icons/si";
|
||||
|
||||
import { ModeToggle } from "@/components/theme/ModeToggle";
|
||||
import { Avatar, AvatarFallback, AvatarImage } from "../ui/avatar";
|
||||
import { toast } from "sonner";
|
||||
@@ -64,7 +64,7 @@ export function Header({ currentPage, onNavigate }: HeaderProps) {
|
||||
}}
|
||||
className="flex items-center gap-2 py-1 hover:opacity-80 transition-opacity"
|
||||
>
|
||||
<SiGitea className="h-6 w-6" />
|
||||
<img src="/logo.svg" alt="Gitea Mirror" className="h-6 w-6" />
|
||||
<span className="text-xl font-bold">Gitea Mirror</span>
|
||||
</button>
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ export function VersionInfo() {
|
||||
return (
|
||||
<div className="text-xs text-muted-foreground text-center pt-2 pb-3 border-t border-border mt-2">
|
||||
{versionInfo.updateAvailable ? (
|
||||
<div className="flex flex-col">
|
||||
<div className="flex flex-col gap-1">
|
||||
<span>v{versionInfo.current}</span>
|
||||
<span className="text-primary">v{versionInfo.latest} available</span>
|
||||
</div>
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
import { useCallback, useEffect, useState } from "react";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Search, RefreshCw, FlipHorizontal, Plus } from "lucide-react";
|
||||
import { Search, RefreshCw, FlipHorizontal } from "lucide-react";
|
||||
import type { MirrorJob, Organization } from "@/lib/db/schema";
|
||||
import { OrganizationList } from "./OrganizationsList";
|
||||
import AddOrganizationDialog from "./AddOrganizationDialog";
|
||||
import { useAuth } from "@/hooks/useAuth";
|
||||
import { apiRequest } from "@/lib/utils";
|
||||
import { apiRequest, showErrorToast } from "@/lib/utils";
|
||||
import {
|
||||
membershipRoleEnum,
|
||||
type AddOrganizationApiRequest,
|
||||
@@ -26,6 +26,7 @@ import { useFilterParams } from "@/hooks/useFilterParams";
|
||||
import { toast } from "sonner";
|
||||
import { useConfigStatus } from "@/hooks/useConfigStatus";
|
||||
import { useNavigation } from "@/components/layout/MainLayout";
|
||||
import { useLiveRefresh } from "@/hooks/useLiveRefresh";
|
||||
|
||||
export function Organization() {
|
||||
const [organizations, setOrganizations] = useState<Organization[]>([]);
|
||||
@@ -34,6 +35,7 @@ export function Organization() {
|
||||
const { user } = useAuth();
|
||||
const { isGitHubConfigured } = useConfigStatus();
|
||||
const { navigationKey } = useNavigation();
|
||||
const { registerRefreshCallback } = useLiveRefresh();
|
||||
const { filter, setFilter } = useFilterParams({
|
||||
searchTerm: "",
|
||||
membershipRole: "",
|
||||
@@ -62,19 +64,23 @@ export function Organization() {
|
||||
onMessage: handleNewMessage,
|
||||
});
|
||||
|
||||
const fetchOrganizations = useCallback(async () => {
|
||||
const fetchOrganizations = useCallback(async (isLiveRefresh = false) => {
|
||||
if (!user?.id) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Don't fetch organizations if GitHub is not configured
|
||||
if (!isGitHubConfigured) {
|
||||
setIsLoading(false);
|
||||
if (!isLiveRefresh) {
|
||||
setIsLoading(false);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
setIsLoading(true);
|
||||
if (!isLiveRefresh) {
|
||||
setIsLoading(true);
|
||||
}
|
||||
|
||||
const response = await apiRequest<OrganizationsApiResponse>(
|
||||
`/github/organizations?userId=${user.id}`,
|
||||
@@ -87,27 +93,47 @@ export function Organization() {
|
||||
setOrganizations(response.organizations);
|
||||
return true;
|
||||
} else {
|
||||
toast.error(response.error || "Error fetching organizations");
|
||||
if (!isLiveRefresh) {
|
||||
toast.error(response.error || "Error fetching organizations");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error fetching organizations"
|
||||
);
|
||||
if (!isLiveRefresh) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error fetching organizations"
|
||||
);
|
||||
}
|
||||
return false;
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
if (!isLiveRefresh) {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
}, [user?.id, isGitHubConfigured]); // Only depend on user.id, not entire user object
|
||||
|
||||
useEffect(() => {
|
||||
// Reset loading state when component becomes active
|
||||
setIsLoading(true);
|
||||
fetchOrganizations();
|
||||
fetchOrganizations(false); // Manual refresh, not live
|
||||
}, [fetchOrganizations, navigationKey]); // Include navigationKey to trigger on navigation
|
||||
|
||||
// Register with global live refresh system
|
||||
useEffect(() => {
|
||||
// Only register for live refresh if GitHub is configured
|
||||
if (!isGitHubConfigured) {
|
||||
return;
|
||||
}
|
||||
|
||||
const unregister = registerRefreshCallback(() => {
|
||||
fetchOrganizations(true); // Live refresh
|
||||
});
|
||||
|
||||
return unregister;
|
||||
}, [registerRefreshCallback, fetchOrganizations, isGitHubConfigured]);
|
||||
|
||||
const handleRefresh = async () => {
|
||||
const success = await fetchOrganizations();
|
||||
const success = await fetchOrganizations(false);
|
||||
if (success) {
|
||||
toast.success("Organizations refreshed successfully.");
|
||||
}
|
||||
@@ -140,6 +166,12 @@ export function Organization() {
|
||||
return updated ? updated : org;
|
||||
})
|
||||
);
|
||||
|
||||
// Refresh organization data to get updated repository breakdown
|
||||
// Use a small delay to allow the backend to process the mirroring request
|
||||
setTimeout(() => {
|
||||
fetchOrganizations(true);
|
||||
}, 1000);
|
||||
} else {
|
||||
toast.error(response.error || "Error starting mirror job");
|
||||
}
|
||||
@@ -193,12 +225,10 @@ export function Organization() {
|
||||
searchTerm: org,
|
||||
}));
|
||||
} else {
|
||||
toast.error(response.error || "Error adding organization");
|
||||
showErrorToast(response.error || "Error adding organization", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error adding organization"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
@@ -250,24 +280,17 @@ export function Organization() {
|
||||
})
|
||||
);
|
||||
} else {
|
||||
toast.error(response.error || "Error starting mirror jobs");
|
||||
showErrorToast(response.error || "Error starting mirror jobs", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error starting mirror jobs"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
// Reset loading states - we'll let the SSE updates handle status changes
|
||||
setLoadingOrgIds(new Set());
|
||||
}
|
||||
};
|
||||
|
||||
// Get unique organization names for combobox (since Organization has no owner field)
|
||||
const ownerOptions = Array.from(
|
||||
new Set(
|
||||
organizations.map((org) => org.name).filter((v): v is string => !!v)
|
||||
)
|
||||
).sort();
|
||||
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-y-8">
|
||||
|
||||
@@ -118,10 +118,38 @@ export function OrganizationList({
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<p className="text-sm text-muted-foreground mb-4">
|
||||
{org.repositoryCount}{" "}
|
||||
{org.repositoryCount === 1 ? "repository" : "repositories"}
|
||||
</p>
|
||||
<div className="text-sm text-muted-foreground mb-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="font-medium">
|
||||
{org.repositoryCount}{" "}
|
||||
{org.repositoryCount === 1 ? "repository" : "repositories"}
|
||||
</span>
|
||||
</div>
|
||||
{(org.publicRepositoryCount !== undefined ||
|
||||
org.privateRepositoryCount !== undefined ||
|
||||
org.forkRepositoryCount !== undefined) && (
|
||||
<div className="flex gap-4 mt-2 text-xs">
|
||||
{org.publicRepositoryCount !== undefined && (
|
||||
<span className="flex items-center gap-1">
|
||||
<div className="h-2 w-2 rounded-full bg-green-500" />
|
||||
{org.publicRepositoryCount} public
|
||||
</span>
|
||||
)}
|
||||
{org.privateRepositoryCount !== undefined && org.privateRepositoryCount > 0 && (
|
||||
<span className="flex items-center gap-1">
|
||||
<div className="h-2 w-2 rounded-full bg-orange-500" />
|
||||
{org.privateRepositoryCount} private
|
||||
</span>
|
||||
)}
|
||||
{org.forkRepositoryCount !== undefined && org.forkRepositoryCount > 0 && (
|
||||
<span className="flex items-center gap-1">
|
||||
<div className="h-2 w-2 rounded-full bg-blue-500" />
|
||||
{org.forkRepositoryCount} fork{org.forkRepositoryCount !== 1 ? 's' : ''}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center">
|
||||
@@ -144,7 +172,7 @@ export function OrganizationList({
|
||||
htmlFor={`include-${org.id}`}
|
||||
className="ml-2 text-sm select-none peer-disabled:cursor-not-allowed peer-disabled:opacity-70"
|
||||
>
|
||||
Include in mirroring
|
||||
Enable mirroring
|
||||
</label>
|
||||
|
||||
{isLoading && (
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
type RepositoryApiResponse,
|
||||
type RepoStatus,
|
||||
} from "@/types/Repository";
|
||||
import { apiRequest } from "@/lib/utils";
|
||||
import { apiRequest, showErrorToast } from "@/lib/utils";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
@@ -108,16 +108,14 @@ export default function Repository() {
|
||||
} else {
|
||||
// Only show error toast for manual refreshes to avoid spam during live updates
|
||||
if (!isLiveRefresh) {
|
||||
toast.error(response.error || "Error fetching repositories");
|
||||
showErrorToast(response.error || "Error fetching repositories", toast);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
} catch (error) {
|
||||
// Only show error toast for manual refreshes to avoid spam during live updates
|
||||
if (!isLiveRefresh) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error fetching repositories"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
}
|
||||
return false;
|
||||
} finally {
|
||||
@@ -184,12 +182,10 @@ export default function Repository() {
|
||||
})
|
||||
);
|
||||
} else {
|
||||
toast.error(response.error || "Error starting mirror job");
|
||||
showErrorToast(response.error || "Error starting mirror job", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error starting mirror job"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setLoadingRepoIds((prev) => {
|
||||
const newSet = new Set(prev);
|
||||
@@ -248,12 +244,10 @@ export default function Repository() {
|
||||
})
|
||||
);
|
||||
} else {
|
||||
toast.error(response.error || "Error starting mirror jobs");
|
||||
showErrorToast(response.error || "Error starting mirror jobs", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error starting mirror jobs"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
// Reset loading states - we'll let the SSE updates handle status changes
|
||||
setLoadingRepoIds(new Set());
|
||||
@@ -287,12 +281,10 @@ export default function Repository() {
|
||||
})
|
||||
);
|
||||
} else {
|
||||
toast.error(response.error || "Error starting sync job");
|
||||
showErrorToast(response.error || "Error starting sync job", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error starting sync job"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setLoadingRepoIds((prev) => {
|
||||
const newSet = new Set(prev);
|
||||
@@ -329,12 +321,10 @@ export default function Repository() {
|
||||
})
|
||||
);
|
||||
} else {
|
||||
toast.error(response.error || "Error retrying job");
|
||||
showErrorToast(response.error || "Error retrying job", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error retrying job"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
} finally {
|
||||
setLoadingRepoIds((prev) => {
|
||||
const newSet = new Set(prev);
|
||||
@@ -381,12 +371,10 @@ export default function Repository() {
|
||||
searchTerm: repo,
|
||||
}));
|
||||
} else {
|
||||
toast.error(response.error || "Error adding repository");
|
||||
showErrorToast(response.error || "Error adding repository", toast);
|
||||
}
|
||||
} catch (error) {
|
||||
toast.error(
|
||||
error instanceof Error ? error.message : "Error adding repository"
|
||||
);
|
||||
showErrorToast(error, toast);
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -249,6 +249,11 @@ export default function RepositoryTable({
|
||||
Private
|
||||
</span>
|
||||
)}
|
||||
{repo.isForked && (
|
||||
<span className="ml-2 rounded-full bg-muted px-2 py-0.5 text-xs">
|
||||
Fork
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Owner */}
|
||||
|
||||
@@ -25,11 +25,222 @@ let sqlite: Database;
|
||||
try {
|
||||
sqlite = new Database(dbPath);
|
||||
console.log("Successfully connected to SQLite database using Bun's native driver");
|
||||
|
||||
// Ensure all required tables exist
|
||||
ensureTablesExist(sqlite);
|
||||
} catch (error) {
|
||||
console.error("Error opening database:", error);
|
||||
throw error;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure all required tables exist in the database
|
||||
*/
|
||||
function ensureTablesExist(db: Database) {
|
||||
const requiredTables = [
|
||||
"users",
|
||||
"configs",
|
||||
"repositories",
|
||||
"organizations",
|
||||
"mirror_jobs",
|
||||
"events",
|
||||
];
|
||||
|
||||
for (const table of requiredTables) {
|
||||
try {
|
||||
// Check if table exists
|
||||
const result = db.query(`SELECT name FROM sqlite_master WHERE type='table' AND name='${table}'`).get();
|
||||
|
||||
if (!result) {
|
||||
console.warn(`⚠️ Table '${table}' is missing. Creating it now...`);
|
||||
createTable(db, table);
|
||||
console.log(`✅ Table '${table}' created successfully`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`❌ Error checking/creating table '${table}':`, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a specific table with its schema
|
||||
*/
|
||||
function createTable(db: Database, tableName: string) {
|
||||
switch (tableName) {
|
||||
case "users":
|
||||
db.exec(`
|
||||
CREATE TABLE users (
|
||||
id TEXT PRIMARY KEY,
|
||||
username TEXT NOT NULL,
|
||||
password TEXT NOT NULL,
|
||||
email TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
updated_at INTEGER NOT NULL
|
||||
)
|
||||
`);
|
||||
break;
|
||||
|
||||
case "configs":
|
||||
db.exec(`
|
||||
CREATE TABLE configs (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
is_active INTEGER NOT NULL DEFAULT 1,
|
||||
github_config TEXT NOT NULL,
|
||||
gitea_config TEXT NOT NULL,
|
||||
include TEXT NOT NULL DEFAULT '["*"]',
|
||||
exclude TEXT NOT NULL DEFAULT '[]',
|
||||
schedule_config TEXT NOT NULL,
|
||||
cleanup_config TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
FOREIGN KEY (user_id) REFERENCES users(id)
|
||||
)
|
||||
`);
|
||||
break;
|
||||
|
||||
case "repositories":
|
||||
db.exec(`
|
||||
CREATE TABLE repositories (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
config_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
full_name TEXT NOT NULL,
|
||||
url TEXT NOT NULL,
|
||||
clone_url TEXT NOT NULL,
|
||||
owner TEXT NOT NULL,
|
||||
organization TEXT,
|
||||
mirrored_location TEXT DEFAULT '',
|
||||
is_private INTEGER NOT NULL DEFAULT 0,
|
||||
is_fork INTEGER NOT NULL DEFAULT 0,
|
||||
forked_from TEXT,
|
||||
has_issues INTEGER NOT NULL DEFAULT 0,
|
||||
is_starred INTEGER NOT NULL DEFAULT 0,
|
||||
language TEXT,
|
||||
description TEXT,
|
||||
default_branch TEXT NOT NULL,
|
||||
visibility TEXT NOT NULL DEFAULT 'public',
|
||||
status TEXT NOT NULL DEFAULT 'imported',
|
||||
last_mirrored INTEGER,
|
||||
error_message TEXT,
|
||||
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
FOREIGN KEY (user_id) REFERENCES users(id),
|
||||
FOREIGN KEY (config_id) REFERENCES configs(id)
|
||||
)
|
||||
`);
|
||||
|
||||
// Create indexes for repositories
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_user_id ON repositories(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_config_id ON repositories(config_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_status ON repositories(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_owner ON repositories(owner);
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_organization ON repositories(organization);
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_is_fork ON repositories(is_fork);
|
||||
CREATE INDEX IF NOT EXISTS idx_repositories_is_starred ON repositories(is_starred);
|
||||
`);
|
||||
break;
|
||||
|
||||
case "organizations":
|
||||
db.exec(`
|
||||
CREATE TABLE organizations (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
config_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
avatar_url TEXT NOT NULL,
|
||||
membership_role TEXT NOT NULL DEFAULT 'member',
|
||||
is_included INTEGER NOT NULL DEFAULT 1,
|
||||
status TEXT NOT NULL DEFAULT 'imported',
|
||||
last_mirrored INTEGER,
|
||||
error_message TEXT,
|
||||
repository_count INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
FOREIGN KEY (user_id) REFERENCES users(id),
|
||||
FOREIGN KEY (config_id) REFERENCES configs(id)
|
||||
)
|
||||
`);
|
||||
|
||||
// Create indexes for organizations
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_organizations_user_id ON organizations(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_organizations_config_id ON organizations(config_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_organizations_status ON organizations(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_organizations_is_included ON organizations(is_included);
|
||||
`);
|
||||
break;
|
||||
|
||||
case "mirror_jobs":
|
||||
db.exec(`
|
||||
CREATE TABLE mirror_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
repository_id TEXT,
|
||||
repository_name TEXT,
|
||||
organization_id TEXT,
|
||||
organization_name TEXT,
|
||||
details TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'imported',
|
||||
message TEXT NOT NULL,
|
||||
timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
|
||||
-- New fields for job resilience
|
||||
job_type TEXT NOT NULL DEFAULT 'mirror',
|
||||
batch_id TEXT,
|
||||
total_items INTEGER,
|
||||
completed_items INTEGER DEFAULT 0,
|
||||
item_ids TEXT, -- JSON array as text
|
||||
completed_item_ids TEXT DEFAULT '[]', -- JSON array as text
|
||||
in_progress INTEGER NOT NULL DEFAULT 0, -- Boolean as integer
|
||||
started_at TIMESTAMP,
|
||||
completed_at TIMESTAMP,
|
||||
last_checkpoint TIMESTAMP,
|
||||
|
||||
FOREIGN KEY (user_id) REFERENCES users(id)
|
||||
)
|
||||
`);
|
||||
|
||||
// Create indexes for mirror_jobs
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_user_id ON mirror_jobs(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_batch_id ON mirror_jobs(batch_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_in_progress ON mirror_jobs(in_progress);
|
||||
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_job_type ON mirror_jobs(job_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_timestamp ON mirror_jobs(timestamp);
|
||||
`);
|
||||
break;
|
||||
|
||||
case "events":
|
||||
db.exec(`
|
||||
CREATE TABLE events (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
payload TEXT NOT NULL,
|
||||
read INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
|
||||
FOREIGN KEY (user_id) REFERENCES users(id)
|
||||
)
|
||||
`);
|
||||
|
||||
// Create indexes for events
|
||||
db.exec(`
|
||||
CREATE INDEX IF NOT EXISTS idx_events_user_channel ON events(user_id, channel);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_read ON events(read);
|
||||
`);
|
||||
break;
|
||||
|
||||
default:
|
||||
throw new Error(`Unknown table: ${tableName}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Create drizzle instance with the SQLite client
|
||||
export const db = drizzle({ client: sqlite });
|
||||
|
||||
|
||||
@@ -152,6 +152,9 @@ export const organizationSchema = z.object({
|
||||
errorMessage: z.string().optional(),
|
||||
|
||||
repositoryCount: z.number().default(0),
|
||||
publicRepositoryCount: z.number().optional(),
|
||||
privateRepositoryCount: z.number().optional(),
|
||||
forkRepositoryCount: z.number().optional(),
|
||||
|
||||
createdAt: z.date().default(() => new Date()),
|
||||
updatedAt: z.date().default(() => new Date()),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import { repoStatusEnum } from "@/types/Repository";
|
||||
import { getOrCreateGiteaOrg } from "./gitea";
|
||||
|
||||
// Mock the isRepoPresentInGitea function
|
||||
const mockIsRepoPresentInGitea = mock(() => Promise.resolve(false));
|
||||
@@ -27,23 +28,17 @@ mock.module("@/lib/helpers", () => {
|
||||
};
|
||||
});
|
||||
|
||||
// Mock superagent
|
||||
mock.module("superagent", () => {
|
||||
const mockPost = mock(() => ({
|
||||
set: () => ({
|
||||
set: () => ({
|
||||
send: () => Promise.resolve({ body: { id: 123 } })
|
||||
})
|
||||
})
|
||||
}));
|
||||
|
||||
const mockGet = mock(() => ({
|
||||
set: () => Promise.resolve({ body: [] })
|
||||
}));
|
||||
|
||||
// Mock http-client
|
||||
mock.module("@/lib/http-client", () => {
|
||||
return {
|
||||
post: mockPost,
|
||||
get: mockGet
|
||||
httpPost: mock(() => Promise.resolve({ data: { id: 123 }, status: 200, statusText: 'OK', headers: new Headers() })),
|
||||
httpGet: mock(() => Promise.resolve({ data: [], status: 200, statusText: 'OK', headers: new Headers() })),
|
||||
HttpError: class MockHttpError extends Error {
|
||||
constructor(message: string, public status: number, public statusText: string, public response?: string) {
|
||||
super(message);
|
||||
this.name = 'HttpError';
|
||||
}
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
@@ -117,4 +112,182 @@ describe("Gitea Repository Mirroring", () => {
|
||||
// Check that the function was called
|
||||
expect(mirrorGithubRepoToGitea).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test("getOrCreateGiteaOrg handles JSON parsing errors gracefully", async () => {
|
||||
// Mock fetch to return invalid JSON
|
||||
const originalFetch = global.fetch;
|
||||
global.fetch = mock(async (url: string) => {
|
||||
if (url.includes("/api/v1/orgs/")) {
|
||||
// Mock response that looks successful but has invalid JSON
|
||||
return {
|
||||
ok: true,
|
||||
status: 200,
|
||||
headers: {
|
||||
get: (name: string) => name === "content-type" ? "application/json" : null
|
||||
},
|
||||
json: () => Promise.reject(new Error("Unexpected token in JSON")),
|
||||
text: () => Promise.resolve("Invalid JSON response"),
|
||||
clone: function() {
|
||||
return {
|
||||
text: () => Promise.resolve("Invalid JSON response")
|
||||
};
|
||||
}
|
||||
} as any;
|
||||
}
|
||||
return originalFetch(url);
|
||||
});
|
||||
|
||||
const config = {
|
||||
userId: "user-id",
|
||||
giteaConfig: {
|
||||
url: "https://gitea.example.com",
|
||||
token: "gitea-token"
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
await getOrCreateGiteaOrg({
|
||||
orgName: "test-org",
|
||||
config
|
||||
});
|
||||
// Should not reach here
|
||||
expect(true).toBe(false);
|
||||
} catch (error) {
|
||||
// Should catch the JSON parsing error with a descriptive message
|
||||
expect(error).toBeInstanceOf(Error);
|
||||
expect((error as Error).message).toContain("Failed to parse JSON response from Gitea API");
|
||||
} finally {
|
||||
// Restore original fetch
|
||||
global.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("getOrCreateGiteaOrg handles non-JSON content-type gracefully", async () => {
|
||||
// Mock fetch to return HTML instead of JSON
|
||||
const originalFetch = global.fetch;
|
||||
global.fetch = mock(async (url: string) => {
|
||||
if (url.includes("/api/v1/orgs/")) {
|
||||
return {
|
||||
ok: true,
|
||||
status: 200,
|
||||
headers: {
|
||||
get: (name: string) => name === "content-type" ? "text/html" : null
|
||||
},
|
||||
text: () => Promise.resolve("<html><body>Error page</body></html>")
|
||||
} as any;
|
||||
}
|
||||
return originalFetch(url);
|
||||
});
|
||||
|
||||
const config = {
|
||||
userId: "user-id",
|
||||
giteaConfig: {
|
||||
url: "https://gitea.example.com",
|
||||
token: "gitea-token"
|
||||
}
|
||||
};
|
||||
|
||||
try {
|
||||
await getOrCreateGiteaOrg({
|
||||
orgName: "test-org",
|
||||
config
|
||||
});
|
||||
// Should not reach here
|
||||
expect(true).toBe(false);
|
||||
} catch (error) {
|
||||
// Should catch the content-type error
|
||||
expect(error).toBeInstanceOf(Error);
|
||||
expect((error as Error).message).toContain("Invalid response format from Gitea API");
|
||||
expect((error as Error).message).toContain("text/html");
|
||||
} finally {
|
||||
// Restore original fetch
|
||||
global.fetch = originalFetch;
|
||||
}
|
||||
});
|
||||
|
||||
test("mirrorGitHubOrgToGitea handles empty organizations correctly", async () => {
|
||||
// Mock the createMirrorJob function
|
||||
const mockCreateMirrorJob = mock(() => Promise.resolve("job-id"));
|
||||
|
||||
// Mock the getOrCreateGiteaOrg function
|
||||
const mockGetOrCreateGiteaOrg = mock(() => Promise.resolve("gitea-org-id"));
|
||||
|
||||
// Create a test version of the function with mocked dependencies
|
||||
const testMirrorGitHubOrgToGitea = async ({
|
||||
organization,
|
||||
config,
|
||||
}: {
|
||||
organization: any;
|
||||
config: any;
|
||||
}) => {
|
||||
// Simulate the function logic for empty organization
|
||||
console.log(`Mirroring organization ${organization.name}`);
|
||||
|
||||
// Mock: get or create Gitea org
|
||||
await mockGetOrCreateGiteaOrg();
|
||||
|
||||
// Mock: query the db with the org name and get the repos
|
||||
const orgRepos: any[] = []; // Empty array to simulate no repositories
|
||||
|
||||
if (orgRepos.length === 0) {
|
||||
console.log(`No repositories found for organization ${organization.name} - marking as successfully mirrored`);
|
||||
} else {
|
||||
console.log(`Mirroring ${orgRepos.length} repositories for organization ${organization.name}`);
|
||||
// Repository processing would happen here
|
||||
}
|
||||
|
||||
console.log(`Organization ${organization.name} mirrored successfully`);
|
||||
|
||||
// Mock: Append log for "mirrored" status
|
||||
await mockCreateMirrorJob({
|
||||
userId: config.userId,
|
||||
organizationId: organization.id,
|
||||
organizationName: organization.name,
|
||||
message: `Successfully mirrored organization: ${organization.name}`,
|
||||
details: orgRepos.length === 0
|
||||
? `Organization ${organization.name} was processed successfully (no repositories found).`
|
||||
: `Organization ${organization.name} was mirrored to Gitea with ${orgRepos.length} repositories.`,
|
||||
status: "mirrored",
|
||||
});
|
||||
};
|
||||
|
||||
// Create mock organization
|
||||
const organization = {
|
||||
id: "org-id",
|
||||
name: "empty-org",
|
||||
status: "imported"
|
||||
};
|
||||
|
||||
// Create mock config
|
||||
const config = {
|
||||
id: "config-id",
|
||||
userId: "user-id",
|
||||
githubConfig: {
|
||||
token: "github-token"
|
||||
},
|
||||
giteaConfig: {
|
||||
url: "https://gitea.example.com",
|
||||
token: "gitea-token"
|
||||
}
|
||||
};
|
||||
|
||||
// Call the test function
|
||||
await testMirrorGitHubOrgToGitea({
|
||||
organization,
|
||||
config
|
||||
});
|
||||
|
||||
// Verify that the mirror job was created with the correct details for empty org
|
||||
expect(mockCreateMirrorJob).toHaveBeenCalledWith({
|
||||
userId: "user-id",
|
||||
organizationId: "org-id",
|
||||
organizationName: "empty-org",
|
||||
message: "Successfully mirrored organization: empty-org",
|
||||
details: "Organization empty-org was processed successfully (no repositories found).",
|
||||
status: "mirrored",
|
||||
});
|
||||
|
||||
// Verify that getOrCreateGiteaOrg was called
|
||||
expect(mockGetOrCreateGiteaOrg).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
331
src/lib/gitea.ts
@@ -6,7 +6,7 @@ import {
|
||||
import { Octokit } from "@octokit/rest";
|
||||
import type { Config } from "@/types/config";
|
||||
import type { Organization, Repository } from "./db/schema";
|
||||
import superagent from "superagent";
|
||||
import { httpPost, httpGet } from "./http-client";
|
||||
import { createMirrorJob } from "./helpers";
|
||||
import { db, organizations, repositories } from "./db";
|
||||
import { eq } from "drizzle-orm";
|
||||
@@ -137,8 +137,32 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
|
||||
if (isExisting) {
|
||||
console.log(
|
||||
`Repository ${repository.name} already exists in Gitea. Skipping migration.`
|
||||
`Repository ${repository.name} already exists in Gitea. Updating database status.`
|
||||
);
|
||||
|
||||
// Update database to reflect that the repository is already mirrored
|
||||
await db
|
||||
.update(repositories)
|
||||
.set({
|
||||
status: repoStatusEnum.parse("mirrored"),
|
||||
updatedAt: new Date(),
|
||||
lastMirrored: new Date(),
|
||||
errorMessage: null,
|
||||
mirroredLocation: `${config.giteaConfig.username}/${repository.name}`,
|
||||
})
|
||||
.where(eq(repositories.id, repository.id!));
|
||||
|
||||
// Append log for "mirrored" status
|
||||
await createMirrorJob({
|
||||
userId: config.userId,
|
||||
repositoryId: repository.id,
|
||||
repositoryName: repository.name,
|
||||
message: `Repository ${repository.name} already exists in Gitea`,
|
||||
details: `Repository ${repository.name} was found to already exist in Gitea and database status was updated.`,
|
||||
status: "mirrored",
|
||||
});
|
||||
|
||||
console.log(`Repository ${repository.name} database status updated to mirrored`);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -181,19 +205,17 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
|
||||
const apiUrl = `${config.giteaConfig.url}/api/v1/repos/migrate`;
|
||||
|
||||
const response = await superagent
|
||||
.post(apiUrl)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`)
|
||||
.set("Content-Type", "application/json")
|
||||
.send({
|
||||
clone_addr: cloneAddress,
|
||||
repo_name: repository.name,
|
||||
mirror: true,
|
||||
private: repository.isPrivate,
|
||||
repo_owner: config.giteaConfig.username,
|
||||
description: "",
|
||||
service: "git",
|
||||
});
|
||||
const response = await httpPost(apiUrl, {
|
||||
clone_addr: cloneAddress,
|
||||
repo_name: repository.name,
|
||||
mirror: true,
|
||||
private: repository.isPrivate,
|
||||
repo_owner: config.giteaConfig.username,
|
||||
description: "",
|
||||
service: "git",
|
||||
}, {
|
||||
"Authorization": `token ${config.giteaConfig.token}`,
|
||||
});
|
||||
|
||||
// clone issues
|
||||
if (config.githubConfig.mirrorIssues) {
|
||||
@@ -229,7 +251,7 @@ export const mirrorGithubRepoToGitea = async ({
|
||||
status: "mirrored",
|
||||
});
|
||||
|
||||
return response.body;
|
||||
return response.data;
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error while mirroring repository ${repository.name}: ${
|
||||
@@ -283,6 +305,8 @@ export async function getOrCreateGiteaOrg({
|
||||
}
|
||||
|
||||
try {
|
||||
console.log(`Attempting to get or create Gitea organization: ${orgName}`);
|
||||
|
||||
const orgRes = await fetch(
|
||||
`${config.giteaConfig.url}/api/v1/orgs/${orgName}`,
|
||||
{
|
||||
@@ -293,13 +317,36 @@ export async function getOrCreateGiteaOrg({
|
||||
}
|
||||
);
|
||||
|
||||
console.log(`Get org response status: ${orgRes.status} for org: ${orgName}`);
|
||||
|
||||
if (orgRes.ok) {
|
||||
const org = await orgRes.json();
|
||||
// Note: Organization events are handled by the main mirroring process
|
||||
// to avoid duplicate events
|
||||
return org.id;
|
||||
// Check if response is actually JSON
|
||||
const contentType = orgRes.headers.get("content-type");
|
||||
if (!contentType || !contentType.includes("application/json")) {
|
||||
console.warn(`Expected JSON response but got content-type: ${contentType}`);
|
||||
const responseText = await orgRes.text();
|
||||
console.warn(`Response body: ${responseText}`);
|
||||
throw new Error(`Invalid response format from Gitea API. Expected JSON but got: ${contentType}`);
|
||||
}
|
||||
|
||||
// Clone the response to handle potential JSON parsing errors
|
||||
const orgResClone = orgRes.clone();
|
||||
|
||||
try {
|
||||
const org = await orgRes.json();
|
||||
console.log(`Successfully retrieved existing org: ${orgName} with ID: ${org.id}`);
|
||||
// Note: Organization events are handled by the main mirroring process
|
||||
// to avoid duplicate events
|
||||
return org.id;
|
||||
} catch (jsonError) {
|
||||
const responseText = await orgResClone.text();
|
||||
console.error(`Failed to parse JSON response for existing org: ${responseText}`);
|
||||
throw new Error(`Failed to parse JSON response from Gitea API: ${jsonError instanceof Error ? jsonError.message : String(jsonError)}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`Organization ${orgName} not found, attempting to create it`);
|
||||
|
||||
const createRes = await fetch(`${config.giteaConfig.url}/api/v1/orgs`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
@@ -314,21 +361,46 @@ export async function getOrCreateGiteaOrg({
|
||||
}),
|
||||
});
|
||||
|
||||
console.log(`Create org response status: ${createRes.status} for org: ${orgName}`);
|
||||
|
||||
if (!createRes.ok) {
|
||||
throw new Error(`Failed to create Gitea org: ${await createRes.text()}`);
|
||||
const errorText = await createRes.text();
|
||||
console.error(`Failed to create org ${orgName}. Status: ${createRes.status}, Response: ${errorText}`);
|
||||
throw new Error(`Failed to create Gitea org: ${errorText}`);
|
||||
}
|
||||
|
||||
// Check if response is actually JSON
|
||||
const createContentType = createRes.headers.get("content-type");
|
||||
if (!createContentType || !createContentType.includes("application/json")) {
|
||||
console.warn(`Expected JSON response but got content-type: ${createContentType}`);
|
||||
const responseText = await createRes.text();
|
||||
console.warn(`Response body: ${responseText}`);
|
||||
throw new Error(`Invalid response format from Gitea API. Expected JSON but got: ${createContentType}`);
|
||||
}
|
||||
|
||||
// Note: Organization creation events are handled by the main mirroring process
|
||||
// to avoid duplicate events
|
||||
|
||||
const newOrg = await createRes.json();
|
||||
return newOrg.id;
|
||||
// Clone the response to handle potential JSON parsing errors
|
||||
const createResClone = createRes.clone();
|
||||
|
||||
try {
|
||||
const newOrg = await createRes.json();
|
||||
console.log(`Successfully created new org: ${orgName} with ID: ${newOrg.id}`);
|
||||
return newOrg.id;
|
||||
} catch (jsonError) {
|
||||
const responseText = await createResClone.text();
|
||||
console.error(`Failed to parse JSON response for new org: ${responseText}`);
|
||||
throw new Error(`Failed to parse JSON response from Gitea API: ${jsonError instanceof Error ? jsonError.message : String(jsonError)}`);
|
||||
}
|
||||
} catch (error) {
|
||||
const errorMessage =
|
||||
error instanceof Error
|
||||
? error.message
|
||||
: "Unknown error occurred in getOrCreateGiteaOrg.";
|
||||
|
||||
console.error(`Error in getOrCreateGiteaOrg for ${orgName}: ${errorMessage}`);
|
||||
|
||||
await createMirrorJob({
|
||||
userId: config.userId,
|
||||
organizationId: orgId,
|
||||
@@ -372,8 +444,32 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
|
||||
if (isExisting) {
|
||||
console.log(
|
||||
`Repository ${repository.name} already exists in Gitea. Skipping migration.`
|
||||
`Repository ${repository.name} already exists in Gitea organization ${orgName}. Updating database status.`
|
||||
);
|
||||
|
||||
// Update database to reflect that the repository is already mirrored
|
||||
await db
|
||||
.update(repositories)
|
||||
.set({
|
||||
status: repoStatusEnum.parse("mirrored"),
|
||||
updatedAt: new Date(),
|
||||
lastMirrored: new Date(),
|
||||
errorMessage: null,
|
||||
mirroredLocation: `${orgName}/${repository.name}`,
|
||||
})
|
||||
.where(eq(repositories.id, repository.id!));
|
||||
|
||||
// Create a mirror job log entry
|
||||
await createMirrorJob({
|
||||
userId: config.userId,
|
||||
repositoryId: repository.id,
|
||||
repositoryName: repository.name,
|
||||
message: `Repository ${repository.name} already exists in Gitea organization ${orgName}`,
|
||||
details: `Repository ${repository.name} was found to already exist in Gitea organization ${orgName} and database status was updated.`,
|
||||
status: "mirrored",
|
||||
});
|
||||
|
||||
console.log(`Repository ${repository.name} database status updated to mirrored in organization ${orgName}`);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -410,17 +506,15 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
|
||||
const apiUrl = `${config.giteaConfig.url}/api/v1/repos/migrate`;
|
||||
|
||||
const migrateRes = await superagent
|
||||
.post(apiUrl)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`)
|
||||
.set("Content-Type", "application/json")
|
||||
.send({
|
||||
clone_addr: cloneAddress,
|
||||
uid: giteaOrgId,
|
||||
repo_name: repository.name,
|
||||
mirror: true,
|
||||
private: repository.isPrivate,
|
||||
});
|
||||
const migrateRes = await httpPost(apiUrl, {
|
||||
clone_addr: cloneAddress,
|
||||
uid: giteaOrgId,
|
||||
repo_name: repository.name,
|
||||
mirror: true,
|
||||
private: repository.isPrivate,
|
||||
}, {
|
||||
"Authorization": `token ${config.giteaConfig.token}`,
|
||||
});
|
||||
|
||||
// Clone issues
|
||||
if (config.githubConfig?.mirrorIssues) {
|
||||
@@ -458,7 +552,7 @@ export async function mirrorGitHubRepoToGiteaOrg({
|
||||
status: "mirrored",
|
||||
});
|
||||
|
||||
return migrateRes.body;
|
||||
return migrateRes.data;
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error while mirroring repository ${repository.name}: ${
|
||||
@@ -583,60 +677,59 @@ export async function mirrorGitHubOrgToGitea({
|
||||
.where(eq(repositories.organization, organization.name));
|
||||
|
||||
if (orgRepos.length === 0) {
|
||||
console.log(`No repositories found for organization ${organization.name}`);
|
||||
return;
|
||||
}
|
||||
console.log(`No repositories found for organization ${organization.name} - marking as successfully mirrored`);
|
||||
} else {
|
||||
console.log(`Mirroring ${orgRepos.length} repositories for organization ${organization.name}`);
|
||||
|
||||
console.log(`Mirroring ${orgRepos.length} repositories for organization ${organization.name}`);
|
||||
// Import the processWithRetry function
|
||||
const { processWithRetry } = await import("@/lib/utils/concurrency");
|
||||
|
||||
// Import the processWithRetry function
|
||||
const { processWithRetry } = await import("@/lib/utils/concurrency");
|
||||
// Process repositories in parallel with concurrency control
|
||||
await processWithRetry(
|
||||
orgRepos,
|
||||
async (repo) => {
|
||||
// Prepare repository data
|
||||
const repoData = {
|
||||
...repo,
|
||||
status: repo.status as RepoStatus,
|
||||
visibility: repo.visibility as RepositoryVisibility,
|
||||
lastMirrored: repo.lastMirrored ?? undefined,
|
||||
errorMessage: repo.errorMessage ?? undefined,
|
||||
organization: repo.organization ?? undefined,
|
||||
forkedFrom: repo.forkedFrom ?? undefined,
|
||||
mirroredLocation: repo.mirroredLocation || "",
|
||||
};
|
||||
|
||||
// Process repositories in parallel with concurrency control
|
||||
await processWithRetry(
|
||||
orgRepos,
|
||||
async (repo) => {
|
||||
// Prepare repository data
|
||||
const repoData = {
|
||||
...repo,
|
||||
status: repo.status as RepoStatus,
|
||||
visibility: repo.visibility as RepositoryVisibility,
|
||||
lastMirrored: repo.lastMirrored ?? undefined,
|
||||
errorMessage: repo.errorMessage ?? undefined,
|
||||
organization: repo.organization ?? undefined,
|
||||
forkedFrom: repo.forkedFrom ?? undefined,
|
||||
mirroredLocation: repo.mirroredLocation || "",
|
||||
};
|
||||
// Log the start of mirroring
|
||||
console.log(`Starting mirror for repository: ${repo.name} in organization ${organization.name}`);
|
||||
|
||||
// Log the start of mirroring
|
||||
console.log(`Starting mirror for repository: ${repo.name} in organization ${organization.name}`);
|
||||
// Mirror the repository
|
||||
await mirrorGitHubRepoToGiteaOrg({
|
||||
octokit,
|
||||
config,
|
||||
repository: repoData,
|
||||
giteaOrgId,
|
||||
orgName: organization.name,
|
||||
});
|
||||
|
||||
// Mirror the repository
|
||||
await mirrorGitHubRepoToGiteaOrg({
|
||||
octokit,
|
||||
config,
|
||||
repository: repoData,
|
||||
giteaOrgId,
|
||||
orgName: organization.name,
|
||||
});
|
||||
|
||||
return repo;
|
||||
},
|
||||
{
|
||||
concurrencyLimit: 3, // Process 3 repositories at a time
|
||||
maxRetries: 2,
|
||||
retryDelay: 2000,
|
||||
onProgress: (completed, total, result) => {
|
||||
const percentComplete = Math.round((completed / total) * 100);
|
||||
if (result) {
|
||||
console.log(`Mirrored repository "${result.name}" in organization ${organization.name} (${completed}/${total}, ${percentComplete}%)`);
|
||||
}
|
||||
return repo;
|
||||
},
|
||||
onRetry: (repo, error, attempt) => {
|
||||
console.log(`Retrying repository ${repo.name} in organization ${organization.name} (attempt ${attempt}): ${error.message}`);
|
||||
{
|
||||
concurrencyLimit: 3, // Process 3 repositories at a time
|
||||
maxRetries: 2,
|
||||
retryDelay: 2000,
|
||||
onProgress: (completed, total, result) => {
|
||||
const percentComplete = Math.round((completed / total) * 100);
|
||||
if (result) {
|
||||
console.log(`Mirrored repository "${result.name}" in organization ${organization.name} (${completed}/${total}, ${percentComplete}%)`);
|
||||
}
|
||||
},
|
||||
onRetry: (repo, error, attempt) => {
|
||||
console.log(`Retrying repository ${repo.name} in organization ${organization.name} (attempt ${attempt}): ${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
);
|
||||
}
|
||||
|
||||
console.log(`Organization ${organization.name} mirrored successfully`);
|
||||
|
||||
@@ -657,7 +750,9 @@ export async function mirrorGitHubOrgToGitea({
|
||||
organizationId: organization.id,
|
||||
organizationName: organization.name,
|
||||
message: `Successfully mirrored organization: ${organization.name}`,
|
||||
details: `Organization ${organization.name} was mirrored to Gitea.`,
|
||||
details: orgRepos.length === 0
|
||||
? `Organization ${organization.name} was processed successfully (no repositories found).`
|
||||
: `Organization ${organization.name} was mirrored to Gitea with ${orgRepos.length} repositories.`,
|
||||
status: repoStatusEnum.parse("mirrored"),
|
||||
});
|
||||
} catch (error) {
|
||||
@@ -751,9 +846,9 @@ export const syncGiteaRepo = async ({
|
||||
// Use the actual owner where the repo was found
|
||||
const apiUrl = `${config.giteaConfig.url}/api/v1/repos/${actualOwner}/${repository.name}/mirror-sync`;
|
||||
|
||||
const response = await superagent
|
||||
.post(apiUrl)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`);
|
||||
const response = await httpPost(apiUrl, undefined, {
|
||||
"Authorization": `token ${config.giteaConfig.token}`,
|
||||
});
|
||||
|
||||
// Mark repo as "synced" in DB
|
||||
await db
|
||||
@@ -779,7 +874,7 @@ export const syncGiteaRepo = async ({
|
||||
|
||||
console.log(`Repository ${repository.name} synced successfully`);
|
||||
|
||||
return response.body;
|
||||
return response.data;
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error while syncing repository ${repository.name}: ${
|
||||
@@ -866,13 +961,14 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
}
|
||||
|
||||
// Get existing labels from Gitea
|
||||
const giteaLabelsRes = await superagent
|
||||
.get(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOrigin}/${repository.name}/labels`
|
||||
)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`);
|
||||
const giteaLabelsRes = await httpGet(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOrigin}/${repository.name}/labels`,
|
||||
{
|
||||
"Authorization": `token ${config.giteaConfig.token}`,
|
||||
}
|
||||
);
|
||||
|
||||
const giteaLabels = giteaLabelsRes.body;
|
||||
const giteaLabels = giteaLabelsRes.data;
|
||||
const labelMap = new Map<string, number>(
|
||||
giteaLabels.map((label: any) => [label.name, label.id])
|
||||
);
|
||||
@@ -897,15 +993,16 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
giteaLabelIds.push(labelMap.get(name)!);
|
||||
} else {
|
||||
try {
|
||||
const created = await superagent
|
||||
.post(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOrigin}/${repository.name}/labels`
|
||||
)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`)
|
||||
.send({ name, color: "#ededed" }); // Default color
|
||||
const created = await httpPost(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${repoOrigin}/${repository.name}/labels`,
|
||||
{ name, color: "#ededed" }, // Default color
|
||||
{
|
||||
"Authorization": `token ${config.giteaConfig!.token}`,
|
||||
}
|
||||
);
|
||||
|
||||
labelMap.set(name, created.body.id);
|
||||
giteaLabelIds.push(created.body.id);
|
||||
labelMap.set(name, created.data.id);
|
||||
giteaLabelIds.push(created.data.id);
|
||||
} catch (labelErr) {
|
||||
console.error(
|
||||
`Failed to create label "${name}" in Gitea: ${labelErr}`
|
||||
@@ -931,12 +1028,13 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
};
|
||||
|
||||
// Create the issue in Gitea
|
||||
const createdIssue = await superagent
|
||||
.post(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOrigin}/${repository.name}/issues`
|
||||
)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`)
|
||||
.send(issuePayload);
|
||||
const createdIssue = await httpPost(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${repoOrigin}/${repository.name}/issues`,
|
||||
issuePayload,
|
||||
{
|
||||
"Authorization": `token ${config.giteaConfig!.token}`,
|
||||
}
|
||||
);
|
||||
|
||||
// Clone comments
|
||||
const comments = await octokit.paginate(
|
||||
@@ -955,21 +1053,22 @@ export const mirrorGitRepoIssuesToGitea = async ({
|
||||
await processWithRetry(
|
||||
comments,
|
||||
async (comment) => {
|
||||
await superagent
|
||||
.post(
|
||||
`${config.giteaConfig.url}/api/v1/repos/${repoOrigin}/${repository.name}/issues/${createdIssue.body.number}/comments`
|
||||
)
|
||||
.set("Authorization", `token ${config.giteaConfig.token}`)
|
||||
.send({
|
||||
await httpPost(
|
||||
`${config.giteaConfig!.url}/api/v1/repos/${repoOrigin}/${repository.name}/issues/${createdIssue.data.number}/comments`,
|
||||
{
|
||||
body: `@${comment.user?.login} commented on GitHub:\n\n${comment.body}`,
|
||||
});
|
||||
},
|
||||
{
|
||||
"Authorization": `token ${config.giteaConfig!.token}`,
|
||||
}
|
||||
);
|
||||
return comment;
|
||||
},
|
||||
{
|
||||
concurrencyLimit: 5,
|
||||
maxRetries: 2,
|
||||
retryDelay: 1000,
|
||||
onRetry: (comment, error, attempt) => {
|
||||
onRetry: (_comment, error, attempt) => {
|
||||
console.log(`Retrying comment (attempt ${attempt}): ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
204
src/lib/http-client.ts
Normal file
@@ -0,0 +1,204 @@
|
||||
/**
|
||||
* HTTP client utility functions using fetch() for consistent error handling
|
||||
*/
|
||||
|
||||
export interface HttpResponse<T = any> {
|
||||
data: T;
|
||||
status: number;
|
||||
statusText: string;
|
||||
headers: Headers;
|
||||
}
|
||||
|
||||
export class HttpError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public status: number,
|
||||
public statusText: string,
|
||||
public response?: string
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'HttpError';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Enhanced fetch with consistent error handling and JSON parsing
|
||||
*/
|
||||
export async function httpRequest<T = any>(
|
||||
url: string,
|
||||
options: RequestInit = {}
|
||||
): Promise<HttpResponse<T>> {
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
...options,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...options.headers,
|
||||
},
|
||||
});
|
||||
|
||||
// Clone response for error handling
|
||||
const responseClone = response.clone();
|
||||
|
||||
if (!response.ok) {
|
||||
let errorMessage = `HTTP ${response.status}: ${response.statusText}`;
|
||||
let responseText = '';
|
||||
|
||||
try {
|
||||
responseText = await responseClone.text();
|
||||
if (responseText) {
|
||||
errorMessage += ` - ${responseText}`;
|
||||
}
|
||||
} catch {
|
||||
// Ignore text parsing errors
|
||||
}
|
||||
|
||||
throw new HttpError(
|
||||
errorMessage,
|
||||
response.status,
|
||||
response.statusText,
|
||||
responseText
|
||||
);
|
||||
}
|
||||
|
||||
// Check content type for JSON responses
|
||||
const contentType = response.headers.get('content-type');
|
||||
let data: T;
|
||||
|
||||
if (contentType && contentType.includes('application/json')) {
|
||||
try {
|
||||
data = await response.json();
|
||||
} catch (jsonError) {
|
||||
const responseText = await responseClone.text();
|
||||
|
||||
// Enhanced JSON parsing error logging
|
||||
console.error("=== JSON PARSING ERROR ===");
|
||||
console.error("URL:", url);
|
||||
console.error("Status:", response.status, response.statusText);
|
||||
console.error("Content-Type:", contentType);
|
||||
console.error("Response length:", responseText.length);
|
||||
console.error("Response preview (first 500 chars):", responseText.substring(0, 500));
|
||||
console.error("JSON Error:", jsonError instanceof Error ? jsonError.message : String(jsonError));
|
||||
console.error("========================");
|
||||
|
||||
throw new HttpError(
|
||||
`Failed to parse JSON response from ${url}: ${jsonError instanceof Error ? jsonError.message : String(jsonError)}. Response: ${responseText.substring(0, 200)}${responseText.length > 200 ? '...' : ''}`,
|
||||
response.status,
|
||||
response.statusText,
|
||||
responseText
|
||||
);
|
||||
}
|
||||
} else {
|
||||
// For non-JSON responses, return text as data
|
||||
data = (await response.text()) as unknown as T;
|
||||
}
|
||||
|
||||
return {
|
||||
data,
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
headers: response.headers,
|
||||
};
|
||||
} catch (error) {
|
||||
if (error instanceof HttpError) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Handle network errors, etc.
|
||||
throw new HttpError(
|
||||
`Network error: ${error instanceof Error ? error.message : String(error)}`,
|
||||
0,
|
||||
'Network Error'
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* GET request
|
||||
*/
|
||||
export async function httpGet<T = any>(
|
||||
url: string,
|
||||
headers?: Record<string, string>
|
||||
): Promise<HttpResponse<T>> {
|
||||
return httpRequest<T>(url, {
|
||||
method: 'GET',
|
||||
headers,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* POST request
|
||||
*/
|
||||
export async function httpPost<T = any>(
|
||||
url: string,
|
||||
body?: any,
|
||||
headers?: Record<string, string>
|
||||
): Promise<HttpResponse<T>> {
|
||||
return httpRequest<T>(url, {
|
||||
method: 'POST',
|
||||
headers,
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* PUT request
|
||||
*/
|
||||
export async function httpPut<T = any>(
|
||||
url: string,
|
||||
body?: any,
|
||||
headers?: Record<string, string>
|
||||
): Promise<HttpResponse<T>> {
|
||||
return httpRequest<T>(url, {
|
||||
method: 'PUT',
|
||||
headers,
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* DELETE request
|
||||
*/
|
||||
export async function httpDelete<T = any>(
|
||||
url: string,
|
||||
headers?: Record<string, string>
|
||||
): Promise<HttpResponse<T>> {
|
||||
return httpRequest<T>(url, {
|
||||
method: 'DELETE',
|
||||
headers,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Gitea-specific HTTP client with authentication
|
||||
*/
|
||||
export class GiteaHttpClient {
|
||||
constructor(
|
||||
private baseUrl: string,
|
||||
private token: string
|
||||
) {}
|
||||
|
||||
private getHeaders(additionalHeaders?: Record<string, string>): Record<string, string> {
|
||||
return {
|
||||
'Authorization': `token ${this.token}`,
|
||||
'Content-Type': 'application/json',
|
||||
...additionalHeaders,
|
||||
};
|
||||
}
|
||||
|
||||
async get<T = any>(endpoint: string): Promise<HttpResponse<T>> {
|
||||
return httpGet<T>(`${this.baseUrl}${endpoint}`, this.getHeaders());
|
||||
}
|
||||
|
||||
async post<T = any>(endpoint: string, body?: any): Promise<HttpResponse<T>> {
|
||||
return httpPost<T>(`${this.baseUrl}${endpoint}`, body, this.getHeaders());
|
||||
}
|
||||
|
||||
async put<T = any>(endpoint: string, body?: any): Promise<HttpResponse<T>> {
|
||||
return httpPut<T>(`${this.baseUrl}${endpoint}`, body, this.getHeaders());
|
||||
}
|
||||
|
||||
async delete<T = any>(endpoint: string): Promise<HttpResponse<T>> {
|
||||
return httpDelete<T>(`${this.baseUrl}${endpoint}`, this.getHeaders());
|
||||
}
|
||||
}
|
||||
@@ -1,35 +1,35 @@
|
||||
import { describe, test, expect } from "bun:test";
|
||||
import { jsonResponse, formatDate, truncate, safeParse } from "./utils";
|
||||
import { jsonResponse, formatDate, truncate, safeParse, parseErrorMessage, showErrorToast } from "./utils";
|
||||
|
||||
describe("jsonResponse", () => {
|
||||
test("creates a Response with JSON content", () => {
|
||||
const data = { message: "Hello, world!" };
|
||||
const response = jsonResponse({ data });
|
||||
|
||||
|
||||
expect(response).toBeInstanceOf(Response);
|
||||
expect(response.status).toBe(200);
|
||||
expect(response.headers.get("Content-Type")).toBe("application/json");
|
||||
});
|
||||
|
||||
|
||||
test("uses the provided status code", () => {
|
||||
const data = { error: "Not found" };
|
||||
const response = jsonResponse({ data, status: 404 });
|
||||
|
||||
|
||||
expect(response.status).toBe(404);
|
||||
});
|
||||
|
||||
|
||||
test("correctly serializes complex objects", async () => {
|
||||
const now = new Date();
|
||||
const data = {
|
||||
const data = {
|
||||
message: "Complex object",
|
||||
date: now,
|
||||
nested: { foo: "bar" },
|
||||
array: [1, 2, 3]
|
||||
};
|
||||
|
||||
|
||||
const response = jsonResponse({ data });
|
||||
const responseBody = await response.json();
|
||||
|
||||
|
||||
expect(responseBody).toEqual({
|
||||
message: "Complex object",
|
||||
date: now.toISOString(),
|
||||
@@ -43,22 +43,22 @@ describe("formatDate", () => {
|
||||
test("formats a date object", () => {
|
||||
const date = new Date("2023-01-15T12:30:45Z");
|
||||
const formatted = formatDate(date);
|
||||
|
||||
|
||||
// The exact format might depend on the locale, so we'll check for parts
|
||||
expect(formatted).toContain("2023");
|
||||
expect(formatted).toContain("January");
|
||||
expect(formatted).toContain("15");
|
||||
});
|
||||
|
||||
|
||||
test("formats a date string", () => {
|
||||
const dateStr = "2023-01-15T12:30:45Z";
|
||||
const formatted = formatDate(dateStr);
|
||||
|
||||
|
||||
expect(formatted).toContain("2023");
|
||||
expect(formatted).toContain("January");
|
||||
expect(formatted).toContain("15");
|
||||
});
|
||||
|
||||
|
||||
test("returns 'Never' for null or undefined", () => {
|
||||
expect(formatDate(null)).toBe("Never");
|
||||
expect(formatDate(undefined)).toBe("Never");
|
||||
@@ -69,18 +69,18 @@ describe("truncate", () => {
|
||||
test("truncates a string that exceeds the length", () => {
|
||||
const str = "This is a long string that needs truncation";
|
||||
const truncated = truncate(str, 10);
|
||||
|
||||
|
||||
expect(truncated).toBe("This is a ...");
|
||||
expect(truncated.length).toBe(13); // 10 chars + "..."
|
||||
});
|
||||
|
||||
|
||||
test("does not truncate a string that is shorter than the length", () => {
|
||||
const str = "Short";
|
||||
const truncated = truncate(str, 10);
|
||||
|
||||
|
||||
expect(truncated).toBe("Short");
|
||||
});
|
||||
|
||||
|
||||
test("handles empty strings", () => {
|
||||
expect(truncate("", 10)).toBe("");
|
||||
});
|
||||
@@ -90,21 +90,71 @@ describe("safeParse", () => {
|
||||
test("parses valid JSON strings", () => {
|
||||
const jsonStr = '{"name":"John","age":30}';
|
||||
const parsed = safeParse(jsonStr);
|
||||
|
||||
|
||||
expect(parsed).toEqual({ name: "John", age: 30 });
|
||||
});
|
||||
|
||||
|
||||
test("returns undefined for invalid JSON strings", () => {
|
||||
const invalidJson = '{"name":"John",age:30}'; // Missing quotes around age
|
||||
const parsed = safeParse(invalidJson);
|
||||
|
||||
|
||||
expect(parsed).toBeUndefined();
|
||||
});
|
||||
|
||||
|
||||
test("returns the original value for non-string inputs", () => {
|
||||
const obj = { name: "John", age: 30 };
|
||||
const parsed = safeParse(obj);
|
||||
|
||||
|
||||
expect(parsed).toBe(obj);
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseErrorMessage", () => {
|
||||
test("parses JSON error with error and troubleshooting fields", () => {
|
||||
const errorMessage = JSON.stringify({
|
||||
error: "Unexpected end of JSON input",
|
||||
errorType: "SyntaxError",
|
||||
timestamp: "2025-05-28T09:08:02.37Z",
|
||||
troubleshooting: "JSON parsing error detected. Check Gitea server status and logs. Ensure Gitea is returning valid JSON responses."
|
||||
});
|
||||
|
||||
const result = parseErrorMessage(errorMessage);
|
||||
|
||||
expect(result.title).toBe("Unexpected end of JSON input");
|
||||
expect(result.description).toBe("JSON parsing error detected. Check Gitea server status and logs. Ensure Gitea is returning valid JSON responses.");
|
||||
expect(result.isStructured).toBe(true);
|
||||
});
|
||||
|
||||
test("parses JSON error with title and description fields", () => {
|
||||
const errorMessage = JSON.stringify({
|
||||
title: "Connection Failed",
|
||||
description: "Unable to connect to the server. Please check your network connection."
|
||||
});
|
||||
|
||||
const result = parseErrorMessage(errorMessage);
|
||||
|
||||
expect(result.title).toBe("Connection Failed");
|
||||
expect(result.description).toBe("Unable to connect to the server. Please check your network connection.");
|
||||
expect(result.isStructured).toBe(true);
|
||||
});
|
||||
|
||||
test("handles plain string error messages", () => {
|
||||
const errorMessage = "Simple error message";
|
||||
|
||||
const result = parseErrorMessage(errorMessage);
|
||||
|
||||
expect(result.title).toBe("Simple error message");
|
||||
expect(result.description).toBeUndefined();
|
||||
expect(result.isStructured).toBe(false);
|
||||
});
|
||||
|
||||
test("handles Error objects", () => {
|
||||
const error = new Error("Something went wrong");
|
||||
|
||||
const result = parseErrorMessage(error);
|
||||
|
||||
expect(result.title).toBe("Something went wrong");
|
||||
expect(result.description).toBeUndefined();
|
||||
expect(result.isStructured).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
138
src/lib/utils.ts
@@ -1,7 +1,6 @@
|
||||
import { clsx, type ClassValue } from "clsx";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import axios from "axios";
|
||||
import type { AxiosError, AxiosRequestConfig } from "axios";
|
||||
import { httpRequest, HttpError } from "@/lib/http-client";
|
||||
import type { RepoStatus } from "@/types/Repository";
|
||||
|
||||
export const API_BASE = "/api";
|
||||
@@ -37,27 +36,148 @@ export function safeParse<T>(value: unknown): T | undefined {
|
||||
return value as T;
|
||||
}
|
||||
|
||||
// Enhanced error message parsing for toast notifications
|
||||
export interface ParsedErrorMessage {
|
||||
title: string;
|
||||
description?: string;
|
||||
isStructured: boolean;
|
||||
}
|
||||
|
||||
export function parseErrorMessage(error: unknown): ParsedErrorMessage {
|
||||
// Handle Error objects
|
||||
if (error instanceof Error) {
|
||||
return parseErrorMessage(error.message);
|
||||
}
|
||||
|
||||
// Handle string messages
|
||||
if (typeof error === "string") {
|
||||
// Try to parse as JSON first
|
||||
try {
|
||||
const parsed = JSON.parse(error);
|
||||
|
||||
// Check for common structured error formats
|
||||
if (typeof parsed === "object" && parsed !== null) {
|
||||
// Format 1: { error: "message", errorType: "type", troubleshooting: "info" }
|
||||
if (parsed.error) {
|
||||
return {
|
||||
title: parsed.error,
|
||||
description: parsed.troubleshooting || parsed.errorType || undefined,
|
||||
isStructured: true,
|
||||
};
|
||||
}
|
||||
|
||||
// Format 2: { title: "title", description: "desc" }
|
||||
if (parsed.title) {
|
||||
return {
|
||||
title: parsed.title,
|
||||
description: parsed.description || undefined,
|
||||
isStructured: true,
|
||||
};
|
||||
}
|
||||
|
||||
// Format 3: { message: "msg", details: "details" }
|
||||
if (parsed.message) {
|
||||
return {
|
||||
title: parsed.message,
|
||||
description: parsed.details || undefined,
|
||||
isStructured: true,
|
||||
};
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Not valid JSON, treat as plain string
|
||||
}
|
||||
|
||||
// Plain string message
|
||||
return {
|
||||
title: error,
|
||||
description: undefined,
|
||||
isStructured: false,
|
||||
};
|
||||
}
|
||||
|
||||
// Handle objects directly
|
||||
if (typeof error === "object" && error !== null) {
|
||||
const errorObj = error as any;
|
||||
|
||||
if (errorObj.error) {
|
||||
return {
|
||||
title: errorObj.error,
|
||||
description: errorObj.troubleshooting || errorObj.errorType || undefined,
|
||||
isStructured: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (errorObj.title) {
|
||||
return {
|
||||
title: errorObj.title,
|
||||
description: errorObj.description || undefined,
|
||||
isStructured: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (errorObj.message) {
|
||||
return {
|
||||
title: errorObj.message,
|
||||
description: errorObj.details || undefined,
|
||||
isStructured: true,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback for unknown types
|
||||
return {
|
||||
title: String(error),
|
||||
description: undefined,
|
||||
isStructured: false,
|
||||
};
|
||||
}
|
||||
|
||||
// Enhanced toast helper that parses structured error messages
|
||||
export function showErrorToast(error: unknown, toast: any) {
|
||||
const parsed = parseErrorMessage(error);
|
||||
|
||||
if (parsed.description) {
|
||||
// Use sonner's rich toast format with title and description
|
||||
toast.error(parsed.title, {
|
||||
description: parsed.description,
|
||||
});
|
||||
} else {
|
||||
// Simple error toast
|
||||
toast.error(parsed.title);
|
||||
}
|
||||
}
|
||||
|
||||
// Helper function for API requests
|
||||
|
||||
export async function apiRequest<T>(
|
||||
endpoint: string,
|
||||
options: AxiosRequestConfig = {}
|
||||
options: (RequestInit & { data?: any }) = {}
|
||||
): Promise<T> {
|
||||
try {
|
||||
const response = await axios<T>(`${API_BASE}${endpoint}`, {
|
||||
// Handle the custom 'data' property by converting it to 'body'
|
||||
const { data, ...requestOptions } = options;
|
||||
const finalOptions: RequestInit = {
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
...(options.headers || {}),
|
||||
...(requestOptions.headers || {}),
|
||||
},
|
||||
...options,
|
||||
});
|
||||
...requestOptions,
|
||||
};
|
||||
|
||||
// If data is provided, stringify it and set as body
|
||||
if (data !== undefined) {
|
||||
finalOptions.body = JSON.stringify(data);
|
||||
}
|
||||
|
||||
const response = await httpRequest<T>(`${API_BASE}${endpoint}`, finalOptions);
|
||||
|
||||
return response.data;
|
||||
} catch (err) {
|
||||
const error = err as AxiosError<{ message?: string }>;
|
||||
const error = err as HttpError;
|
||||
|
||||
const message =
|
||||
error.response?.data?.message ||
|
||||
error.response ||
|
||||
error.message ||
|
||||
"An unknown error occurred";
|
||||
|
||||
|
||||
@@ -5,19 +5,19 @@ describe("processInParallel", () => {
|
||||
test("processes items in parallel with concurrency control", async () => {
|
||||
// Create an array of numbers to process
|
||||
const items = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
|
||||
|
||||
|
||||
// Create a mock function to track execution
|
||||
const processItem = mock(async (item: number) => {
|
||||
// Simulate async work
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
return item * 2;
|
||||
});
|
||||
|
||||
|
||||
// Create a mock progress callback
|
||||
const onProgress = mock((completed: number, total: number, result?: number) => {
|
||||
// Progress tracking
|
||||
});
|
||||
|
||||
|
||||
// Process the items with a concurrency limit of 3
|
||||
const results = await processInParallel(
|
||||
items,
|
||||
@@ -25,25 +25,25 @@ describe("processInParallel", () => {
|
||||
3,
|
||||
onProgress
|
||||
);
|
||||
|
||||
|
||||
// Verify results
|
||||
expect(results).toEqual([2, 4, 6, 8, 10, 12, 14, 16, 18, 20]);
|
||||
|
||||
|
||||
// Verify that processItem was called for each item
|
||||
expect(processItem).toHaveBeenCalledTimes(10);
|
||||
|
||||
|
||||
// Verify that onProgress was called for each item
|
||||
expect(onProgress).toHaveBeenCalledTimes(10);
|
||||
|
||||
|
||||
// Verify the last call to onProgress had the correct completed/total values
|
||||
expect(onProgress.mock.calls[9][0]).toBe(10); // completed
|
||||
expect(onProgress.mock.calls[9][1]).toBe(10); // total
|
||||
});
|
||||
|
||||
|
||||
test("handles errors in processing", async () => {
|
||||
// Create an array of numbers to process
|
||||
const items = [1, 2, 3, 4, 5];
|
||||
|
||||
|
||||
// Create a mock function that throws an error for item 3
|
||||
const processItem = mock(async (item: number) => {
|
||||
if (item === 3) {
|
||||
@@ -51,24 +51,24 @@ describe("processInParallel", () => {
|
||||
}
|
||||
return item * 2;
|
||||
});
|
||||
|
||||
|
||||
// Create a spy for console.error
|
||||
const originalConsoleError = console.error;
|
||||
const consoleErrorMock = mock(() => {});
|
||||
console.error = consoleErrorMock;
|
||||
|
||||
|
||||
try {
|
||||
// Process the items
|
||||
const results = await processInParallel(items, processItem);
|
||||
|
||||
|
||||
// Verify results (should have 4 items, missing the one that errored)
|
||||
expect(results).toEqual([2, 4, 8, 10]);
|
||||
|
||||
|
||||
// Verify that processItem was called for each item
|
||||
expect(processItem).toHaveBeenCalledTimes(5);
|
||||
|
||||
// Verify that console.error was called once
|
||||
expect(consoleErrorMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Verify that console.error was called (enhanced logging calls it multiple times)
|
||||
expect(consoleErrorMock).toHaveBeenCalled();
|
||||
} finally {
|
||||
// Restore console.error
|
||||
console.error = originalConsoleError;
|
||||
@@ -80,51 +80,51 @@ describe("processWithRetry", () => {
|
||||
test("retries failed operations", async () => {
|
||||
// Create an array of numbers to process
|
||||
const items = [1, 2, 3];
|
||||
|
||||
|
||||
// Create a counter to track retry attempts
|
||||
const attemptCounts: Record<number, number> = { 1: 0, 2: 0, 3: 0 };
|
||||
|
||||
|
||||
// Create a mock function that fails on first attempt for item 2
|
||||
const processItem = mock(async (item: number) => {
|
||||
attemptCounts[item]++;
|
||||
|
||||
|
||||
if (item === 2 && attemptCounts[item] === 1) {
|
||||
throw new Error("Temporary error");
|
||||
}
|
||||
|
||||
|
||||
return item * 2;
|
||||
});
|
||||
|
||||
|
||||
// Create a mock for the onRetry callback
|
||||
const onRetry = mock((item: number, error: Error, attempt: number) => {
|
||||
// Retry tracking
|
||||
});
|
||||
|
||||
|
||||
// Process the items with retry
|
||||
const results = await processWithRetry(items, processItem, {
|
||||
maxRetries: 2,
|
||||
retryDelay: 10,
|
||||
onRetry,
|
||||
});
|
||||
|
||||
|
||||
// Verify results
|
||||
expect(results).toEqual([2, 4, 6]);
|
||||
|
||||
|
||||
// Verify that item 2 was retried once
|
||||
expect(attemptCounts[1]).toBe(1); // No retries
|
||||
expect(attemptCounts[2]).toBe(2); // One retry
|
||||
expect(attemptCounts[3]).toBe(1); // No retries
|
||||
|
||||
|
||||
// Verify that onRetry was called once
|
||||
expect(onRetry).toHaveBeenCalledTimes(1);
|
||||
expect(onRetry.mock.calls[0][0]).toBe(2); // item
|
||||
expect(onRetry.mock.calls[0][2]).toBe(1); // attempt
|
||||
});
|
||||
|
||||
|
||||
test("gives up after max retries", async () => {
|
||||
// Create an array of numbers to process
|
||||
const items = [1, 2];
|
||||
|
||||
|
||||
// Create a mock function that always fails for item 2
|
||||
const processItem = mock(async (item: number) => {
|
||||
if (item === 2) {
|
||||
@@ -132,17 +132,17 @@ describe("processWithRetry", () => {
|
||||
}
|
||||
return item * 2;
|
||||
});
|
||||
|
||||
|
||||
// Create a mock for the onRetry callback
|
||||
const onRetry = mock((item: number, error: Error, attempt: number) => {
|
||||
// Retry tracking
|
||||
});
|
||||
|
||||
|
||||
// Create a spy for console.error
|
||||
const originalConsoleError = console.error;
|
||||
const consoleErrorMock = mock(() => {});
|
||||
console.error = consoleErrorMock;
|
||||
|
||||
|
||||
try {
|
||||
// Process the items with retry
|
||||
const results = await processWithRetry(items, processItem, {
|
||||
@@ -150,15 +150,15 @@ describe("processWithRetry", () => {
|
||||
retryDelay: 10,
|
||||
onRetry,
|
||||
});
|
||||
|
||||
|
||||
// Verify results (should have 1 item, missing the one that errored)
|
||||
expect(results).toEqual([2]);
|
||||
|
||||
|
||||
// Verify that onRetry was called twice (for 2 retry attempts)
|
||||
expect(onRetry).toHaveBeenCalledTimes(2);
|
||||
|
||||
// Verify that console.error was called once
|
||||
expect(consoleErrorMock).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Verify that console.error was called (enhanced logging calls it multiple times)
|
||||
expect(consoleErrorMock).toHaveBeenCalled();
|
||||
} finally {
|
||||
// Restore console.error
|
||||
console.error = originalConsoleError;
|
||||
|
||||
@@ -46,11 +46,25 @@ export async function processInParallel<T, R>(
|
||||
const batchResults = await Promise.allSettled(batchPromises);
|
||||
|
||||
// Process results and handle errors
|
||||
for (const result of batchResults) {
|
||||
for (let j = 0; j < batchResults.length; j++) {
|
||||
const result = batchResults[j];
|
||||
if (result.status === 'fulfilled') {
|
||||
results.push(result.value);
|
||||
} else {
|
||||
console.error('Error processing item:', result.reason);
|
||||
const itemIndex = i + j;
|
||||
console.error("=== BATCH ITEM PROCESSING ERROR ===");
|
||||
console.error("Batch index:", Math.floor(i / concurrencyLimit));
|
||||
console.error("Item index in batch:", j);
|
||||
console.error("Global item index:", itemIndex);
|
||||
console.error("Error type:", result.reason?.constructor?.name);
|
||||
console.error("Error message:", result.reason instanceof Error ? result.reason.message : String(result.reason));
|
||||
|
||||
if (result.reason instanceof Error && result.reason.message.includes('JSON')) {
|
||||
console.error("🚨 JSON parsing error in batch processing");
|
||||
console.error("This indicates an API response issue from Gitea");
|
||||
}
|
||||
|
||||
console.error("==================================");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -139,6 +153,21 @@ export async function processWithRetry<T, R>(
|
||||
const delay = retryDelay * Math.pow(2, attempt - 1);
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
} else {
|
||||
// Enhanced error logging for final failure
|
||||
console.error("=== ITEM PROCESSING FAILED (MAX RETRIES EXCEEDED) ===");
|
||||
console.error("Item:", getItemId ? getItemId(item) : 'unknown');
|
||||
console.error("Error type:", lastError.constructor.name);
|
||||
console.error("Error message:", lastError.message);
|
||||
console.error("Attempts made:", maxRetries + 1);
|
||||
|
||||
if (lastError.message.includes('JSON')) {
|
||||
console.error("🚨 JSON-related error detected in item processing");
|
||||
console.error("This suggests an issue with API responses from Gitea");
|
||||
}
|
||||
|
||||
console.error("Stack trace:", lastError.stack);
|
||||
console.error("================================================");
|
||||
|
||||
throw lastError;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { describe, test, expect, mock, beforeEach, afterEach } from "bun:test";
|
||||
import axios from "axios";
|
||||
|
||||
// Mock the POST function
|
||||
const mockPOST = mock(async ({ request }) => {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { APIRoute } from 'astro';
|
||||
import axios from 'axios';
|
||||
import { httpGet, HttpError } from '@/lib/http-client';
|
||||
|
||||
export const POST: APIRoute = async ({ request }) => {
|
||||
try {
|
||||
@@ -25,11 +25,9 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
const baseUrl = url.endsWith('/') ? url.slice(0, -1) : url;
|
||||
|
||||
// Test the connection by fetching the authenticated user
|
||||
const response = await axios.get(`${baseUrl}/api/v1/user`, {
|
||||
headers: {
|
||||
'Authorization': `token ${token}`,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
const response = await httpGet(`${baseUrl}/api/v1/user`, {
|
||||
'Authorization': `token ${token}`,
|
||||
'Accept': 'application/json',
|
||||
});
|
||||
|
||||
const data = response.data;
|
||||
@@ -72,8 +70,8 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
console.error('Gitea connection test failed:', error);
|
||||
|
||||
// Handle specific error types
|
||||
if (axios.isAxiosError(error) && error.response) {
|
||||
if (error.response.status === 401) {
|
||||
if (error instanceof HttpError) {
|
||||
if (error.status === 401) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
@@ -86,7 +84,7 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
},
|
||||
}
|
||||
);
|
||||
} else if (error.response.status === 404) {
|
||||
} else if (error.status === 404) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
@@ -99,25 +97,23 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
},
|
||||
}
|
||||
);
|
||||
} else if (error.status === 0) {
|
||||
// Network error
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
message: 'Could not connect to Gitea server. Please check the URL.',
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle connection errors
|
||||
if (axios.isAxiosError(error) && (error.code === 'ECONNREFUSED' || error.code === 'ENOTFOUND')) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
message: 'Could not connect to Gitea server. Please check the URL.',
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
// Generic error response
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { APIRoute } from "astro";
|
||||
import { db } from "@/lib/db";
|
||||
import { organizations } from "@/lib/db";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { organizations, repositories, configs } from "@/lib/db";
|
||||
import { eq, sql, and, count } from "drizzle-orm";
|
||||
import {
|
||||
membershipRoleEnum,
|
||||
type OrganizationsApiResponse,
|
||||
@@ -25,24 +25,114 @@ export const GET: APIRoute = async ({ request }) => {
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch the user's active configuration to respect filtering settings
|
||||
const [config] = await db
|
||||
.select()
|
||||
.from(configs)
|
||||
.where(and(eq(configs.userId, userId), eq(configs.isActive, true)));
|
||||
|
||||
if (!config) {
|
||||
return jsonResponse({
|
||||
data: {
|
||||
success: false,
|
||||
error: "No active configuration found for this user",
|
||||
},
|
||||
status: 404,
|
||||
});
|
||||
}
|
||||
|
||||
const githubConfig = config.githubConfig as {
|
||||
mirrorStarred: boolean;
|
||||
skipForks: boolean;
|
||||
privateRepositories: boolean;
|
||||
};
|
||||
|
||||
const rawOrgs = await db
|
||||
.select()
|
||||
.from(organizations)
|
||||
.where(eq(organizations.userId, userId))
|
||||
.orderBy(sql`name COLLATE NOCASE`);
|
||||
|
||||
const orgsWithIds: Organization[] = rawOrgs.map((org) => ({
|
||||
...org,
|
||||
status: repoStatusEnum.parse(org.status),
|
||||
membershipRole: membershipRoleEnum.parse(org.membershipRole),
|
||||
lastMirrored: org.lastMirrored ?? undefined,
|
||||
errorMessage: org.errorMessage ?? undefined,
|
||||
}));
|
||||
// Calculate repository breakdowns for each organization
|
||||
const orgsWithBreakdown = await Promise.all(
|
||||
rawOrgs.map(async (org) => {
|
||||
// Build base conditions for this organization (without private/fork filters)
|
||||
const baseConditions = [
|
||||
eq(repositories.userId, userId),
|
||||
eq(repositories.organization, org.name)
|
||||
];
|
||||
|
||||
if (!githubConfig.mirrorStarred) {
|
||||
baseConditions.push(eq(repositories.isStarred, false));
|
||||
}
|
||||
|
||||
// Get total count with all user config filters applied
|
||||
const totalConditions = [...baseConditions];
|
||||
if (githubConfig.skipForks) {
|
||||
totalConditions.push(eq(repositories.isForked, false));
|
||||
}
|
||||
if (!githubConfig.privateRepositories) {
|
||||
totalConditions.push(eq(repositories.isPrivate, false));
|
||||
}
|
||||
|
||||
const [totalCount] = await db
|
||||
.select({ count: count() })
|
||||
.from(repositories)
|
||||
.where(and(...totalConditions));
|
||||
|
||||
// Get public count
|
||||
const publicConditions = [...baseConditions, eq(repositories.isPrivate, false)];
|
||||
if (githubConfig.skipForks) {
|
||||
publicConditions.push(eq(repositories.isForked, false));
|
||||
}
|
||||
|
||||
const [publicCount] = await db
|
||||
.select({ count: count() })
|
||||
.from(repositories)
|
||||
.where(and(...publicConditions));
|
||||
|
||||
// Get private count (only if private repos are enabled in config)
|
||||
const [privateCount] = githubConfig.privateRepositories ? await db
|
||||
.select({ count: count() })
|
||||
.from(repositories)
|
||||
.where(
|
||||
and(
|
||||
...baseConditions,
|
||||
eq(repositories.isPrivate, true),
|
||||
...(githubConfig.skipForks ? [eq(repositories.isForked, false)] : [])
|
||||
)
|
||||
) : [{ count: 0 }];
|
||||
|
||||
// Get fork count (only if forks are enabled in config)
|
||||
const [forkCount] = !githubConfig.skipForks ? await db
|
||||
.select({ count: count() })
|
||||
.from(repositories)
|
||||
.where(
|
||||
and(
|
||||
...baseConditions,
|
||||
eq(repositories.isForked, true),
|
||||
...(!githubConfig.privateRepositories ? [eq(repositories.isPrivate, false)] : [])
|
||||
)
|
||||
) : [{ count: 0 }];
|
||||
|
||||
return {
|
||||
...org,
|
||||
status: repoStatusEnum.parse(org.status),
|
||||
membershipRole: membershipRoleEnum.parse(org.membershipRole),
|
||||
lastMirrored: org.lastMirrored ?? undefined,
|
||||
errorMessage: org.errorMessage ?? undefined,
|
||||
repositoryCount: totalCount.count,
|
||||
publicRepositoryCount: publicCount.count,
|
||||
privateRepositoryCount: privateCount.count,
|
||||
forkRepositoryCount: forkCount.count,
|
||||
};
|
||||
})
|
||||
);
|
||||
|
||||
const resPayload: OrganizationsApiResponse = {
|
||||
success: true,
|
||||
message: "Organizations fetched successfully",
|
||||
organizations: orgsWithIds,
|
||||
organizations: orgsWithBreakdown,
|
||||
};
|
||||
|
||||
return jsonResponse({ data: resPayload, status: 200 });
|
||||
|
||||
@@ -4,7 +4,7 @@ import { db } from "@/lib/db";
|
||||
import { ENV } from "@/lib/config";
|
||||
import { getRecoveryStatus, hasJobsNeedingRecovery } from "@/lib/recovery";
|
||||
import os from "os";
|
||||
import axios from "axios";
|
||||
import { httpGet } from "@/lib/http-client";
|
||||
|
||||
// Track when the server started
|
||||
const serverStartTime = new Date();
|
||||
@@ -197,9 +197,9 @@ async function checkLatestVersion(): Promise<string> {
|
||||
|
||||
try {
|
||||
// Fetch the latest release from GitHub
|
||||
const response = await axios.get(
|
||||
const response = await httpGet(
|
||||
'https://api.github.com/repos/arunavo4/gitea-mirror/releases/latest',
|
||||
{ headers: { 'Accept': 'application/vnd.github.v3+json' } }
|
||||
{ 'Accept': 'application/vnd.github.v3+json' }
|
||||
);
|
||||
|
||||
// Extract version from tag_name (remove 'v' prefix if present)
|
||||
|
||||
@@ -165,11 +165,43 @@ export const POST: APIRoute = async ({ request }) => {
|
||||
headers: { "Content-Type": "application/json" },
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("Error mirroring repositories:", error);
|
||||
// Enhanced error logging for better debugging
|
||||
console.error("=== ERROR MIRRORING REPOSITORIES ===");
|
||||
console.error("Error type:", error?.constructor?.name);
|
||||
console.error("Error message:", error instanceof Error ? error.message : String(error));
|
||||
|
||||
if (error instanceof Error) {
|
||||
console.error("Error stack:", error.stack);
|
||||
}
|
||||
|
||||
// Log additional context
|
||||
console.error("Request details:");
|
||||
console.error("- URL:", request.url);
|
||||
console.error("- Method:", request.method);
|
||||
console.error("- Headers:", Object.fromEntries(request.headers.entries()));
|
||||
|
||||
// If it's a JSON parsing error, provide more context
|
||||
if (error instanceof SyntaxError && error.message.includes('JSON')) {
|
||||
console.error("🚨 JSON PARSING ERROR DETECTED:");
|
||||
console.error("This suggests the response from Gitea API is not valid JSON");
|
||||
console.error("Common causes:");
|
||||
console.error("- Gitea server returned HTML error page instead of JSON");
|
||||
console.error("- Network connection interrupted");
|
||||
console.error("- Gitea server is down or misconfigured");
|
||||
console.error("- Authentication token is invalid");
|
||||
console.error("Check your Gitea server logs and configuration");
|
||||
}
|
||||
|
||||
console.error("=====================================");
|
||||
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error:
|
||||
error instanceof Error ? error.message : "An unknown error occurred",
|
||||
error: error instanceof Error ? error.message : "An unknown error occurred",
|
||||
errorType: error?.constructor?.name || "Unknown",
|
||||
timestamp: new Date().toISOString(),
|
||||
troubleshooting: error instanceof SyntaxError && error.message.includes('JSON')
|
||||
? "JSON parsing error detected. Check Gitea server status and logs. Ensure Gitea is returning valid JSON responses."
|
||||
: "Check application logs for more details"
|
||||
}),
|
||||
{ status: 500, headers: { "Content-Type": "application/json" } }
|
||||
);
|
||||
|
||||
@@ -33,6 +33,9 @@ export interface GitOrg {
|
||||
isIncluded: boolean;
|
||||
status: RepoStatus;
|
||||
repositoryCount: number;
|
||||
publicRepositoryCount?: number;
|
||||
privateRepositoryCount?: number;
|
||||
forkRepositoryCount?: number;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||