Compare commits

..

14 Commits

Author SHA1 Message Date
Arunavo Ray
e24b856416 chore: bump version to 2.12.0
- Fixed SQLite 'no such table: mirror_jobs' error during application startup
- Implemented automatic database table creation during database initialization
- Resolved database schema inconsistencies between development and production environments
- Enhanced database initialization process with automatic table creation and indexing
- Added comprehensive error handling for database table creation
- Integrated database repair functionality into application startup for better reliability
2025-06-02 15:08:10 +05:30
Arunavo Ray
612805f030 feat: add table creation and existence check for database initialization 2025-06-02 15:05:20 +05:30
Arunavo Ray
7705dffee0 chore: bump version to 2.11.2 2025-05-28 20:29:23 +05:30
Arunavo Ray
3dceb34174 feat: replace SiGitea icons with custom logo
- Replace SiGitea icon with custom logo.svg in LoginForm and Header components
- Add custom logo.svg file with theme-aware styling
- Update favicon.svg to use the same custom logo design
- Remove unused SiGitea imports and clean up dependencies
- Logo automatically adapts to light/dark themes via CSS media queries
2025-05-28 20:28:59 +05:30
Arunavo Ray
6b747ba891 chore: bump version to 2.11.1 2025-05-28 19:58:46 +05:30
Arunavo Ray
ddd67faeab fix: resolve repository mirroring status inconsistencies
- Fix 'already exists, skipping migration' logic that left repositories with incorrect 'imported' status
- Update database status to 'mirrored' when repository already exists in Gitea
- Add automatic startup repair to fix existing inconsistencies on container start
- Create diagnostic and repair tools for troubleshooting mirroring issues
- Ensure consistent state between Gitea and application database

Resolves issue where repositories showed successful mirroring logs but remained
in 'imported' status, causing UI confusion and preventing proper status tracking.

Changes:
- src/lib/gitea.ts: Fixed mirrorGithubRepoToGitea() and mirrorGitHubRepoToGiteaOrg()
- docker-entrypoint.sh: Added automatic repository status repair on startup
- scripts/investigate-repo.ts: New diagnostic tool for repository analysis
- scripts/repair-mirrored-repos.ts: New repair tool with startup mode support
- scripts/cleanup-duplicate-repos.ts: New tool for removing duplicate entries

Fixes multiple user reports of misleading 'successfully mirrored' logs
while repositories remained in inconsistent state.
2025-05-28 19:58:15 +05:30
Arunavo Ray
832b57538d chore: bump version to 2.11.0 2025-05-28 14:08:45 +05:30
Arunavo Ray
415bff8e41 feat: enhance Organizations page with live refresh and fix repository breakdown bug
- Add live refresh functionality to Organizations page using the same pattern as Repositories and Activity Log pages
- Fix repository breakdown bug where public/private/fork counts disappeared after toggling mirroring
- Change toggle text from 'Include in mirroring' to 'Enable mirroring' for better clarity
- Automatically refresh organization data after mirroring starts to maintain breakdown visibility
- Clean up unused imports and variables for better code quality
2025-05-28 14:08:07 +05:30
Arunavo Ray
13c3ddea04 Added a small gap to Verison Info 2025-05-28 13:55:50 +05:30
Arunavo Ray
b917b30830 docs: add Docker bind mount vs named volume permission guidance
- Add new section 'Docker Volume Types and Permissions'
- Explain difference between named volumes and bind mounts
- Provide solution for bind mount permission issues (UID 1001)
- Clarify why named volumes are recommended and used in official docker-compose.yml
- Address SQLite permission errors in Docker environments using bind mounts

Addresses issue reported by user using bind mounts in Portainer.
2025-05-28 13:37:07 +05:30
Arunavo Ray
b34ed5595b chore: bump version to 2.10.0 2025-05-28 13:27:04 +05:30
Arunavo Ray
cbc11155ef fix: resolve organizations getting stuck on mirroring status when empty
- Fixed mirrorGitHubOrgToGitea function to properly handle empty organizations
- Organizations with no repositories now transition from 'mirroring' to 'mirrored' status
- Enhanced logging with clearer messages for empty organization processing
- Improved activity log details to distinguish between empty and non-empty orgs
- Added comprehensive test coverage for empty organization scenarios
- Ensures consistent status lifecycle for all organizations regardless of repository count
2025-05-28 13:26:20 +05:30
Arunavo Ray
941f61830f feat: implement comprehensive auto-save for all config forms and remove manual save button
- Add auto-save functionality to all GitHub config form fields (text inputs and checkboxes)
- Add auto-save functionality to all Gitea config form fields (text inputs and select dropdown)
- Extend existing auto-save pattern to cover text inputs with 500ms debounce
- Remove Save Configuration button and related manual save logic
- Update Import GitHub Data button to depend on form validation instead of saved state
- Remove isConfigSaved dependency from all auto-save functions for immediate activation
- Add proper cleanup for all auto-save timeouts on component unmount
- Maintain silent auto-save operation without intrusive notifications

All configuration changes now auto-save seamlessly, providing a better UX while maintaining data consistency and error handling.
2025-05-28 13:17:48 +05:30
Arunavo Ray
5b60cffaae Add fork tags to repository UI and enhance organization cards with repository breakdown
- Add fork tags to repository table and dashboard list components
- Display 'Fork' badge for repositories where isForked is true
- Enhance organization cards to show breakdown of public, private, and fork repositories
- Update organization API to respect user configuration filters (private repos, forks)
- Add visual indicators with colored dots for each repository type
- Ensure consistent filtering between repository and organization APIs
- Fix issue where private repositories weren't showing due to configuration filtering
2025-05-28 12:53:32 +05:30
25 changed files with 1786 additions and 161 deletions

View File

@@ -5,6 +5,18 @@ All notable changes to the Gitea Mirror project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [2.12.0] - 2025-01-27
### Fixed
- Fixed SQLite "no such table: mirror_jobs" error during application startup
- Implemented automatic database table creation during database initialization
- Resolved database schema inconsistencies between development and production environments
### Improved
- Enhanced database initialization process with automatic table creation and indexing
- Added comprehensive error handling for database table creation
- Integrated database repair functionality into application startup for better reliability
## [2.5.3] - 2025-05-22 ## [2.5.3] - 2025-05-22
### Added ### Added

View File

@@ -513,6 +513,36 @@ Try the following steps:
> >
> This setup provides a complete containerized deployment for the Gitea Mirror application. > This setup provides a complete containerized deployment for the Gitea Mirror application.
#### Docker Volume Types and Permissions
> [!IMPORTANT]
> **Named Volumes vs Bind Mounts**: If you encounter SQLite permission errors even when using Docker, check your volume configuration:
**✅ Named Volumes (Recommended):**
```yaml
volumes:
- gitea-mirror-data:/app/data # Docker manages permissions automatically
```
**⚠️ Bind Mounts (Requires Manual Permission Setup):**
```yaml
volumes:
- /host/path/to/data:/app/data # Host filesystem permissions apply
```
**If using bind mounts**, ensure the host directory is owned by UID 1001 (the `gitea-mirror` user):
```bash
# Set correct ownership for bind mount
sudo chown -R 1001:1001 /host/path/to/data
sudo chmod -R 755 /host/path/to/data
```
**Why named volumes work better:**
- Docker automatically handles permissions
- Better portability across different hosts
- No manual permission setup required
- Used by our official docker-compose.yml
#### Database Maintenance #### Database Maintenance

View File

@@ -232,6 +232,28 @@ else
echo "❌ Startup recovery failed with exit code $RECOVERY_EXIT_CODE" echo "❌ Startup recovery failed with exit code $RECOVERY_EXIT_CODE"
fi fi
# Run repository status repair to fix any inconsistent mirroring states
echo "Running repository status repair..."
if [ -f "dist/scripts/repair-mirrored-repos.js" ]; then
echo "Running repository repair using compiled script..."
bun dist/scripts/repair-mirrored-repos.js --startup
REPAIR_EXIT_CODE=$?
elif [ -f "scripts/repair-mirrored-repos.ts" ]; then
echo "Running repository repair using TypeScript script..."
bun scripts/repair-mirrored-repos.ts --startup
REPAIR_EXIT_CODE=$?
else
echo "Warning: Repository repair script not found. Skipping repair."
REPAIR_EXIT_CODE=0
fi
# Log repair result
if [ $REPAIR_EXIT_CODE -eq 0 ]; then
echo "✅ Repository status repair completed successfully"
else
echo "⚠️ Repository status repair completed with warnings (exit code $REPAIR_EXIT_CODE)"
fi
# Function to handle shutdown signals # Function to handle shutdown signals
shutdown_handler() { shutdown_handler() {
echo "🛑 Received shutdown signal, forwarding to application..." echo "🛑 Received shutdown signal, forwarding to application..."

View File

@@ -1,7 +1,7 @@
{ {
"name": "gitea-mirror", "name": "gitea-mirror",
"type": "module", "type": "module",
"version": "2.9.3", "version": "2.12.0",
"engines": { "engines": {
"bun": ">=1.2.9" "bun": ">=1.2.9"
}, },

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 749 B

After

Width:  |  Height:  |  Size: 21 KiB

200
public/logo.svg Normal file

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 21 KiB

View File

@@ -0,0 +1,129 @@
#!/usr/bin/env bun
/**
* Script to find and clean up duplicate repositories in the database
* Keeps the most recent entry and removes older duplicates
*
* Usage: bun scripts/cleanup-duplicate-repos.ts [--dry-run] [--repo-name=<name>]
*/
import { db, repositories, mirrorJobs } from "@/lib/db";
import { eq, and, desc } from "drizzle-orm";
const isDryRun = process.argv.includes("--dry-run");
const specificRepo = process.argv.find(arg => arg.startsWith("--repo-name="))?.split("=")[1];
async function findDuplicateRepositories() {
console.log("🔍 Finding duplicate repositories");
console.log("=" .repeat(40));
if (isDryRun) {
console.log("🔍 DRY RUN MODE - No changes will be made");
console.log("");
}
if (specificRepo) {
console.log(`🎯 Targeting specific repository: ${specificRepo}`);
console.log("");
}
try {
// Find all repositories, grouped by name and fullName
let allRepos = await db.select().from(repositories);
if (specificRepo) {
allRepos = allRepos.filter(repo => repo.name === specificRepo);
}
// Group repositories by name and fullName
const repoGroups = new Map<string, typeof allRepos>();
for (const repo of allRepos) {
const key = `${repo.name}|${repo.fullName}`;
if (!repoGroups.has(key)) {
repoGroups.set(key, []);
}
repoGroups.get(key)!.push(repo);
}
// Find groups with duplicates
const duplicateGroups = Array.from(repoGroups.entries())
.filter(([_, repos]) => repos.length > 1);
if (duplicateGroups.length === 0) {
console.log("✅ No duplicate repositories found");
return;
}
console.log(`📋 Found ${duplicateGroups.length} sets of duplicate repositories:`);
console.log("");
let totalDuplicates = 0;
let totalRemoved = 0;
for (const [key, repos] of duplicateGroups) {
const [name, fullName] = key.split("|");
console.log(`🔄 Processing duplicates for: ${name} (${fullName})`);
console.log(` Found ${repos.length} entries:`);
// Sort by updatedAt descending to keep the most recent
repos.sort((a, b) => new Date(b.updatedAt).getTime() - new Date(a.updatedAt).getTime());
const keepRepo = repos[0];
const removeRepos = repos.slice(1);
console.log(` ✅ Keeping: ID ${keepRepo.id} (Status: ${keepRepo.status}, Updated: ${new Date(keepRepo.updatedAt).toISOString()})`);
for (const repo of removeRepos) {
console.log(` ❌ Removing: ID ${repo.id} (Status: ${repo.status}, Updated: ${new Date(repo.updatedAt).toISOString()})`);
if (!isDryRun) {
try {
// First, delete related mirror jobs
await db
.delete(mirrorJobs)
.where(eq(mirrorJobs.repositoryId, repo.id!));
// Then delete the repository
await db
.delete(repositories)
.where(eq(repositories.id, repo.id!));
console.log(` 🗑️ Deleted repository and related mirror jobs`);
totalRemoved++;
} catch (error) {
console.log(` ❌ Error deleting repository: ${error instanceof Error ? error.message : String(error)}`);
}
} else {
console.log(` 🗑️ Would delete repository and related mirror jobs`);
totalRemoved++;
}
}
totalDuplicates += removeRepos.length;
console.log("");
}
console.log("📊 Cleanup Summary:");
console.log(` Duplicate sets found: ${duplicateGroups.length}`);
console.log(` Total duplicates: ${totalDuplicates}`);
console.log(` ${isDryRun ? 'Would remove' : 'Removed'}: ${totalRemoved}`);
if (isDryRun && totalRemoved > 0) {
console.log("");
console.log("💡 To apply these changes, run the script without --dry-run");
}
} catch (error) {
console.error("❌ Error during cleanup process:", error);
}
}
// Run the cleanup
findDuplicateRepositories().then(() => {
console.log("Cleanup process complete.");
process.exit(0);
}).catch((error) => {
console.error("Fatal error:", error);
process.exit(1);
});

178
scripts/investigate-repo.ts Normal file
View File

@@ -0,0 +1,178 @@
#!/usr/bin/env bun
/**
* Script to investigate a specific repository's mirroring status
* Usage: bun scripts/investigate-repo.ts [repository-name]
*/
import { db, repositories, mirrorJobs, configs } from "@/lib/db";
import { eq, desc, and } from "drizzle-orm";
const repoName = process.argv[2] || "EruditionPaper";
async function investigateRepository() {
console.log(`🔍 Investigating repository: ${repoName}`);
console.log("=" .repeat(50));
try {
// Find the repository in the database
const repos = await db
.select()
.from(repositories)
.where(eq(repositories.name, repoName));
if (repos.length === 0) {
console.log(`❌ Repository "${repoName}" not found in database`);
return;
}
const repo = repos[0];
console.log(`✅ Found repository: ${repo.name}`);
console.log(` ID: ${repo.id}`);
console.log(` Full Name: ${repo.fullName}`);
console.log(` Owner: ${repo.owner}`);
console.log(` Organization: ${repo.organization || "None"}`);
console.log(` Status: ${repo.status}`);
console.log(` Is Private: ${repo.isPrivate}`);
console.log(` Is Forked: ${repo.isForked}`);
console.log(` Mirrored Location: ${repo.mirroredLocation || "Not set"}`);
console.log(` Last Mirrored: ${repo.lastMirrored ? new Date(repo.lastMirrored).toISOString() : "Never"}`);
console.log(` Error Message: ${repo.errorMessage || "None"}`);
console.log(` Created At: ${new Date(repo.createdAt).toISOString()}`);
console.log(` Updated At: ${new Date(repo.updatedAt).toISOString()}`);
console.log("\n📋 Recent Mirror Jobs:");
console.log("-".repeat(30));
// Find recent mirror jobs for this repository
const jobs = await db
.select()
.from(mirrorJobs)
.where(eq(mirrorJobs.repositoryId, repo.id))
.orderBy(desc(mirrorJobs.timestamp))
.limit(10);
if (jobs.length === 0) {
console.log(" No mirror jobs found for this repository");
} else {
jobs.forEach((job, index) => {
console.log(` ${index + 1}. ${new Date(job.timestamp).toISOString()}`);
console.log(` Status: ${job.status}`);
console.log(` Message: ${job.message}`);
if (job.details) {
console.log(` Details: ${job.details}`);
}
console.log("");
});
}
// Get user configuration
console.log("⚙️ User Configuration:");
console.log("-".repeat(20));
const config = await db
.select()
.from(configs)
.where(eq(configs.id, repo.configId))
.limit(1);
if (config.length > 0) {
const userConfig = config[0];
console.log(` User ID: ${userConfig.userId}`);
console.log(` GitHub Username: ${userConfig.githubConfig?.username || "Not set"}`);
console.log(` Gitea URL: ${userConfig.giteaConfig?.url || "Not set"}`);
console.log(` Gitea Username: ${userConfig.giteaConfig?.username || "Not set"}`);
console.log(` Preserve Org Structure: ${userConfig.githubConfig?.preserveOrgStructure || false}`);
console.log(` Mirror Issues: ${userConfig.githubConfig?.mirrorIssues || false}`);
}
// Check for any active jobs
console.log("\n🔄 Active Jobs:");
console.log("-".repeat(15));
const activeJobs = await db
.select()
.from(mirrorJobs)
.where(
and(
eq(mirrorJobs.repositoryId, repo.id),
eq(mirrorJobs.inProgress, true)
)
);
if (activeJobs.length === 0) {
console.log(" No active jobs found");
} else {
activeJobs.forEach((job, index) => {
console.log(` ${index + 1}. Job ID: ${job.id}`);
console.log(` Type: ${job.jobType || "mirror"}`);
console.log(` Batch ID: ${job.batchId || "None"}`);
console.log(` Started: ${job.startedAt ? new Date(job.startedAt).toISOString() : "Unknown"}`);
console.log(` Last Checkpoint: ${job.lastCheckpoint ? new Date(job.lastCheckpoint).toISOString() : "None"}`);
console.log(` Progress: ${job.completedItems || 0}/${job.totalItems || 0}`);
console.log("");
});
}
// Check if repository exists in Gitea
if (config.length > 0) {
const userConfig = config[0];
console.log("\n🔗 Gitea Repository Check:");
console.log("-".repeat(25));
try {
const giteaUrl = userConfig.giteaConfig?.url;
const giteaToken = userConfig.giteaConfig?.token;
const giteaUsername = userConfig.giteaConfig?.username;
if (giteaUrl && giteaToken && giteaUsername) {
const checkUrl = `${giteaUrl}/api/v1/repos/${giteaUsername}/${repo.name}`;
console.log(` Checking: ${checkUrl}`);
const response = await fetch(checkUrl, {
headers: {
Authorization: `token ${giteaToken}`,
},
});
console.log(` Response Status: ${response.status} ${response.statusText}`);
if (response.ok) {
const repoData = await response.json();
console.log(` ✅ Repository exists in Gitea`);
console.log(` Name: ${repoData.name}`);
console.log(` Full Name: ${repoData.full_name}`);
console.log(` Private: ${repoData.private}`);
console.log(` Mirror: ${repoData.mirror}`);
console.log(` Clone URL: ${repoData.clone_url}`);
console.log(` Created: ${new Date(repoData.created_at).toISOString()}`);
console.log(` Updated: ${new Date(repoData.updated_at).toISOString()}`);
if (repoData.mirror_updated) {
console.log(` Mirror Updated: ${new Date(repoData.mirror_updated).toISOString()}`);
}
} else {
console.log(` ❌ Repository not found in Gitea`);
const errorText = await response.text();
console.log(` Error: ${errorText}`);
}
} else {
console.log(" ⚠️ Missing Gitea configuration");
}
} catch (error) {
console.log(` ❌ Error checking Gitea: ${error instanceof Error ? error.message : String(error)}`);
}
}
} catch (error) {
console.error("❌ Error investigating repository:", error);
}
}
// Run the investigation
investigateRepository().then(() => {
console.log("Investigation complete.");
process.exit(0);
}).catch((error) => {
console.error("Fatal error:", error);
process.exit(1);
});

View File

@@ -0,0 +1,277 @@
#!/usr/bin/env bun
/**
* Script to repair repositories that exist in Gitea but have incorrect status in the database
* This fixes the issue where repositories show as "imported" but are actually mirrored in Gitea
*
* Usage: bun scripts/repair-mirrored-repos.ts [--dry-run] [--repo-name=<name>]
*/
import { db, repositories, configs } from "@/lib/db";
import { eq, and, or } from "drizzle-orm";
import { createMirrorJob } from "@/lib/helpers";
import { repoStatusEnum } from "@/types/Repository";
const isDryRun = process.argv.includes("--dry-run");
const specificRepo = process.argv.find(arg => arg.startsWith("--repo-name="))?.split("=")[1];
const isStartupMode = process.argv.includes("--startup");
async function checkRepoInGitea(config: any, owner: string, repoName: string): Promise<boolean> {
try {
if (!config.giteaConfig?.url || !config.giteaConfig?.token) {
return false;
}
const response = await fetch(
`${config.giteaConfig.url}/api/v1/repos/${owner}/${repoName}`,
{
headers: {
Authorization: `token ${config.giteaConfig.token}`,
},
}
);
return response.ok;
} catch (error) {
console.error(`Error checking repo ${owner}/${repoName} in Gitea:`, error);
return false;
}
}
async function getRepoDetailsFromGitea(config: any, owner: string, repoName: string): Promise<any> {
try {
if (!config.giteaConfig?.url || !config.giteaConfig?.token) {
return null;
}
const response = await fetch(
`${config.giteaConfig.url}/api/v1/repos/${owner}/${repoName}`,
{
headers: {
Authorization: `token ${config.giteaConfig.token}`,
},
}
);
if (response.ok) {
return await response.json();
}
return null;
} catch (error) {
console.error(`Error getting repo details for ${owner}/${repoName}:`, error);
return null;
}
}
async function repairMirroredRepositories() {
if (!isStartupMode) {
console.log("🔧 Repairing mirrored repositories database status");
console.log("=" .repeat(60));
if (isDryRun) {
console.log("🔍 DRY RUN MODE - No changes will be made");
console.log("");
}
if (specificRepo) {
console.log(`🎯 Targeting specific repository: ${specificRepo}`);
console.log("");
}
}
try {
// Find repositories that might need repair
let query = db
.select()
.from(repositories)
.where(
or(
eq(repositories.status, "imported"),
eq(repositories.status, "failed")
)
);
if (specificRepo) {
query = query.where(eq(repositories.name, specificRepo));
}
const repos = await query;
if (repos.length === 0) {
if (!isStartupMode) {
console.log("✅ No repositories found that need repair");
}
return;
}
if (!isStartupMode) {
console.log(`📋 Found ${repos.length} repositories to check:`);
console.log("");
}
let repairedCount = 0;
let skippedCount = 0;
let errorCount = 0;
for (const repo of repos) {
if (!isStartupMode) {
console.log(`🔍 Checking repository: ${repo.name}`);
console.log(` Current status: ${repo.status}`);
console.log(` Mirrored location: ${repo.mirroredLocation || "Not set"}`);
}
try {
// Get user configuration
const config = await db
.select()
.from(configs)
.where(eq(configs.id, repo.configId))
.limit(1);
if (config.length === 0) {
if (!isStartupMode) {
console.log(` ❌ No configuration found for repository`);
}
errorCount++;
continue;
}
const userConfig = config[0];
const giteaUsername = userConfig.giteaConfig?.username;
if (!giteaUsername) {
if (!isStartupMode) {
console.log(` ❌ No Gitea username in configuration`);
}
errorCount++;
continue;
}
// Check if repository exists in Gitea (try both user and organization)
let existsInGitea = false;
let actualOwner = giteaUsername;
let giteaRepoDetails = null;
// First check user location
existsInGitea = await checkRepoInGitea(userConfig, giteaUsername, repo.name);
if (existsInGitea) {
giteaRepoDetails = await getRepoDetailsFromGitea(userConfig, giteaUsername, repo.name);
}
// If not found in user location and repo has organization, check organization
if (!existsInGitea && repo.organization) {
existsInGitea = await checkRepoInGitea(userConfig, repo.organization, repo.name);
if (existsInGitea) {
actualOwner = repo.organization;
giteaRepoDetails = await getRepoDetailsFromGitea(userConfig, repo.organization, repo.name);
}
}
if (!existsInGitea) {
if (!isStartupMode) {
console.log(` ⏭️ Repository not found in Gitea - skipping`);
}
skippedCount++;
continue;
}
if (!isStartupMode) {
console.log(` ✅ Repository found in Gitea at: ${actualOwner}/${repo.name}`);
if (giteaRepoDetails) {
console.log(` 📊 Gitea details:`);
console.log(` Mirror: ${giteaRepoDetails.mirror}`);
console.log(` Created: ${new Date(giteaRepoDetails.created_at).toISOString()}`);
console.log(` Updated: ${new Date(giteaRepoDetails.updated_at).toISOString()}`);
if (giteaRepoDetails.mirror_updated) {
console.log(` Mirror Updated: ${new Date(giteaRepoDetails.mirror_updated).toISOString()}`);
}
}
} else if (repairedCount === 0) {
// In startup mode, only log the first repair to indicate activity
console.log(`Repairing repository status inconsistencies...`);
}
if (!isDryRun) {
// Update repository status in database
const mirrorUpdated = giteaRepoDetails?.mirror_updated
? new Date(giteaRepoDetails.mirror_updated)
: new Date();
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: mirrorUpdated,
errorMessage: null,
mirroredLocation: `${actualOwner}/${repo.name}`,
})
.where(eq(repositories.id, repo.id!));
// Create a mirror job log entry
await createMirrorJob({
userId: userConfig.userId || "",
repositoryId: repo.id,
repositoryName: repo.name,
message: `Repository status repaired - found existing mirror in Gitea`,
details: `Repository ${repo.name} was found to already exist in Gitea at ${actualOwner}/${repo.name} and database status was updated from ${repo.status} to mirrored.`,
status: "mirrored",
});
if (!isStartupMode) {
console.log(` 🔧 Repaired: Updated status to 'mirrored'`);
}
} else {
if (!isStartupMode) {
console.log(` 🔧 Would repair: Update status from '${repo.status}' to 'mirrored'`);
}
}
repairedCount++;
} catch (error) {
if (!isStartupMode) {
console.log(` ❌ Error processing repository: ${error instanceof Error ? error.message : String(error)}`);
}
errorCount++;
}
if (!isStartupMode) {
console.log("");
}
}
if (isStartupMode) {
// In startup mode, only log if there were repairs or errors
if (repairedCount > 0) {
console.log(`Repaired ${repairedCount} repository status inconsistencies`);
}
if (errorCount > 0) {
console.log(`Warning: ${errorCount} repositories had errors during repair`);
}
} else {
console.log("📊 Repair Summary:");
console.log(` Repaired: ${repairedCount}`);
console.log(` Skipped: ${skippedCount}`);
console.log(` Errors: ${errorCount}`);
if (isDryRun && repairedCount > 0) {
console.log("");
console.log("💡 To apply these changes, run the script without --dry-run");
}
}
} catch (error) {
console.error("❌ Error during repair process:", error);
}
}
// Run the repair
repairMirroredRepositories().then(() => {
console.log("Repair process complete.");
process.exit(0);
}).catch((error) => {
console.error("Fatal error:", error);
process.exit(1);
});

View File

@@ -1,13 +1,13 @@
'use client'; 'use client';
import * as React from 'react'; import * as React from 'react';
import { useState, useEffect } from 'react'; import { useState } from 'react';
import { Button } from '@/components/ui/button'; import { Button } from '@/components/ui/button';
import { Card, CardContent, CardDescription, CardFooter, CardHeader, CardTitle } from '@/components/ui/card'; import { Card, CardContent, CardDescription, CardFooter, CardHeader, CardTitle } from '@/components/ui/card';
import { SiGitea } from 'react-icons/si';
import { toast, Toaster } from 'sonner'; import { toast, Toaster } from 'sonner';
import { showErrorToast } from '@/lib/utils'; import { showErrorToast } from '@/lib/utils';
import { FlipHorizontal } from 'lucide-react';
export function LoginForm() { export function LoginForm() {
const [isLoading, setIsLoading] = useState(false); const [isLoading, setIsLoading] = useState(false);
@@ -60,7 +60,7 @@ export function LoginForm() {
<Card className="w-full max-w-md"> <Card className="w-full max-w-md">
<CardHeader className="text-center"> <CardHeader className="text-center">
<div className="flex justify-center mb-4"> <div className="flex justify-center mb-4">
<SiGitea className="h-10 w-10" /> <img src="/logo.svg" alt="Gitea Mirror" className="h-10 w-10" />
</div> </div>
<CardTitle className="text-2xl">Gitea Mirror</CardTitle> <CardTitle className="text-2xl">Gitea Mirror</CardTitle>
<CardDescription> <CardDescription>

View File

@@ -56,14 +56,18 @@ export function ConfigTabs() {
retentionDays: 604800, // 7 days in seconds retentionDays: 604800, // 7 days in seconds
}, },
}); });
const { user, refreshUser } = useAuth(); const { user } = useAuth();
const [isLoading, setIsLoading] = useState(true); const [isLoading, setIsLoading] = useState(true);
const [isSyncing, setIsSyncing] = useState<boolean>(false); const [isSyncing, setIsSyncing] = useState<boolean>(false);
const [isConfigSaved, setIsConfigSaved] = useState<boolean>(false);
const [isAutoSavingSchedule, setIsAutoSavingSchedule] = useState<boolean>(false); const [isAutoSavingSchedule, setIsAutoSavingSchedule] = useState<boolean>(false);
const [isAutoSavingCleanup, setIsAutoSavingCleanup] = useState<boolean>(false); const [isAutoSavingCleanup, setIsAutoSavingCleanup] = useState<boolean>(false);
const [isAutoSavingGitHub, setIsAutoSavingGitHub] = useState<boolean>(false);
const [isAutoSavingGitea, setIsAutoSavingGitea] = useState<boolean>(false);
const autoSaveScheduleTimeoutRef = useRef<NodeJS.Timeout | null>(null); const autoSaveScheduleTimeoutRef = useRef<NodeJS.Timeout | null>(null);
const autoSaveCleanupTimeoutRef = useRef<NodeJS.Timeout | null>(null); const autoSaveCleanupTimeoutRef = useRef<NodeJS.Timeout | null>(null);
const autoSaveGitHubTimeoutRef = useRef<NodeJS.Timeout | null>(null);
const autoSaveGiteaTimeoutRef = useRef<NodeJS.Timeout | null>(null);
const isConfigFormValid = (): boolean => { const isConfigFormValid = (): boolean => {
const { githubConfig, giteaConfig } = config; const { githubConfig, giteaConfig } = config;
@@ -109,44 +113,9 @@ export function ConfigTabs() {
} }
}; };
const handleSaveConfig = async () => {
if (!user?.id) return;
const reqPayload: SaveConfigApiRequest = {
userId: user.id,
githubConfig: config.githubConfig,
giteaConfig: config.giteaConfig,
scheduleConfig: config.scheduleConfig,
cleanupConfig: config.cleanupConfig,
};
try {
const response = await fetch('/api/config', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reqPayload),
});
const result: SaveConfigApiResponse = await response.json();
if (result.success) {
await refreshUser();
setIsConfigSaved(true);
// Invalidate config cache so other components get fresh data
invalidateConfigCache();
toast.success(
'Configuration saved successfully! Now import your GitHub data to begin.',
);
} else {
showErrorToast(
`Failed to save configuration: ${result.message || 'Unknown error'}`,
toast
);
}
} catch (error) {
showErrorToast(error, toast);
}
};
// Auto-save function specifically for schedule config changes // Auto-save function specifically for schedule config changes
const autoSaveScheduleConfig = useCallback(async (scheduleConfig: ScheduleConfig) => { const autoSaveScheduleConfig = useCallback(async (scheduleConfig: ScheduleConfig) => {
if (!user?.id || !isConfigSaved) return; // Only auto-save if config was previously saved if (!user?.id) return;
// Clear any existing timeout // Clear any existing timeout
if (autoSaveScheduleTimeoutRef.current) { if (autoSaveScheduleTimeoutRef.current) {
@@ -206,11 +175,11 @@ export function ConfigTabs() {
setIsAutoSavingSchedule(false); setIsAutoSavingSchedule(false);
} }
}, 500); // 500ms debounce }, 500); // 500ms debounce
}, [user?.id, isConfigSaved, config.githubConfig, config.giteaConfig, config.cleanupConfig]); }, [user?.id, config.githubConfig, config.giteaConfig, config.cleanupConfig]);
// Auto-save function specifically for cleanup config changes // Auto-save function specifically for cleanup config changes
const autoSaveCleanupConfig = useCallback(async (cleanupConfig: DatabaseCleanupConfig) => { const autoSaveCleanupConfig = useCallback(async (cleanupConfig: DatabaseCleanupConfig) => {
if (!user?.id || !isConfigSaved) return; // Only auto-save if config was previously saved if (!user?.id) return;
// Clear any existing timeout // Clear any existing timeout
if (autoSaveCleanupTimeoutRef.current) { if (autoSaveCleanupTimeoutRef.current) {
@@ -269,7 +238,101 @@ export function ConfigTabs() {
setIsAutoSavingCleanup(false); setIsAutoSavingCleanup(false);
} }
}, 500); // 500ms debounce }, 500); // 500ms debounce
}, [user?.id, isConfigSaved, config.githubConfig, config.giteaConfig, config.scheduleConfig]); }, [user?.id, config.githubConfig, config.giteaConfig, config.scheduleConfig]);
// Auto-save function specifically for GitHub config changes
const autoSaveGitHubConfig = useCallback(async (githubConfig: GitHubConfig) => {
if (!user?.id) return;
// Clear any existing timeout
if (autoSaveGitHubTimeoutRef.current) {
clearTimeout(autoSaveGitHubTimeoutRef.current);
}
// Debounce the auto-save to prevent excessive API calls
autoSaveGitHubTimeoutRef.current = setTimeout(async () => {
setIsAutoSavingGitHub(true);
const reqPayload: SaveConfigApiRequest = {
userId: user.id!,
githubConfig: githubConfig,
giteaConfig: config.giteaConfig,
scheduleConfig: config.scheduleConfig,
cleanupConfig: config.cleanupConfig,
};
try {
const response = await fetch('/api/config', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reqPayload),
});
const result: SaveConfigApiResponse = await response.json();
if (result.success) {
// Silent success - no toast for auto-save
// Invalidate config cache so other components get fresh data
invalidateConfigCache();
} else {
showErrorToast(
`Auto-save failed: ${result.message || 'Unknown error'}`,
toast
);
}
} catch (error) {
showErrorToast(error, toast);
} finally {
setIsAutoSavingGitHub(false);
}
}, 500); // 500ms debounce
}, [user?.id, config.giteaConfig, config.scheduleConfig, config.cleanupConfig]);
// Auto-save function specifically for Gitea config changes
const autoSaveGiteaConfig = useCallback(async (giteaConfig: GiteaConfig) => {
if (!user?.id) return;
// Clear any existing timeout
if (autoSaveGiteaTimeoutRef.current) {
clearTimeout(autoSaveGiteaTimeoutRef.current);
}
// Debounce the auto-save to prevent excessive API calls
autoSaveGiteaTimeoutRef.current = setTimeout(async () => {
setIsAutoSavingGitea(true);
const reqPayload: SaveConfigApiRequest = {
userId: user.id!,
githubConfig: config.githubConfig,
giteaConfig: giteaConfig,
scheduleConfig: config.scheduleConfig,
cleanupConfig: config.cleanupConfig,
};
try {
const response = await fetch('/api/config', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(reqPayload),
});
const result: SaveConfigApiResponse = await response.json();
if (result.success) {
// Silent success - no toast for auto-save
// Invalidate config cache so other components get fresh data
invalidateConfigCache();
} else {
showErrorToast(
`Auto-save failed: ${result.message || 'Unknown error'}`,
toast
);
}
} catch (error) {
showErrorToast(error, toast);
} finally {
setIsAutoSavingGitea(false);
}
}, 500); // 500ms debounce
}, [user?.id, config.githubConfig, config.scheduleConfig, config.cleanupConfig]);
// Cleanup timeouts on unmount // Cleanup timeouts on unmount
useEffect(() => { useEffect(() => {
@@ -280,6 +343,12 @@ export function ConfigTabs() {
if (autoSaveCleanupTimeoutRef.current) { if (autoSaveCleanupTimeoutRef.current) {
clearTimeout(autoSaveCleanupTimeoutRef.current); clearTimeout(autoSaveCleanupTimeoutRef.current);
} }
if (autoSaveGitHubTimeoutRef.current) {
clearTimeout(autoSaveGitHubTimeoutRef.current);
}
if (autoSaveGiteaTimeoutRef.current) {
clearTimeout(autoSaveGiteaTimeoutRef.current);
}
}; };
}, []); }, []);
@@ -304,7 +373,7 @@ export function ConfigTabs() {
cleanupConfig: cleanupConfig:
response.cleanupConfig || config.cleanupConfig, response.cleanupConfig || config.cleanupConfig,
}); });
if (response.id) setIsConfigSaved(true);
} }
} catch (error) { } catch (error) {
console.warn( console.warn(
@@ -401,10 +470,10 @@ export function ConfigTabs() {
<div className="flex gap-x-4"> <div className="flex gap-x-4">
<Button <Button
onClick={handleImportGitHubData} onClick={handleImportGitHubData}
disabled={isSyncing || !isConfigSaved} disabled={isSyncing || !isConfigFormValid()}
title={ title={
!isConfigSaved !isConfigFormValid()
? 'Save configuration first' ? 'Please fill all required GitHub and Gitea fields'
: isSyncing : isSyncing
? 'Import in progress' ? 'Import in progress'
: 'Import GitHub Data' : 'Import GitHub Data'
@@ -422,17 +491,6 @@ export function ConfigTabs() {
</> </>
)} )}
</Button> </Button>
<Button
onClick={handleSaveConfig}
disabled={!isConfigFormValid()}
title={
!isConfigFormValid()
? 'Please fill all required fields'
: 'Save Configuration'
}
>
Save Configuration
</Button>
</div> </div>
</div> </div>
@@ -450,6 +508,8 @@ export function ConfigTabs() {
: update, : update,
})) }))
} }
onAutoSave={autoSaveGitHubConfig}
isAutoSaving={isAutoSavingGitHub}
/> />
<GiteaConfigForm <GiteaConfigForm
config={config.giteaConfig} config={config.giteaConfig}
@@ -462,6 +522,8 @@ export function ConfigTabs() {
: update, : update,
})) }))
} }
onAutoSave={autoSaveGiteaConfig}
isAutoSaving={isAutoSavingGitea}
/> />
</div> </div>
<div className="flex gap-x-4"> <div className="flex gap-x-4">

View File

@@ -20,9 +20,11 @@ import { Tooltip, TooltipContent, TooltipTrigger } from "../ui/tooltip";
interface GitHubConfigFormProps { interface GitHubConfigFormProps {
config: GitHubConfig; config: GitHubConfig;
setConfig: React.Dispatch<React.SetStateAction<GitHubConfig>>; setConfig: React.Dispatch<React.SetStateAction<GitHubConfig>>;
onAutoSave?: (githubConfig: GitHubConfig) => Promise<void>;
isAutoSaving?: boolean;
} }
export function GitHubConfigForm({ config, setConfig }: GitHubConfigFormProps) { export function GitHubConfigForm({ config, setConfig, onAutoSave, isAutoSaving }: GitHubConfigFormProps) {
const [isLoading, setIsLoading] = useState(false); const [isLoading, setIsLoading] = useState(false);
const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => { const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => {
@@ -43,10 +45,17 @@ export function GitHubConfigForm({ config, setConfig }: GitHubConfigFormProps) {
); );
} }
setConfig({ const newConfig = {
...config, ...config,
[name]: type === "checkbox" ? checked : value, [name]: type === "checkbox" ? checked : value,
}); };
setConfig(newConfig);
// Auto-save for all field changes
if (onAutoSave) {
onAutoSave(newConfig);
}
}; };
const testConnection = async () => { const testConnection = async () => {

View File

@@ -21,19 +21,27 @@ import { toast } from "sonner";
interface GiteaConfigFormProps { interface GiteaConfigFormProps {
config: GiteaConfig; config: GiteaConfig;
setConfig: React.Dispatch<React.SetStateAction<GiteaConfig>>; setConfig: React.Dispatch<React.SetStateAction<GiteaConfig>>;
onAutoSave?: (giteaConfig: GiteaConfig) => Promise<void>;
isAutoSaving?: boolean;
} }
export function GiteaConfigForm({ config, setConfig }: GiteaConfigFormProps) { export function GiteaConfigForm({ config, setConfig, onAutoSave, isAutoSaving }: GiteaConfigFormProps) {
const [isLoading, setIsLoading] = useState(false); const [isLoading, setIsLoading] = useState(false);
const handleChange = ( const handleChange = (
e: React.ChangeEvent<HTMLInputElement | HTMLSelectElement> e: React.ChangeEvent<HTMLInputElement | HTMLSelectElement>
) => { ) => {
const { name, value } = e.target; const { name, value } = e.target;
setConfig({ const newConfig = {
...config, ...config,
[name]: value, [name]: value,
}); };
setConfig(newConfig);
// Auto-save for all field changes
if (onAutoSave) {
onAutoSave(newConfig);
}
}; };
const testConnection = async () => { const testConnection = async () => {

View File

@@ -81,6 +81,11 @@ export function RepositoryList({ repositories }: RepositoryListProps) {
Private Private
</span> </span>
)} )}
{repo.isForked && (
<span className="rounded-full bg-muted px-2 py-0.5 text-xs">
Fork
</span>
)}
</div> </div>
<div className="flex items-center gap-2 mt-1"> <div className="flex items-center gap-2 mt-1">
<span className="text-xs text-muted-foreground"> <span className="text-xs text-muted-foreground">

View File

@@ -1,6 +1,6 @@
import { useAuth } from "@/hooks/useAuth"; import { useAuth } from "@/hooks/useAuth";
import { Button } from "@/components/ui/button"; import { Button } from "@/components/ui/button";
import { SiGitea } from "react-icons/si";
import { ModeToggle } from "@/components/theme/ModeToggle"; import { ModeToggle } from "@/components/theme/ModeToggle";
import { Avatar, AvatarFallback, AvatarImage } from "../ui/avatar"; import { Avatar, AvatarFallback, AvatarImage } from "../ui/avatar";
import { toast } from "sonner"; import { toast } from "sonner";
@@ -64,7 +64,7 @@ export function Header({ currentPage, onNavigate }: HeaderProps) {
}} }}
className="flex items-center gap-2 py-1 hover:opacity-80 transition-opacity" className="flex items-center gap-2 py-1 hover:opacity-80 transition-opacity"
> >
<SiGitea className="h-6 w-6" /> <img src="/logo.svg" alt="Gitea Mirror" className="h-6 w-6" />
<span className="text-xl font-bold">Gitea Mirror</span> <span className="text-xl font-bold">Gitea Mirror</span>
</button> </button>

View File

@@ -37,7 +37,7 @@ export function VersionInfo() {
return ( return (
<div className="text-xs text-muted-foreground text-center pt-2 pb-3 border-t border-border mt-2"> <div className="text-xs text-muted-foreground text-center pt-2 pb-3 border-t border-border mt-2">
{versionInfo.updateAvailable ? ( {versionInfo.updateAvailable ? (
<div className="flex flex-col"> <div className="flex flex-col gap-1">
<span>v{versionInfo.current}</span> <span>v{versionInfo.current}</span>
<span className="text-primary">v{versionInfo.latest} available</span> <span className="text-primary">v{versionInfo.latest} available</span>
</div> </div>

View File

@@ -1,6 +1,6 @@
import { useCallback, useEffect, useState } from "react"; import { useCallback, useEffect, useState } from "react";
import { Button } from "@/components/ui/button"; import { Button } from "@/components/ui/button";
import { Search, RefreshCw, FlipHorizontal, Plus } from "lucide-react"; import { Search, RefreshCw, FlipHorizontal } from "lucide-react";
import type { MirrorJob, Organization } from "@/lib/db/schema"; import type { MirrorJob, Organization } from "@/lib/db/schema";
import { OrganizationList } from "./OrganizationsList"; import { OrganizationList } from "./OrganizationsList";
import AddOrganizationDialog from "./AddOrganizationDialog"; import AddOrganizationDialog from "./AddOrganizationDialog";
@@ -26,6 +26,7 @@ import { useFilterParams } from "@/hooks/useFilterParams";
import { toast } from "sonner"; import { toast } from "sonner";
import { useConfigStatus } from "@/hooks/useConfigStatus"; import { useConfigStatus } from "@/hooks/useConfigStatus";
import { useNavigation } from "@/components/layout/MainLayout"; import { useNavigation } from "@/components/layout/MainLayout";
import { useLiveRefresh } from "@/hooks/useLiveRefresh";
export function Organization() { export function Organization() {
const [organizations, setOrganizations] = useState<Organization[]>([]); const [organizations, setOrganizations] = useState<Organization[]>([]);
@@ -34,6 +35,7 @@ export function Organization() {
const { user } = useAuth(); const { user } = useAuth();
const { isGitHubConfigured } = useConfigStatus(); const { isGitHubConfigured } = useConfigStatus();
const { navigationKey } = useNavigation(); const { navigationKey } = useNavigation();
const { registerRefreshCallback } = useLiveRefresh();
const { filter, setFilter } = useFilterParams({ const { filter, setFilter } = useFilterParams({
searchTerm: "", searchTerm: "",
membershipRole: "", membershipRole: "",
@@ -62,19 +64,23 @@ export function Organization() {
onMessage: handleNewMessage, onMessage: handleNewMessage,
}); });
const fetchOrganizations = useCallback(async () => { const fetchOrganizations = useCallback(async (isLiveRefresh = false) => {
if (!user?.id) { if (!user?.id) {
return false; return false;
} }
// Don't fetch organizations if GitHub is not configured // Don't fetch organizations if GitHub is not configured
if (!isGitHubConfigured) { if (!isGitHubConfigured) {
if (!isLiveRefresh) {
setIsLoading(false); setIsLoading(false);
}
return false; return false;
} }
try { try {
if (!isLiveRefresh) {
setIsLoading(true); setIsLoading(true);
}
const response = await apiRequest<OrganizationsApiResponse>( const response = await apiRequest<OrganizationsApiResponse>(
`/github/organizations?userId=${user.id}`, `/github/organizations?userId=${user.id}`,
@@ -87,27 +93,47 @@ export function Organization() {
setOrganizations(response.organizations); setOrganizations(response.organizations);
return true; return true;
} else { } else {
if (!isLiveRefresh) {
toast.error(response.error || "Error fetching organizations"); toast.error(response.error || "Error fetching organizations");
}
return false; return false;
} }
} catch (error) { } catch (error) {
if (!isLiveRefresh) {
toast.error( toast.error(
error instanceof Error ? error.message : "Error fetching organizations" error instanceof Error ? error.message : "Error fetching organizations"
); );
}
return false; return false;
} finally { } finally {
if (!isLiveRefresh) {
setIsLoading(false); setIsLoading(false);
} }
}
}, [user?.id, isGitHubConfigured]); // Only depend on user.id, not entire user object }, [user?.id, isGitHubConfigured]); // Only depend on user.id, not entire user object
useEffect(() => { useEffect(() => {
// Reset loading state when component becomes active // Reset loading state when component becomes active
setIsLoading(true); setIsLoading(true);
fetchOrganizations(); fetchOrganizations(false); // Manual refresh, not live
}, [fetchOrganizations, navigationKey]); // Include navigationKey to trigger on navigation }, [fetchOrganizations, navigationKey]); // Include navigationKey to trigger on navigation
// Register with global live refresh system
useEffect(() => {
// Only register for live refresh if GitHub is configured
if (!isGitHubConfigured) {
return;
}
const unregister = registerRefreshCallback(() => {
fetchOrganizations(true); // Live refresh
});
return unregister;
}, [registerRefreshCallback, fetchOrganizations, isGitHubConfigured]);
const handleRefresh = async () => { const handleRefresh = async () => {
const success = await fetchOrganizations(); const success = await fetchOrganizations(false);
if (success) { if (success) {
toast.success("Organizations refreshed successfully."); toast.success("Organizations refreshed successfully.");
} }
@@ -140,6 +166,12 @@ export function Organization() {
return updated ? updated : org; return updated ? updated : org;
}) })
); );
// Refresh organization data to get updated repository breakdown
// Use a small delay to allow the backend to process the mirroring request
setTimeout(() => {
fetchOrganizations(true);
}, 1000);
} else { } else {
toast.error(response.error || "Error starting mirror job"); toast.error(response.error || "Error starting mirror job");
} }
@@ -258,12 +290,7 @@ export function Organization() {
} }
}; };
// Get unique organization names for combobox (since Organization has no owner field)
const ownerOptions = Array.from(
new Set(
organizations.map((org) => org.name).filter((v): v is string => !!v)
)
).sort();
return ( return (
<div className="flex flex-col gap-y-8"> <div className="flex flex-col gap-y-8">

View File

@@ -118,10 +118,38 @@ export function OrganizationList({
</span> </span>
</div> </div>
<p className="text-sm text-muted-foreground mb-4"> <div className="text-sm text-muted-foreground mb-4">
<div className="flex items-center justify-between">
<span className="font-medium">
{org.repositoryCount}{" "} {org.repositoryCount}{" "}
{org.repositoryCount === 1 ? "repository" : "repositories"} {org.repositoryCount === 1 ? "repository" : "repositories"}
</p> </span>
</div>
{(org.publicRepositoryCount !== undefined ||
org.privateRepositoryCount !== undefined ||
org.forkRepositoryCount !== undefined) && (
<div className="flex gap-4 mt-2 text-xs">
{org.publicRepositoryCount !== undefined && (
<span className="flex items-center gap-1">
<div className="h-2 w-2 rounded-full bg-green-500" />
{org.publicRepositoryCount} public
</span>
)}
{org.privateRepositoryCount !== undefined && org.privateRepositoryCount > 0 && (
<span className="flex items-center gap-1">
<div className="h-2 w-2 rounded-full bg-orange-500" />
{org.privateRepositoryCount} private
</span>
)}
{org.forkRepositoryCount !== undefined && org.forkRepositoryCount > 0 && (
<span className="flex items-center gap-1">
<div className="h-2 w-2 rounded-full bg-blue-500" />
{org.forkRepositoryCount} fork{org.forkRepositoryCount !== 1 ? 's' : ''}
</span>
)}
</div>
)}
</div>
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div className="flex items-center"> <div className="flex items-center">
@@ -144,7 +172,7 @@ export function OrganizationList({
htmlFor={`include-${org.id}`} htmlFor={`include-${org.id}`}
className="ml-2 text-sm select-none peer-disabled:cursor-not-allowed peer-disabled:opacity-70" className="ml-2 text-sm select-none peer-disabled:cursor-not-allowed peer-disabled:opacity-70"
> >
Include in mirroring Enable mirroring
</label> </label>
{isLoading && ( {isLoading && (

View File

@@ -249,6 +249,11 @@ export default function RepositoryTable({
Private Private
</span> </span>
)} )}
{repo.isForked && (
<span className="ml-2 rounded-full bg-muted px-2 py-0.5 text-xs">
Fork
</span>
)}
</div> </div>
{/* Owner */} {/* Owner */}

View File

@@ -25,11 +25,222 @@ let sqlite: Database;
try { try {
sqlite = new Database(dbPath); sqlite = new Database(dbPath);
console.log("Successfully connected to SQLite database using Bun's native driver"); console.log("Successfully connected to SQLite database using Bun's native driver");
// Ensure all required tables exist
ensureTablesExist(sqlite);
} catch (error) { } catch (error) {
console.error("Error opening database:", error); console.error("Error opening database:", error);
throw error; throw error;
} }
/**
* Ensure all required tables exist in the database
*/
function ensureTablesExist(db: Database) {
const requiredTables = [
"users",
"configs",
"repositories",
"organizations",
"mirror_jobs",
"events",
];
for (const table of requiredTables) {
try {
// Check if table exists
const result = db.query(`SELECT name FROM sqlite_master WHERE type='table' AND name='${table}'`).get();
if (!result) {
console.warn(`⚠️ Table '${table}' is missing. Creating it now...`);
createTable(db, table);
console.log(`✅ Table '${table}' created successfully`);
}
} catch (error) {
console.error(`❌ Error checking/creating table '${table}':`, error);
throw error;
}
}
}
/**
* Create a specific table with its schema
*/
function createTable(db: Database, tableName: string) {
switch (tableName) {
case "users":
db.exec(`
CREATE TABLE users (
id TEXT PRIMARY KEY,
username TEXT NOT NULL,
password TEXT NOT NULL,
email TEXT NOT NULL,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)
`);
break;
case "configs":
db.exec(`
CREATE TABLE configs (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
name TEXT NOT NULL,
is_active INTEGER NOT NULL DEFAULT 1,
github_config TEXT NOT NULL,
gitea_config TEXT NOT NULL,
include TEXT NOT NULL DEFAULT '["*"]',
exclude TEXT NOT NULL DEFAULT '[]',
schedule_config TEXT NOT NULL,
cleanup_config TEXT NOT NULL,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id)
)
`);
break;
case "repositories":
db.exec(`
CREATE TABLE repositories (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
config_id TEXT NOT NULL,
name TEXT NOT NULL,
full_name TEXT NOT NULL,
url TEXT NOT NULL,
clone_url TEXT NOT NULL,
owner TEXT NOT NULL,
organization TEXT,
mirrored_location TEXT DEFAULT '',
is_private INTEGER NOT NULL DEFAULT 0,
is_fork INTEGER NOT NULL DEFAULT 0,
forked_from TEXT,
has_issues INTEGER NOT NULL DEFAULT 0,
is_starred INTEGER NOT NULL DEFAULT 0,
language TEXT,
description TEXT,
default_branch TEXT NOT NULL,
visibility TEXT NOT NULL DEFAULT 'public',
status TEXT NOT NULL DEFAULT 'imported',
last_mirrored INTEGER,
error_message TEXT,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id),
FOREIGN KEY (config_id) REFERENCES configs(id)
)
`);
// Create indexes for repositories
db.exec(`
CREATE INDEX IF NOT EXISTS idx_repositories_user_id ON repositories(user_id);
CREATE INDEX IF NOT EXISTS idx_repositories_config_id ON repositories(config_id);
CREATE INDEX IF NOT EXISTS idx_repositories_status ON repositories(status);
CREATE INDEX IF NOT EXISTS idx_repositories_owner ON repositories(owner);
CREATE INDEX IF NOT EXISTS idx_repositories_organization ON repositories(organization);
CREATE INDEX IF NOT EXISTS idx_repositories_is_fork ON repositories(is_fork);
CREATE INDEX IF NOT EXISTS idx_repositories_is_starred ON repositories(is_starred);
`);
break;
case "organizations":
db.exec(`
CREATE TABLE organizations (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
config_id TEXT NOT NULL,
name TEXT NOT NULL,
avatar_url TEXT NOT NULL,
membership_role TEXT NOT NULL DEFAULT 'member',
is_included INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT 'imported',
last_mirrored INTEGER,
error_message TEXT,
repository_count INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
updated_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id),
FOREIGN KEY (config_id) REFERENCES configs(id)
)
`);
// Create indexes for organizations
db.exec(`
CREATE INDEX IF NOT EXISTS idx_organizations_user_id ON organizations(user_id);
CREATE INDEX IF NOT EXISTS idx_organizations_config_id ON organizations(config_id);
CREATE INDEX IF NOT EXISTS idx_organizations_status ON organizations(status);
CREATE INDEX IF NOT EXISTS idx_organizations_is_included ON organizations(is_included);
`);
break;
case "mirror_jobs":
db.exec(`
CREATE TABLE mirror_jobs (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
repository_id TEXT,
repository_name TEXT,
organization_id TEXT,
organization_name TEXT,
details TEXT,
status TEXT NOT NULL DEFAULT 'imported',
message TEXT NOT NULL,
timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- New fields for job resilience
job_type TEXT NOT NULL DEFAULT 'mirror',
batch_id TEXT,
total_items INTEGER,
completed_items INTEGER DEFAULT 0,
item_ids TEXT, -- JSON array as text
completed_item_ids TEXT DEFAULT '[]', -- JSON array as text
in_progress INTEGER NOT NULL DEFAULT 0, -- Boolean as integer
started_at TIMESTAMP,
completed_at TIMESTAMP,
last_checkpoint TIMESTAMP,
FOREIGN KEY (user_id) REFERENCES users(id)
)
`);
// Create indexes for mirror_jobs
db.exec(`
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_user_id ON mirror_jobs(user_id);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_batch_id ON mirror_jobs(batch_id);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_in_progress ON mirror_jobs(in_progress);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_job_type ON mirror_jobs(job_type);
CREATE INDEX IF NOT EXISTS idx_mirror_jobs_timestamp ON mirror_jobs(timestamp);
`);
break;
case "events":
db.exec(`
CREATE TABLE events (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
channel TEXT NOT NULL,
payload TEXT NOT NULL,
read INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL DEFAULT (strftime('%s','now')),
FOREIGN KEY (user_id) REFERENCES users(id)
)
`);
// Create indexes for events
db.exec(`
CREATE INDEX IF NOT EXISTS idx_events_user_channel ON events(user_id, channel);
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at);
CREATE INDEX IF NOT EXISTS idx_events_read ON events(read);
`);
break;
default:
throw new Error(`Unknown table: ${tableName}`);
}
}
// Create drizzle instance with the SQLite client // Create drizzle instance with the SQLite client
export const db = drizzle({ client: sqlite }); export const db = drizzle({ client: sqlite });

View File

@@ -152,6 +152,9 @@ export const organizationSchema = z.object({
errorMessage: z.string().optional(), errorMessage: z.string().optional(),
repositoryCount: z.number().default(0), repositoryCount: z.number().default(0),
publicRepositoryCount: z.number().optional(),
privateRepositoryCount: z.number().optional(),
forkRepositoryCount: z.number().optional(),
createdAt: z.date().default(() => new Date()), createdAt: z.date().default(() => new Date()),
updatedAt: z.date().default(() => new Date()), updatedAt: z.date().default(() => new Date()),

View File

@@ -204,4 +204,90 @@ describe("Gitea Repository Mirroring", () => {
global.fetch = originalFetch; global.fetch = originalFetch;
} }
}); });
test("mirrorGitHubOrgToGitea handles empty organizations correctly", async () => {
// Mock the createMirrorJob function
const mockCreateMirrorJob = mock(() => Promise.resolve("job-id"));
// Mock the getOrCreateGiteaOrg function
const mockGetOrCreateGiteaOrg = mock(() => Promise.resolve("gitea-org-id"));
// Create a test version of the function with mocked dependencies
const testMirrorGitHubOrgToGitea = async ({
organization,
config,
}: {
organization: any;
config: any;
}) => {
// Simulate the function logic for empty organization
console.log(`Mirroring organization ${organization.name}`);
// Mock: get or create Gitea org
await mockGetOrCreateGiteaOrg();
// Mock: query the db with the org name and get the repos
const orgRepos: any[] = []; // Empty array to simulate no repositories
if (orgRepos.length === 0) {
console.log(`No repositories found for organization ${organization.name} - marking as successfully mirrored`);
} else {
console.log(`Mirroring ${orgRepos.length} repositories for organization ${organization.name}`);
// Repository processing would happen here
}
console.log(`Organization ${organization.name} mirrored successfully`);
// Mock: Append log for "mirrored" status
await mockCreateMirrorJob({
userId: config.userId,
organizationId: organization.id,
organizationName: organization.name,
message: `Successfully mirrored organization: ${organization.name}`,
details: orgRepos.length === 0
? `Organization ${organization.name} was processed successfully (no repositories found).`
: `Organization ${organization.name} was mirrored to Gitea with ${orgRepos.length} repositories.`,
status: "mirrored",
});
};
// Create mock organization
const organization = {
id: "org-id",
name: "empty-org",
status: "imported"
};
// Create mock config
const config = {
id: "config-id",
userId: "user-id",
githubConfig: {
token: "github-token"
},
giteaConfig: {
url: "https://gitea.example.com",
token: "gitea-token"
}
};
// Call the test function
await testMirrorGitHubOrgToGitea({
organization,
config
});
// Verify that the mirror job was created with the correct details for empty org
expect(mockCreateMirrorJob).toHaveBeenCalledWith({
userId: "user-id",
organizationId: "org-id",
organizationName: "empty-org",
message: "Successfully mirrored organization: empty-org",
details: "Organization empty-org was processed successfully (no repositories found).",
status: "mirrored",
});
// Verify that getOrCreateGiteaOrg was called
expect(mockGetOrCreateGiteaOrg).toHaveBeenCalled();
});
}); });

View File

@@ -137,8 +137,32 @@ export const mirrorGithubRepoToGitea = async ({
if (isExisting) { if (isExisting) {
console.log( console.log(
`Repository ${repository.name} already exists in Gitea. Skipping migration.` `Repository ${repository.name} already exists in Gitea. Updating database status.`
); );
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${config.giteaConfig.username}/${repository.name}`,
})
.where(eq(repositories.id, repository.id!));
// Append log for "mirrored" status
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea`,
details: `Repository ${repository.name} was found to already exist in Gitea and database status was updated.`,
status: "mirrored",
});
console.log(`Repository ${repository.name} database status updated to mirrored`);
return; return;
} }
@@ -420,8 +444,32 @@ export async function mirrorGitHubRepoToGiteaOrg({
if (isExisting) { if (isExisting) {
console.log( console.log(
`Repository ${repository.name} already exists in Gitea. Skipping migration.` `Repository ${repository.name} already exists in Gitea organization ${orgName}. Updating database status.`
); );
// Update database to reflect that the repository is already mirrored
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("mirrored"),
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${orgName}/${repository.name}`,
})
.where(eq(repositories.id, repository.id!));
// Create a mirror job log entry
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Repository ${repository.name} already exists in Gitea organization ${orgName}`,
details: `Repository ${repository.name} was found to already exist in Gitea organization ${orgName} and database status was updated.`,
status: "mirrored",
});
console.log(`Repository ${repository.name} database status updated to mirrored in organization ${orgName}`);
return; return;
} }
@@ -629,10 +677,8 @@ export async function mirrorGitHubOrgToGitea({
.where(eq(repositories.organization, organization.name)); .where(eq(repositories.organization, organization.name));
if (orgRepos.length === 0) { if (orgRepos.length === 0) {
console.log(`No repositories found for organization ${organization.name}`); console.log(`No repositories found for organization ${organization.name} - marking as successfully mirrored`);
return; } else {
}
console.log(`Mirroring ${orgRepos.length} repositories for organization ${organization.name}`); console.log(`Mirroring ${orgRepos.length} repositories for organization ${organization.name}`);
// Import the processWithRetry function // Import the processWithRetry function
@@ -683,6 +729,7 @@ export async function mirrorGitHubOrgToGitea({
} }
} }
); );
}
console.log(`Organization ${organization.name} mirrored successfully`); console.log(`Organization ${organization.name} mirrored successfully`);
@@ -703,7 +750,9 @@ export async function mirrorGitHubOrgToGitea({
organizationId: organization.id, organizationId: organization.id,
organizationName: organization.name, organizationName: organization.name,
message: `Successfully mirrored organization: ${organization.name}`, message: `Successfully mirrored organization: ${organization.name}`,
details: `Organization ${organization.name} was mirrored to Gitea.`, details: orgRepos.length === 0
? `Organization ${organization.name} was processed successfully (no repositories found).`
: `Organization ${organization.name} was mirrored to Gitea with ${orgRepos.length} repositories.`,
status: repoStatusEnum.parse("mirrored"), status: repoStatusEnum.parse("mirrored"),
}); });
} catch (error) { } catch (error) {

View File

@@ -1,7 +1,7 @@
import type { APIRoute } from "astro"; import type { APIRoute } from "astro";
import { db } from "@/lib/db"; import { db } from "@/lib/db";
import { organizations } from "@/lib/db"; import { organizations, repositories, configs } from "@/lib/db";
import { eq, sql } from "drizzle-orm"; import { eq, sql, and, count } from "drizzle-orm";
import { import {
membershipRoleEnum, membershipRoleEnum,
type OrganizationsApiResponse, type OrganizationsApiResponse,
@@ -25,24 +25,114 @@ export const GET: APIRoute = async ({ request }) => {
} }
try { try {
// Fetch the user's active configuration to respect filtering settings
const [config] = await db
.select()
.from(configs)
.where(and(eq(configs.userId, userId), eq(configs.isActive, true)));
if (!config) {
return jsonResponse({
data: {
success: false,
error: "No active configuration found for this user",
},
status: 404,
});
}
const githubConfig = config.githubConfig as {
mirrorStarred: boolean;
skipForks: boolean;
privateRepositories: boolean;
};
const rawOrgs = await db const rawOrgs = await db
.select() .select()
.from(organizations) .from(organizations)
.where(eq(organizations.userId, userId)) .where(eq(organizations.userId, userId))
.orderBy(sql`name COLLATE NOCASE`); .orderBy(sql`name COLLATE NOCASE`);
const orgsWithIds: Organization[] = rawOrgs.map((org) => ({ // Calculate repository breakdowns for each organization
const orgsWithBreakdown = await Promise.all(
rawOrgs.map(async (org) => {
// Build base conditions for this organization (without private/fork filters)
const baseConditions = [
eq(repositories.userId, userId),
eq(repositories.organization, org.name)
];
if (!githubConfig.mirrorStarred) {
baseConditions.push(eq(repositories.isStarred, false));
}
// Get total count with all user config filters applied
const totalConditions = [...baseConditions];
if (githubConfig.skipForks) {
totalConditions.push(eq(repositories.isForked, false));
}
if (!githubConfig.privateRepositories) {
totalConditions.push(eq(repositories.isPrivate, false));
}
const [totalCount] = await db
.select({ count: count() })
.from(repositories)
.where(and(...totalConditions));
// Get public count
const publicConditions = [...baseConditions, eq(repositories.isPrivate, false)];
if (githubConfig.skipForks) {
publicConditions.push(eq(repositories.isForked, false));
}
const [publicCount] = await db
.select({ count: count() })
.from(repositories)
.where(and(...publicConditions));
// Get private count (only if private repos are enabled in config)
const [privateCount] = githubConfig.privateRepositories ? await db
.select({ count: count() })
.from(repositories)
.where(
and(
...baseConditions,
eq(repositories.isPrivate, true),
...(githubConfig.skipForks ? [eq(repositories.isForked, false)] : [])
)
) : [{ count: 0 }];
// Get fork count (only if forks are enabled in config)
const [forkCount] = !githubConfig.skipForks ? await db
.select({ count: count() })
.from(repositories)
.where(
and(
...baseConditions,
eq(repositories.isForked, true),
...(!githubConfig.privateRepositories ? [eq(repositories.isPrivate, false)] : [])
)
) : [{ count: 0 }];
return {
...org, ...org,
status: repoStatusEnum.parse(org.status), status: repoStatusEnum.parse(org.status),
membershipRole: membershipRoleEnum.parse(org.membershipRole), membershipRole: membershipRoleEnum.parse(org.membershipRole),
lastMirrored: org.lastMirrored ?? undefined, lastMirrored: org.lastMirrored ?? undefined,
errorMessage: org.errorMessage ?? undefined, errorMessage: org.errorMessage ?? undefined,
})); repositoryCount: totalCount.count,
publicRepositoryCount: publicCount.count,
privateRepositoryCount: privateCount.count,
forkRepositoryCount: forkCount.count,
};
})
);
const resPayload: OrganizationsApiResponse = { const resPayload: OrganizationsApiResponse = {
success: true, success: true,
message: "Organizations fetched successfully", message: "Organizations fetched successfully",
organizations: orgsWithIds, organizations: orgsWithBreakdown,
}; };
return jsonResponse({ data: resPayload, status: 200 }); return jsonResponse({ data: resPayload, status: 200 });

View File

@@ -33,6 +33,9 @@ export interface GitOrg {
isIncluded: boolean; isIncluded: boolean;
status: RepoStatus; status: RepoStatus;
repositoryCount: number; repositoryCount: number;
publicRepositoryCount?: number;
privateRepositoryCount?: number;
forkRepositoryCount?: number;
createdAt: Date; createdAt: Date;
updatedAt: Date; updatedAt: Date;
} }