Compare commits

..

11 Commits

Author SHA1 Message Date
Arunavo Ray
c87513b648 chore: bump version to 3.14.2 2026-03-27 13:55:56 +05:30
ARUNAVO RAY
4f3cbc866e fix private github mirror auth (#255) 2026-03-27 13:49:36 +05:30
ARUNAVO RAY
60548f2062 fix sync target resolution for mirrored repos (#249) 2026-03-27 12:33:59 +05:30
dependabot[bot]
74dab43e89 build(deps): bump picomatch (#251)
Bumps the npm_and_yarn group with 1 update in the /www directory: [picomatch](https://github.com/micromatch/picomatch).


Updates `picomatch` from 2.3.1 to 2.3.2
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/2.3.1...2.3.2)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 2.3.2
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-27 09:39:17 +05:30
dependabot[bot]
01a8025140 build(deps): bump smol-toml (#250)
Bumps the npm_and_yarn group with 1 update in the /www directory: [smol-toml](https://github.com/squirrelchat/smol-toml).


Updates `smol-toml` from 1.6.0 to 1.6.1
- [Release notes](https://github.com/squirrelchat/smol-toml/releases)
- [Commits](https://github.com/squirrelchat/smol-toml/compare/v1.6.0...v1.6.1)

---
updated-dependencies:
- dependency-name: smol-toml
  dependency-version: 1.6.1
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-26 22:35:57 +05:30
Arunavo Ray
8346748f5a fix: move --accept-flake-config before -- in bun2nix step
The flag was being passed to bun2nix instead of nix, causing
"unexpected argument" error.
2026-03-24 08:22:04 +05:30
Arunavo Ray
38002019ea fix: regenerate bun.nix in CI to prevent stale dependency errors
The Nix build has been failing since v3.9.6 because bun.nix fell out
of sync with bun.lock. During the sandboxed build bun install cannot
fetch missing packages, causing ConnectionRefused errors.

- Add bun2nix regeneration step before nix build in CI
- Trigger workflow on bun.lock and package.json changes
- Update flake.nix version from 3.9.6 to 3.14.1
2026-03-24 08:20:26 +05:30
Arunavo Ray
32eb27c8a6 chore: bump version to 3.14.1 2026-03-24 07:35:36 +05:30
dependabot[bot]
d33b4ff64f build(deps): bump h3 (#244)
Bumps the npm_and_yarn group with 1 update in the /www directory: [h3](https://github.com/h3js/h3).


Updates `h3` from 1.15.8 to 1.15.9
- [Release notes](https://github.com/h3js/h3/releases)
- [Changelog](https://github.com/h3js/h3/blob/v1.15.9/CHANGELOG.md)
- [Commits](https://github.com/h3js/h3/compare/v1.15.8...v1.15.9)

---
updated-dependencies:
- dependency-name: h3
  dependency-version: 1.15.9
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 07:34:40 +05:30
ARUNAVO RAY
6f2e0cbca0 Add GitHub starred-list filtering with searchable selector (#247)
* feat: add starred list filtering and selector UI

* docs: add starred lists UI screenshot

* lib: improve starred list name matching
2026-03-24 07:33:46 +05:30
dependabot[bot]
95e6eb7602 build(deps): bump h3 (#242)
Bumps the npm_and_yarn group with 1 update in the /www directory: [h3](https://github.com/h3js/h3).


Updates `h3` from 1.15.5 to 1.15.8
- [Release notes](https://github.com/h3js/h3/releases)
- [Changelog](https://github.com/h3js/h3/blob/main/CHANGELOG.md)
- [Commits](https://github.com/h3js/h3/compare/v1.15.5...v1.15.8)

---
updated-dependencies:
- dependency-name: h3
  dependency-version: 1.15.8
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-19 08:24:09 +05:30
25 changed files with 1286 additions and 73 deletions

View File

@@ -63,6 +63,7 @@ DOCKER_TAG=latest
# INCLUDE_ARCHIVED=false
# SKIP_FORKS=false
# MIRROR_STARRED=false
# MIRROR_STARRED_LISTS=homelab,dottools # Optional: comma-separated star list names; empty = all starred repos
# STARRED_REPOS_ORG=starred # Organization name for starred repos
# STARRED_REPOS_MODE=dedicated-org # dedicated-org | preserve-owner

View File

@@ -9,6 +9,8 @@ on:
- 'flake.nix'
- 'flake.lock'
- 'bun.nix'
- 'bun.lock'
- 'package.json'
- '.github/workflows/nix-build.yml'
pull_request:
branches: [main]
@@ -16,6 +18,8 @@ on:
- 'flake.nix'
- 'flake.lock'
- 'bun.nix'
- 'bun.lock'
- 'package.json'
- '.github/workflows/nix-build.yml'
permissions:
@@ -39,6 +43,9 @@ jobs:
- name: Setup Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@main
- name: Regenerate bun.nix from bun.lock
run: nix run --accept-flake-config github:nix-community/bun2nix -- -o bun.nix
- name: Check flake
run: nix flake check --accept-flake-config

View File

@@ -61,6 +61,7 @@ Settings for connecting to and configuring GitHub repository sources.
| `INCLUDE_ARCHIVED` | Include archived repositories | `false` | `true`, `false` |
| `SKIP_FORKS` | Skip forked repositories | `false` | `true`, `false` |
| `MIRROR_STARRED` | Mirror starred repositories | `false` | `true`, `false` |
| `MIRROR_STARRED_LISTS` | Optional comma-separated GitHub Star List names to mirror (only used when `MIRROR_STARRED=true`) | - | Comma-separated list names (empty = all starred repos) |
| `STARRED_REPOS_ORG` | Organization name for starred repos | `starred` | Any string |
| `STARRED_REPOS_MODE` | How starred repos are mirrored | `dedicated-org` | `dedicated-org`, `preserve-owner` |

Binary file not shown.

After

Width:  |  Height:  |  Size: 144 KiB

View File

@@ -31,7 +31,7 @@
# Build the application
gitea-mirror = pkgs.stdenv.mkDerivation {
pname = "gitea-mirror";
version = "3.9.6";
version = "3.14.1";
src = ./.;

View File

@@ -1,7 +1,7 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.14.0",
"version": "3.14.2",
"engines": {
"bun": ">=1.2.9"
},

View File

@@ -42,6 +42,7 @@ export function ConfigTabs() {
token: '',
privateRepositories: false,
mirrorStarred: false,
starredLists: [],
},
giteaConfig: {
url: '',

View File

@@ -4,6 +4,7 @@ import { Label } from "@/components/ui/label";
import { Separator } from "@/components/ui/separator";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import {
Tooltip,
TooltipContent,
@@ -17,6 +18,7 @@ import {
} from "@/components/ui/popover";
import {
Info,
Check,
GitBranch,
Star,
Lock,
@@ -31,7 +33,9 @@ import {
ChevronDown,
Funnel,
HardDrive,
FileCode2
FileCode2,
Plus,
X
} from "lucide-react";
import type { GitHubConfig, MirrorOptions, AdvancedOptions, DuplicateNameStrategy } from "@/types/config";
import {
@@ -41,7 +45,16 @@ import {
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import {
Command,
CommandEmpty,
CommandGroup,
CommandInput,
CommandItem,
CommandList,
} from "@/components/ui/command";
import { cn } from "@/lib/utils";
import { githubApi } from "@/lib/api";
interface GitHubMirrorSettingsProps {
githubConfig: GitHubConfig;
@@ -60,8 +73,42 @@ export function GitHubMirrorSettings({
onMirrorOptionsChange,
onAdvancedOptionsChange,
}: GitHubMirrorSettingsProps) {
const handleGitHubChange = (field: keyof GitHubConfig, value: boolean | string) => {
const [starListsOpen, setStarListsOpen] = React.useState(false);
const [starListSearch, setStarListSearch] = React.useState("");
const [customStarListName, setCustomStarListName] = React.useState("");
const [availableStarLists, setAvailableStarLists] = React.useState<string[]>([]);
const [loadingStarLists, setLoadingStarLists] = React.useState(false);
const [loadedStarLists, setLoadedStarLists] = React.useState(false);
const [attemptedStarListLoad, setAttemptedStarListLoad] = React.useState(false);
const normalizeStarListNames = React.useCallback((lists: string[] | undefined): string[] => {
if (!Array.isArray(lists)) return [];
const seen = new Set<string>();
const normalized: string[] = [];
for (const list of lists) {
const trimmed = list.trim();
if (!trimmed) continue;
const key = trimmed.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
normalized.push(trimmed);
}
return normalized;
}, []);
const selectedStarLists = React.useMemo(
() => normalizeStarListNames(githubConfig.starredLists),
[githubConfig.starredLists, normalizeStarListNames],
);
const allKnownStarLists = React.useMemo(
() => normalizeStarListNames([...availableStarLists, ...selectedStarLists]),
[availableStarLists, selectedStarLists, normalizeStarListNames],
);
const handleGitHubChange = (field: keyof GitHubConfig, value: boolean | string | string[]) => {
onGitHubConfigChange({ ...githubConfig, [field]: value });
};
@@ -83,6 +130,59 @@ export function GitHubMirrorSettings({
onAdvancedOptionsChange({ ...advancedOptions, [field]: value });
};
const setSelectedStarLists = React.useCallback((lists: string[]) => {
onGitHubConfigChange({
...githubConfig,
starredLists: normalizeStarListNames(lists),
});
}, [githubConfig, normalizeStarListNames, onGitHubConfigChange]);
const loadStarLists = React.useCallback(async () => {
if (
loadingStarLists ||
loadedStarLists ||
attemptedStarListLoad ||
!githubConfig.mirrorStarred
) return;
setAttemptedStarListLoad(true);
setLoadingStarLists(true);
try {
const response = await githubApi.getStarredLists();
setAvailableStarLists(normalizeStarListNames(response.lists));
setLoadedStarLists(true);
} catch {
// Keep UX usable with manual custom input even if list fetch fails.
// Allow retry on next popover open.
setLoadedStarLists(false);
} finally {
setLoadingStarLists(false);
}
}, [
attemptedStarListLoad,
githubConfig.mirrorStarred,
loadedStarLists,
loadingStarLists,
normalizeStarListNames,
]);
React.useEffect(() => {
if (!starListsOpen || !githubConfig.mirrorStarred) return;
void loadStarLists();
}, [starListsOpen, githubConfig.mirrorStarred, loadStarLists]);
React.useEffect(() => {
if (!githubConfig.mirrorStarred) {
setStarListsOpen(false);
}
}, [githubConfig.mirrorStarred]);
React.useEffect(() => {
if (!starListsOpen) {
setAttemptedStarListLoad(false);
}
}, [starListsOpen]);
// When metadata is disabled, all components should be disabled
const isMetadataEnabled = mirrorOptions.mirrorMetadata;
@@ -98,6 +198,17 @@ export function GitHubMirrorSettings({
const starredContentCount = Object.entries(starredRepoContent).filter(([key, value]) => key !== 'code' && value).length;
const totalStarredOptions = 4; // releases, issues, PRs, wiki
const normalizedStarListSearch = starListSearch.trim();
const canAddSearchAsStarList = normalizedStarListSearch.length > 0
&& !allKnownStarLists.some((list) => list.toLowerCase() === normalizedStarListSearch.toLowerCase());
const addCustomStarList = () => {
const trimmed = customStarListName.trim();
if (!trimmed) return;
setSelectedStarLists([...selectedStarLists, trimmed]);
setCustomStarListName("");
};
return (
<div className="space-y-6">
{/* Repository Selection Section */}
@@ -312,6 +423,143 @@ export function GitHubMirrorSettings({
</div>
)}
{/* Star list selection */}
{githubConfig.mirrorStarred && (
<div className="mt-4 space-y-2">
<Label className="text-xs font-medium text-muted-foreground">
Star Lists (optional)
</Label>
<Popover open={starListsOpen} onOpenChange={setStarListsOpen}>
<PopoverTrigger asChild>
<Button
variant="outline"
role="combobox"
aria-expanded={starListsOpen}
className="w-full justify-between h-9 text-xs font-normal"
>
<span className="truncate text-left">
{selectedStarLists.length === 0
? "All starred repositories"
: `${selectedStarLists.length} list${selectedStarLists.length === 1 ? "" : "s"} selected`}
</span>
<ChevronDown className="ml-2 h-3 w-3 opacity-50 shrink-0" />
</Button>
</PopoverTrigger>
<PopoverContent className="w-[360px] p-0" align="start">
<Command>
<CommandInput
value={starListSearch}
onValueChange={setStarListSearch}
placeholder="Search GitHub star lists..."
/>
<CommandList>
<CommandEmpty>
{loadingStarLists ? "Loading star lists..." : "No matching lists"}
</CommandEmpty>
<CommandGroup>
{allKnownStarLists.map((list) => {
const isSelected = selectedStarLists.some(
(selected) => selected.toLowerCase() === list.toLowerCase(),
);
return (
<CommandItem
key={list}
value={list}
onSelect={() => {
if (isSelected) {
setSelectedStarLists(
selectedStarLists.filter(
(selected) => selected.toLowerCase() !== list.toLowerCase(),
),
);
} else {
setSelectedStarLists([...selectedStarLists, list]);
}
}}
>
<Check
className={cn(
"mr-2 h-4 w-4",
isSelected ? "opacity-100" : "opacity-0",
)}
/>
<span className="truncate">{list}</span>
</CommandItem>
);
})}
</CommandGroup>
</CommandList>
</Command>
{canAddSearchAsStarList && (
<div className="border-t p-2">
<Button
variant="ghost"
size="sm"
className="w-full justify-start text-xs"
onClick={() => {
setSelectedStarLists([...selectedStarLists, normalizedStarListSearch]);
setStarListSearch("");
}}
>
<Plus className="mr-2 h-3.5 w-3.5" />
Add "{normalizedStarListSearch}"
</Button>
</div>
)}
</PopoverContent>
</Popover>
<p className="text-xs text-muted-foreground">
Leave empty to mirror all starred repositories. Select one or more lists to limit syncing.
</p>
{selectedStarLists.length > 0 && (
<div className="flex flex-wrap gap-1.5">
{selectedStarLists.map((list) => (
<Badge key={list} variant="secondary" className="gap-1">
<span>{list}</span>
<button
type="button"
onClick={() =>
setSelectedStarLists(
selectedStarLists.filter(
(selected) => selected.toLowerCase() !== list.toLowerCase(),
),
)
}
className="rounded-sm hover:text-foreground/80"
aria-label={`Remove ${list} list`}
>
<X className="h-3 w-3" />
</button>
</Badge>
))}
</div>
)}
<div className="flex items-center gap-2">
<Input
value={customStarListName}
onChange={(event) => setCustomStarListName(event.target.value)}
placeholder="Add custom list name"
className="h-8 text-xs"
/>
<Button
type="button"
variant="outline"
size="sm"
className="h-8"
onClick={addCustomStarList}
disabled={!customStarListName.trim()}
>
Add
</Button>
</div>
</div>
)}
{/* Duplicate name handling for starred repos */}
{githubConfig.mirrorStarred && (
<div className="mt-4 space-y-2">

View File

@@ -78,6 +78,10 @@ export const githubApi = {
method: "POST",
body: JSON.stringify({ token }),
}),
getStarredLists: () =>
apiRequest<{ success: boolean; lists: string[] }>("/github/starred-lists", {
method: "GET",
}),
};
// Gitea API

View File

@@ -26,6 +26,7 @@ export const githubConfigSchema = z.object({
includeOrganizations: z.array(z.string()).default([]),
starredReposOrg: z.string().optional(),
starredReposMode: z.enum(["dedicated-org", "preserve-owner"]).default("dedicated-org"),
starredLists: z.array(z.string()).default([]),
mirrorStrategy: z.enum(["preserve", "single-org", "flat-user", "mixed"]).default("preserve"),
defaultOrg: z.string().optional(),
starredCodeOnly: z.boolean().default(false),

View File

@@ -25,6 +25,7 @@ interface EnvConfig {
autoMirrorStarred?: boolean;
starredReposOrg?: string;
starredReposMode?: 'dedicated-org' | 'preserve-owner';
starredLists?: string[];
mirrorStrategy?: 'preserve' | 'single-org' | 'flat-user' | 'mixed';
};
gitea: {
@@ -99,6 +100,9 @@ function parseEnvConfig(): EnvConfig {
const protectedRepos = process.env.CLEANUP_PROTECTED_REPOS
? process.env.CLEANUP_PROTECTED_REPOS.split(',').map(r => r.trim()).filter(Boolean)
: undefined;
const starredLists = process.env.MIRROR_STARRED_LISTS
? process.env.MIRROR_STARRED_LISTS.split(',').map((list) => list.trim()).filter(Boolean)
: undefined;
return {
github: {
@@ -117,6 +121,7 @@ function parseEnvConfig(): EnvConfig {
autoMirrorStarred: process.env.AUTO_MIRROR_STARRED === 'true',
starredReposOrg: process.env.STARRED_REPOS_ORG,
starredReposMode: process.env.STARRED_REPOS_MODE as 'dedicated-org' | 'preserve-owner',
starredLists,
mirrorStrategy: process.env.MIRROR_STRATEGY as 'preserve' | 'single-org' | 'flat-user' | 'mixed',
},
gitea: {
@@ -267,6 +272,7 @@ export async function initializeConfigFromEnv(): Promise<void> {
defaultOrg: envConfig.gitea.organization || existingConfig?.[0]?.githubConfig?.defaultOrg || 'github-mirrors',
starredCodeOnly: envConfig.github.starredCodeOnly ?? existingConfig?.[0]?.githubConfig?.starredCodeOnly ?? false,
autoMirrorStarred: envConfig.github.autoMirrorStarred ?? existingConfig?.[0]?.githubConfig?.autoMirrorStarred ?? false,
starredLists: envConfig.github.starredLists ?? existingConfig?.[0]?.githubConfig?.starredLists ?? [],
};
// Build Gitea config

View File

@@ -555,6 +555,63 @@ describe("Enhanced Gitea Operations", () => {
expect(releaseCall.octokit).toBeDefined();
});
test("prefers recorded mirroredLocation when owner resolution changes", async () => {
mockGetGiteaRepoOwnerAsync.mockImplementation(() => Promise.resolve("ceph"));
const config: Partial<Config> = {
userId: "user123",
githubConfig: {
username: "testuser",
token: "github-token",
privateRepositories: false,
mirrorStarred: true,
},
giteaConfig: {
url: "https://gitea.example.com",
token: "encrypted-token",
defaultOwner: "testuser",
mirrorReleases: true,
},
};
const repository: Repository = {
id: "repo789",
name: "test-repo",
fullName: "ceph/test-repo",
owner: "ceph",
cloneUrl: "https://github.com/ceph/test-repo.git",
isPrivate: false,
isStarred: true,
status: repoStatusEnum.parse("mirrored"),
visibility: "public",
userId: "user123",
mirroredLocation: "starred/test-repo",
createdAt: new Date(),
updatedAt: new Date(),
};
const result = await syncGiteaRepoEnhanced(
{ config, repository },
{
getGiteaRepoOwnerAsync: mockGetGiteaRepoOwnerAsync,
mirrorGitHubReleasesToGitea: mockMirrorGitHubReleasesToGitea,
mirrorGitRepoIssuesToGitea: mockMirrorGitRepoIssuesToGitea,
mirrorGitRepoPullRequestsToGitea: mockMirrorGitRepoPullRequestsToGitea,
mirrorGitRepoLabelsToGitea: mockMirrorGitRepoLabelsToGitea,
mirrorGitRepoMilestonesToGitea: mockMirrorGitRepoMilestonesToGitea,
}
);
expect(result).toEqual({ success: true });
const mirrorSyncCalls = mockHttpPost.mock.calls.filter((call) =>
String(call[0]).includes("/mirror-sync")
);
expect(mirrorSyncCalls).toHaveLength(1);
expect(String(mirrorSyncCalls[0][0])).toContain("/api/v1/repos/starred/test-repo/mirror-sync");
expect(String(mirrorSyncCalls[0][0])).not.toContain("/api/v1/repos/ceph/test-repo/mirror-sync");
});
test("blocks sync when pre-sync snapshot fails and blocking is enabled", async () => {
mockShouldCreatePreSyncBackup = true;
mockShouldBlockSyncOnBackupFailure = true;

View File

@@ -52,6 +52,41 @@ interface GiteaRepoInfo {
private: boolean;
}
interface SyncTargetCandidate {
owner: string;
repoName: string;
}
function parseMirroredLocation(location?: string | null): SyncTargetCandidate | null {
if (!location) return null;
const trimmed = location.trim();
if (!trimmed) return null;
const slashIndex = trimmed.indexOf("/");
if (slashIndex <= 0 || slashIndex === trimmed.length - 1) return null;
const owner = trimmed.slice(0, slashIndex).trim();
const repoName = trimmed.slice(slashIndex + 1).trim();
if (!owner || !repoName) return null;
return { owner, repoName };
}
function dedupeSyncTargets(targets: SyncTargetCandidate[]): SyncTargetCandidate[] {
const seen = new Set<string>();
const deduped: SyncTargetCandidate[] = [];
for (const target of targets) {
const key = `${target.owner}/${target.repoName}`.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
deduped.push(target);
}
return deduped;
}
/**
* Check if a repository exists in Gitea and return its details
*/
@@ -285,19 +320,78 @@ export async function syncGiteaRepoEnhanced({
})
.where(eq(repositories.id, repository.id!));
// Get the expected owner
// Resolve sync target in a backward-compatible order:
// 1) recorded mirroredLocation (actual historical mirror location)
// 2) owner derived from current strategy/config
const dependencies = deps ?? (await import("./gitea"));
const repoOwner = await dependencies.getGiteaRepoOwnerAsync({ config, repository });
const expectedOwner = await dependencies.getGiteaRepoOwnerAsync({ config, repository });
const recordedTarget = parseMirroredLocation(repository.mirroredLocation);
const candidateTargets = dedupeSyncTargets([
...(recordedTarget ? [recordedTarget] : []),
{ owner: expectedOwner, repoName: repository.name },
]);
// Check if repo exists and get its info
const repoInfo = await getGiteaRepoInfo({
config,
owner: repoOwner,
repoName: repository.name,
});
let repoOwner = expectedOwner;
let repoName = repository.name;
let repoInfo: GiteaRepoInfo | null = null;
let firstNonMirrorTarget: SyncTargetCandidate | null = null;
for (const target of candidateTargets) {
const candidateInfo = await getGiteaRepoInfo({
config,
owner: target.owner,
repoName: target.repoName,
});
if (!candidateInfo) {
continue;
}
if (!candidateInfo.mirror) {
if (!firstNonMirrorTarget) {
firstNonMirrorTarget = target;
}
continue;
}
repoOwner = target.owner;
repoName = target.repoName;
repoInfo = candidateInfo;
break;
}
if (!repoInfo) {
throw new Error(`Repository ${repository.name} not found in Gitea at ${repoOwner}/${repository.name}`);
if (firstNonMirrorTarget) {
console.warn(
`[Sync] Repository ${repository.name} exists at ${firstNonMirrorTarget.owner}/${firstNonMirrorTarget.repoName} but is not configured as a mirror`
);
await db
.update(repositories)
.set({
status: repoStatusEnum.parse("failed"),
updatedAt: new Date(),
errorMessage: "Repository exists in Gitea but is not configured as a mirror. Manual intervention required.",
})
.where(eq(repositories.id, repository.id!));
await createMirrorJob({
userId: config.userId,
repositoryId: repository.id,
repositoryName: repository.name,
message: `Cannot sync ${repository.name}: Not a mirror repository`,
details: `Repository ${repository.name} exists in Gitea but is not configured as a mirror. You may need to delete and recreate it as a mirror, or manually configure it as a mirror in Gitea.`,
status: "failed",
});
throw new Error(`Repository ${repository.name} is not a mirror. Cannot sync.`);
}
throw new Error(
`Repository ${repository.name} not found in Gitea. Tried locations: ${candidateTargets
.map((t) => `${t.owner}/${t.repoName}`)
.join(", ")}`
);
}
// Check if it's a mirror repository
@@ -342,7 +436,7 @@ export async function syncGiteaRepoEnhanced({
giteaUrl: config.giteaConfig.url,
giteaToken: decryptedConfig.giteaConfig.token,
giteaOwner: repoOwner,
giteaRepo: repository.name,
giteaRepo: repoName,
octokit: fpOctokit,
githubOwner: repository.owner,
githubRepo: repository.name,
@@ -407,13 +501,13 @@ export async function syncGiteaRepoEnhanced({
if (shouldBackupForStrategy(backupStrategy, forcePushDetected)) {
const cloneUrl =
repoInfo.clone_url ||
`${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repository.name}.git`;
`${config.giteaConfig.url.replace(/\/$/, "")}/${repoOwner}/${repoName}.git`;
try {
const backupResult = await createPreSyncBundleBackup({
config,
owner: repoOwner,
repoName: repository.name,
repoName,
cloneUrl,
force: true, // Strategy already decided to backup; skip legacy gate
});
@@ -464,22 +558,22 @@ export async function syncGiteaRepoEnhanced({
// Update mirror interval if needed
if (config.giteaConfig?.mirrorInterval) {
try {
console.log(`[Sync] Updating mirror interval for ${repository.name} to ${config.giteaConfig.mirrorInterval}`);
const updateUrl = `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repository.name}`;
console.log(`[Sync] Updating mirror interval for ${repoOwner}/${repoName} to ${config.giteaConfig.mirrorInterval}`);
const updateUrl = `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}`;
await httpPatch(updateUrl, {
mirror_interval: config.giteaConfig.mirrorInterval,
}, {
Authorization: `token ${decryptedConfig.giteaConfig.token}`,
});
console.log(`[Sync] Successfully updated mirror interval for ${repository.name}`);
console.log(`[Sync] Successfully updated mirror interval for ${repoOwner}/${repoName}`);
} catch (updateError) {
console.warn(`[Sync] Failed to update mirror interval for ${repository.name}:`, updateError);
console.warn(`[Sync] Failed to update mirror interval for ${repoOwner}/${repoName}:`, updateError);
// Continue with sync even if interval update fails
}
}
// Perform the sync
const apiUrl = `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repository.name}/mirror-sync`;
const apiUrl = `${config.giteaConfig.url}/api/v1/repos/${repoOwner}/${repoName}/mirror-sync`;
try {
const response = await httpPost(apiUrl, undefined, {
@@ -536,7 +630,7 @@ export async function syncGiteaRepoEnhanced({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: repository.name,
giteaRepoName: repoName,
});
metadataState.components.releases = true;
metadataUpdated = true;
@@ -568,7 +662,7 @@ export async function syncGiteaRepoEnhanced({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: repository.name,
giteaRepoName: repoName,
});
metadataState.components.issues = true;
metadataState.components.labels = true;
@@ -601,7 +695,7 @@ export async function syncGiteaRepoEnhanced({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: repository.name,
giteaRepoName: repoName,
});
metadataState.components.pullRequests = true;
metadataUpdated = true;
@@ -631,7 +725,7 @@ export async function syncGiteaRepoEnhanced({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: repository.name,
giteaRepoName: repoName,
});
metadataState.components.labels = true;
metadataUpdated = true;
@@ -670,7 +764,7 @@ export async function syncGiteaRepoEnhanced({
octokit,
repository,
giteaOwner: repoOwner,
giteaRepoName: repository.name,
giteaRepoName: repoName,
});
metadataState.components.milestones = true;
metadataUpdated = true;
@@ -708,7 +802,7 @@ export async function syncGiteaRepoEnhanced({
updatedAt: new Date(),
lastMirrored: new Date(),
errorMessage: null,
mirroredLocation: `${repoOwner}/${repository.name}`,
mirroredLocation: `${repoOwner}/${repoName}`,
metadata: metadataUpdated
? serializeRepositoryMetadataState(metadataState)
: repository.metadata ?? null,
@@ -720,7 +814,7 @@ export async function syncGiteaRepoEnhanced({
repositoryId: repository.id,
repositoryName: repository.name,
message: `Sync requested for repository: ${repository.name}`,
details: `Mirror sync was requested for ${repository.name}.`,
details: `Mirror sync was requested for ${repoOwner}/${repoName}.`,
status: "synced",
});

View File

@@ -13,6 +13,7 @@ import { db, organizations, repositories } from "./db";
import { eq, and, ne } from "drizzle-orm";
import { decryptConfigTokens } from "./utils/config-encryption";
import { formatDateShort } from "./utils";
import { buildGithubSourceAuthPayload } from "./utils/mirror-source-auth";
import {
parseRepositoryMetadataState,
serializeRepositoryMetadataState,
@@ -816,14 +817,22 @@ export const mirrorGithubRepoToGitea = async ({
// Add authentication for private repositories
if (repository.isPrivate) {
if (!config.githubConfig.token) {
throw new Error(
"GitHub token is required to mirror private repositories."
);
}
// Use separate auth fields (required for Forgejo 12+ compatibility)
migratePayload.auth_username = "oauth2"; // GitHub tokens work with any username
migratePayload.auth_token = decryptedConfig.githubConfig.token;
const githubOwner =
(
config.githubConfig as typeof config.githubConfig & {
owner?: string;
}
).owner || "";
Object.assign(
migratePayload,
buildGithubSourceAuthPayload({
token: decryptedConfig.githubConfig.token,
githubOwner,
githubUsername: config.githubConfig.username,
repositoryOwner: repository.owner,
})
);
}
// Track whether the Gitea migrate call succeeded so the catch block
@@ -1496,14 +1505,22 @@ export async function mirrorGitHubRepoToGiteaOrg({
// Add authentication for private repositories
if (repository.isPrivate) {
if (!config.githubConfig?.token) {
throw new Error(
"GitHub token is required to mirror private repositories."
);
}
// Use separate auth fields (required for Forgejo 12+ compatibility)
migratePayload.auth_username = "oauth2"; // GitHub tokens work with any username
migratePayload.auth_token = decryptedConfig.githubConfig.token;
const githubOwner =
(
config.githubConfig as typeof config.githubConfig & {
owner?: string;
}
)?.owner || "";
Object.assign(
migratePayload,
buildGithubSourceAuthPayload({
token: decryptedConfig.githubConfig?.token,
githubOwner,
githubUsername: config.githubConfig?.username,
repositoryOwner: repository.owner,
})
);
}
let migrateSucceeded = false;

View File

@@ -0,0 +1,319 @@
import { describe, expect, test, mock } from "bun:test";
import {
getGithubStarredListNames,
getGithubStarredRepositories,
} from "@/lib/github";
function makeRestStarredRepo(overrides: Record<string, unknown> = {}) {
return {
name: "demo",
full_name: "acme/demo",
html_url: "https://github.com/acme/demo",
clone_url: "https://github.com/acme/demo.git",
owner: {
login: "acme",
type: "Organization",
},
private: false,
fork: false,
has_issues: true,
archived: false,
size: 123,
language: "TypeScript",
description: "Demo",
default_branch: "main",
visibility: "public",
disabled: false,
created_at: "2024-01-01T00:00:00Z",
updated_at: "2024-01-02T00:00:00Z",
...overrides,
};
}
function makeGraphqlListRepo(
nameWithOwner: string,
overrides: Record<string, unknown> = {},
) {
const [owner, name] = nameWithOwner.split("/");
return {
__typename: "Repository" as const,
name,
nameWithOwner,
url: `https://github.com/${nameWithOwner}`,
sshUrl: `git@github.com:${nameWithOwner}.git`,
isPrivate: false,
isFork: false,
isArchived: false,
isDisabled: false,
hasIssuesEnabled: true,
diskUsage: 456,
description: `${name} repo`,
defaultBranchRef: { name: "main" },
visibility: "PUBLIC" as const,
updatedAt: "2024-01-02T00:00:00Z",
createdAt: "2024-01-01T00:00:00Z",
owner: {
__typename: "Organization" as const,
login: owner,
},
primaryLanguage: { name: "TypeScript" },
...overrides,
};
}
describe("GitHub starred lists support", () => {
test("falls back to REST starred endpoint when no lists are configured", async () => {
const paginate = mock(async () => [makeRestStarredRepo()]);
const graphql = mock(async () => {
throw new Error("GraphQL should not be used in REST fallback path");
});
const octokit = {
paginate,
graphql,
activity: {
listReposStarredByAuthenticatedUser: () => {},
},
} as any;
const repos = await getGithubStarredRepositories({
octokit,
config: { githubConfig: { starredLists: [] } } as any,
});
expect(repos).toHaveLength(1);
expect(repos[0].fullName).toBe("acme/demo");
expect(repos[0].isStarred).toBe(true);
expect(paginate).toHaveBeenCalledTimes(1);
expect(graphql).toHaveBeenCalledTimes(0);
});
test("filters starred repositories by configured list names and de-duplicates", async () => {
const paginate = mock(async () => []);
const graphql = mock(async (_query: string, variables?: Record<string, unknown>) => {
if (!variables || !("listId" in variables)) {
return {
viewer: {
lists: {
nodes: [
null,
{ id: "list-1", name: "HomeLab" },
{ id: "list-2", name: "DotTools" },
{ id: "list-3", name: "Ideas" },
],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
}
if (variables.listId === "list-1") {
return {
node: {
items: {
nodes: [
null,
makeGraphqlListRepo("acme/repo-a"),
makeGraphqlListRepo("acme/repo-b"),
],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
}
return {
node: {
items: {
nodes: [
makeGraphqlListRepo("acme/repo-b"),
makeGraphqlListRepo("acme/repo-c"),
],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
});
const octokit = {
paginate,
graphql,
activity: {
listReposStarredByAuthenticatedUser: () => {},
},
} as any;
const repos = await getGithubStarredRepositories({
octokit,
config: {
githubConfig: {
starredLists: ["homelab", "dottools"],
},
} as any,
});
expect(repos).toHaveLength(3);
expect(repos.map((repo) => repo.fullName).sort()).toEqual([
"acme/repo-a",
"acme/repo-b",
"acme/repo-c",
]);
expect(paginate).toHaveBeenCalledTimes(0);
});
test("matches configured list names even when separators differ", async () => {
const paginate = mock(async () => []);
const graphql = mock(async (_query: string, variables?: Record<string, unknown>) => {
if (!variables || !("listId" in variables)) {
return {
viewer: {
lists: {
nodes: [
{ id: "list-1", name: "UI Frontend" },
{ id: "list-2", name: "Email | Self - Hosted" },
{ id: "list-3", name: "PaaS | Hosting | Deploy" },
],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
}
if (variables.listId === "list-1") {
return {
node: {
items: {
nodes: [makeGraphqlListRepo("acme/ui-app")],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
}
if (variables.listId === "list-2") {
return {
node: {
items: {
nodes: [makeGraphqlListRepo("acme/email-app")],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
}
return {
node: {
items: {
nodes: [makeGraphqlListRepo("acme/paas-app")],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
});
const octokit = {
paginate,
graphql,
activity: {
listReposStarredByAuthenticatedUser: () => {},
},
} as any;
const repos = await getGithubStarredRepositories({
octokit,
config: {
githubConfig: {
starredLists: ["ui-frontend", "email-self-hosted", "paas-hosting-deploy"],
},
} as any,
});
expect(repos).toHaveLength(3);
expect(repos.map((repo) => repo.fullName).sort()).toEqual([
"acme/email-app",
"acme/paas-app",
"acme/ui-app",
]);
expect(paginate).toHaveBeenCalledTimes(0);
});
test("throws when configured star list names do not match any GitHub list", async () => {
const paginate = mock(async () => []);
const graphql = mock(async (_query: string, variables?: Record<string, unknown>) => {
if (!variables || !("listId" in variables)) {
return {
viewer: {
lists: {
nodes: [{ id: "list-1", name: "HomeLab" }],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
}
return {
node: {
items: {
nodes: [],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
});
const octokit = {
paginate,
graphql,
activity: {
listReposStarredByAuthenticatedUser: () => {},
},
} as any;
await expect(
getGithubStarredRepositories({
octokit,
config: {
githubConfig: {
starredLists: ["MissingList"],
},
} as any,
}),
).rejects.toThrow("Configured GitHub star lists not found");
expect(paginate).toHaveBeenCalledTimes(0);
});
test("returns all available starred list names with pagination", async () => {
const graphql = mock(async (_query: string, variables?: Record<string, unknown>) => {
if (!variables?.after) {
return {
viewer: {
lists: {
nodes: [
null,
{ id: "a", name: "HomeLab" },
{ id: "b", name: "DotTools" },
],
pageInfo: { hasNextPage: true, endCursor: "cursor-1" },
},
},
};
}
return {
viewer: {
lists: {
nodes: [
{ id: "c", name: "Ideas" },
],
pageInfo: { hasNextPage: false, endCursor: null },
},
},
};
});
const octokit = { graphql } as any;
const lists = await getGithubStarredListNames({ octokit });
expect(lists).toEqual(["HomeLab", "DotTools", "Ideas"]);
expect(graphql).toHaveBeenCalledTimes(2);
});
});

View File

@@ -300,6 +300,239 @@ export async function getGithubRepositories({
}
}
function getStarredListMatchKey(rawValue: string): string {
const normalized = rawValue.normalize("NFKC").trim().toLowerCase();
const tokens = normalized.match(/[\p{L}\p{N}]+/gu);
return tokens ? tokens.join("") : "";
}
function normalizeStarredListNames(rawLists: unknown): string[] {
if (!Array.isArray(rawLists)) return [];
const deduped = new Map<string, string>();
for (const value of rawLists) {
if (typeof value !== "string") continue;
const trimmed = value.trim();
if (!trimmed) continue;
const matchKey = getStarredListMatchKey(trimmed);
if (!matchKey || deduped.has(matchKey)) continue;
deduped.set(matchKey, trimmed);
}
return [...deduped.values()];
}
function toHttpsCloneUrl(repoUrl: string): string {
return repoUrl.endsWith(".git") ? repoUrl : `${repoUrl}.git`;
}
interface GitHubStarListNode {
id: string;
name: string;
}
interface GitHubRepositoryListItem {
__typename: "Repository";
name: string;
nameWithOwner: string;
url: string;
sshUrl: string;
isPrivate: boolean;
isFork: boolean;
isArchived: boolean;
isDisabled: boolean;
hasIssuesEnabled: boolean;
diskUsage: number;
description: string | null;
defaultBranchRef: { name: string } | null;
visibility: "PUBLIC" | "PRIVATE" | "INTERNAL";
updatedAt: string;
createdAt: string;
owner: {
__typename: "Organization" | "User" | string;
login: string;
};
primaryLanguage: {
name: string;
} | null;
}
async function getGithubStarLists(octokit: Octokit): Promise<GitHubStarListNode[]> {
const allLists: GitHubStarListNode[] = [];
let cursor: string | null = null;
do {
const result = await octokit.graphql<{
viewer: {
lists: {
nodes: Array<GitHubStarListNode | null> | null;
pageInfo: {
hasNextPage: boolean;
endCursor: string | null;
};
};
};
}>(
`
query($after: String) {
viewer {
lists(first: 50, after: $after) {
nodes {
id
name
}
pageInfo {
hasNextPage
endCursor
}
}
}
}
`,
{ after: cursor },
);
const lists = (result.viewer.lists.nodes ?? []).filter(
(list): list is GitHubStarListNode =>
!!list &&
typeof list.id === "string" &&
typeof list.name === "string",
);
allLists.push(...lists);
if (!result.viewer.lists.pageInfo.hasNextPage) break;
cursor = result.viewer.lists.pageInfo.endCursor;
} while (cursor);
return allLists;
}
async function getGithubRepositoriesForStarList(
octokit: Octokit,
listId: string,
): Promise<GitHubRepositoryListItem[]> {
const repositories: GitHubRepositoryListItem[] = [];
let cursor: string | null = null;
do {
const result = await octokit.graphql<{
node: {
items: {
nodes: Array<GitHubRepositoryListItem | null> | null;
pageInfo: {
hasNextPage: boolean;
endCursor: string | null;
};
};
} | null;
}>(
`
query($listId: ID!, $after: String) {
node(id: $listId) {
... on UserList {
items(first: 100, after: $after) {
nodes {
__typename
... on Repository {
name
nameWithOwner
url
sshUrl
isPrivate
isFork
isArchived
isDisabled
hasIssuesEnabled
diskUsage
description
defaultBranchRef {
name
}
visibility
updatedAt
createdAt
owner {
__typename
login
}
primaryLanguage {
name
}
}
}
pageInfo {
hasNextPage
endCursor
}
}
}
}
}
`,
{ listId, after: cursor },
);
const listNode = result.node;
if (!listNode) break;
const nodes = listNode.items.nodes ?? [];
for (const node of nodes) {
if (node?.__typename === "Repository") {
repositories.push(node);
}
}
if (!listNode.items.pageInfo.hasNextPage) break;
cursor = listNode.items.pageInfo.endCursor;
} while (cursor);
return repositories;
}
function mapGraphqlRepoToGitRepo(repo: GitHubRepositoryListItem): GitRepo {
const visibility = (repo.visibility ?? "PUBLIC").toLowerCase() as GitRepo["visibility"];
const createdAt = repo.createdAt ? new Date(repo.createdAt) : new Date();
const updatedAt = repo.updatedAt ? new Date(repo.updatedAt) : new Date();
return {
name: repo.name,
fullName: repo.nameWithOwner,
url: repo.url,
cloneUrl: toHttpsCloneUrl(repo.url),
owner: repo.owner.login,
organization: repo.owner.__typename === "Organization" ? repo.owner.login : undefined,
mirroredLocation: "",
destinationOrg: null,
isPrivate: repo.isPrivate,
isForked: repo.isFork,
forkedFrom: undefined,
hasIssues: repo.hasIssuesEnabled,
isStarred: true,
isArchived: repo.isArchived,
size: repo.diskUsage ?? 0,
hasLFS: false,
hasSubmodules: false,
language: repo.primaryLanguage?.name ?? null,
description: repo.description,
defaultBranch: repo.defaultBranchRef?.name || "main",
visibility,
status: "imported",
isDisabled: repo.isDisabled,
lastMirrored: undefined,
errorMessage: undefined,
importedAt: new Date(),
createdAt,
updatedAt,
};
}
export async function getGithubStarredRepositories({
octokit,
config,
@@ -308,6 +541,46 @@ export async function getGithubStarredRepositories({
config: Partial<Config>;
}): Promise<GitRepo[]> {
try {
const configuredLists = normalizeStarredListNames(
config.githubConfig?.starredLists,
);
if (configuredLists.length > 0) {
const allLists = await getGithubStarLists(octokit);
const configuredMatchKeySet = new Set(
configuredLists.map((list) => getStarredListMatchKey(list)),
);
const matchedLists = allLists.filter((list) =>
configuredMatchKeySet.has(getStarredListMatchKey(list.name)),
);
if (matchedLists.length === 0) {
const availableListNames = normalizeStarredListNames(
allLists.map((list) => list.name),
);
const preview = availableListNames.slice(0, 20).join(", ");
const availableSuffix = preview
? `. Available lists: ${preview}${availableListNames.length > 20 ? ", ..." : ""}`
: "";
throw new Error(
`Configured GitHub star lists not found: ${configuredLists.join(", ")}${availableSuffix}`,
);
}
const deduped = new Map<string, GitRepo>();
for (const list of matchedLists) {
const repos = await getGithubRepositoriesForStarList(octokit, list.id);
for (const repo of repos) {
const key = repo.nameWithOwner.toLowerCase();
if (deduped.has(key)) continue;
deduped.set(key, mapGraphqlRepoToGitRepo(repo));
}
}
return [...deduped.values()];
}
const starredRepos = await octokit.paginate(
octokit.activity.listReposStarredByAuthenticatedUser,
{
@@ -362,6 +635,15 @@ export async function getGithubStarredRepositories({
}
}
export async function getGithubStarredListNames({
octokit,
}: {
octokit: Octokit;
}): Promise<string[]> {
const lists = await getGithubStarLists(octokit);
return normalizeStarredListNames(lists.map((list) => list.name));
}
/**
* Get user github organizations
*/

View File

@@ -85,6 +85,7 @@ export async function createDefaultConfig({ userId, envOverrides = {} }: Default
includeOrganizations: [],
starredReposOrg: "starred",
starredReposMode: "dedicated-org",
starredLists: [],
mirrorStrategy: "preserve",
defaultOrg: "github-mirrors",
},

View File

@@ -20,6 +20,17 @@ type DbGiteaConfig = z.infer<typeof giteaConfigSchema>;
type DbScheduleConfig = z.infer<typeof scheduleConfigSchema>;
type DbCleanupConfig = z.infer<typeof cleanupConfigSchema>;
function normalizeStarredLists(lists: string[] | undefined): string[] {
if (!Array.isArray(lists)) return [];
const deduped = new Set<string>();
for (const list of lists) {
const trimmed = list.trim();
if (!trimmed) continue;
deduped.add(trimmed);
}
return [...deduped];
}
/**
* Maps UI config structure to database schema structure
*/
@@ -50,6 +61,7 @@ export function mapUiToDbConfig(
// Starred repos organization
starredReposOrg: giteaConfig.starredReposOrg,
starredReposMode: giteaConfig.starredReposMode || "dedicated-org",
starredLists: normalizeStarredLists(githubConfig.starredLists),
// Mirror strategy
mirrorStrategy: giteaConfig.mirrorStrategy || "preserve",
@@ -131,6 +143,7 @@ export function mapDbToUiConfig(dbConfig: any): {
token: dbConfig.githubConfig?.token || "",
privateRepositories: dbConfig.githubConfig?.includePrivate || false, // Map includePrivate to privateRepositories
mirrorStarred: dbConfig.githubConfig?.includeStarred || false, // Map includeStarred to mirrorStarred
starredLists: normalizeStarredLists(dbConfig.githubConfig?.starredLists),
};
// Map from database Gitea config to UI fields

View File

@@ -0,0 +1,63 @@
import { describe, expect, test } from "bun:test";
import { buildGithubSourceAuthPayload } from "./mirror-source-auth";
describe("buildGithubSourceAuthPayload", () => {
test("uses configured owner when available", () => {
const auth = buildGithubSourceAuthPayload({
token: "ghp_test_token",
githubOwner: "ConfiguredOwner",
githubUsername: "fallback-user",
repositoryOwner: "repo-owner",
});
expect(auth).toEqual({
auth_username: "ConfiguredOwner",
auth_password: "ghp_test_token",
auth_token: "ghp_test_token",
});
});
test("falls back to configured username then repository owner", () => {
const authFromUsername = buildGithubSourceAuthPayload({
token: "token1",
githubUsername: "configured-user",
repositoryOwner: "repo-owner",
});
expect(authFromUsername.auth_username).toBe("configured-user");
const authFromRepoOwner = buildGithubSourceAuthPayload({
token: "token2",
repositoryOwner: "repo-owner",
});
expect(authFromRepoOwner.auth_username).toBe("repo-owner");
});
test("uses x-access-token as last-resort username", () => {
const auth = buildGithubSourceAuthPayload({
token: "ghp_test_token",
});
expect(auth.auth_username).toBe("x-access-token");
});
test("trims token whitespace", () => {
const auth = buildGithubSourceAuthPayload({
token: " ghp_trimmed ",
githubUsername: "user",
});
expect(auth.auth_password).toBe("ghp_trimmed");
expect(auth.auth_token).toBe("ghp_trimmed");
});
test("throws when token is missing", () => {
expect(() =>
buildGithubSourceAuthPayload({
token: " ",
githubUsername: "user",
})
).toThrow("GitHub token is required to mirror private repositories.");
});
});

View File

@@ -0,0 +1,46 @@
interface BuildGithubSourceAuthPayloadParams {
token?: string | null;
githubOwner?: string | null;
githubUsername?: string | null;
repositoryOwner?: string | null;
}
export interface GithubSourceAuthPayload {
auth_username: string;
auth_password: string;
auth_token: string;
}
const DEFAULT_GITHUB_AUTH_USERNAME = "x-access-token";
function normalize(value?: string | null): string {
return typeof value === "string" ? value.trim() : "";
}
/**
* Build source credentials for private GitHub repository mirroring.
* GitHub expects username + token-as-password over HTTPS (not the GitLab-style "oauth2" username).
*/
export function buildGithubSourceAuthPayload({
token,
githubOwner,
githubUsername,
repositoryOwner,
}: BuildGithubSourceAuthPayloadParams): GithubSourceAuthPayload {
const normalizedToken = normalize(token);
if (!normalizedToken) {
throw new Error("GitHub token is required to mirror private repositories.");
}
const authUsername =
normalize(githubOwner) ||
normalize(githubUsername) ||
normalize(repositoryOwner) ||
DEFAULT_GITHUB_AUTH_USERNAME;
return {
auth_username: authUsername,
auth_password: normalizedToken,
auth_token: normalizedToken,
};
}

View File

@@ -13,7 +13,6 @@ try {
activityData = jobs.flatMap((job: any) => {
// Check if log exists before parsing
if (!job.log) {
console.warn(`Job ${job.id} has no log data`);
return [];
}

View File

@@ -0,0 +1,53 @@
import type { APIRoute } from "astro";
import { db, configs } from "@/lib/db";
import { eq } from "drizzle-orm";
import {
createGitHubClient,
getGithubStarredListNames,
} from "@/lib/github";
import { getDecryptedGitHubToken } from "@/lib/utils/config-encryption";
import { jsonResponse, createSecureErrorResponse } from "@/lib/utils";
import { requireAuthenticatedUserId } from "@/lib/auth-guards";
export const GET: APIRoute = async ({ request, locals }) => {
try {
const authResult = await requireAuthenticatedUserId({ request, locals });
if ("response" in authResult) return authResult.response;
const userId = authResult.userId;
const [config] = await db
.select()
.from(configs)
.where(eq(configs.userId, userId))
.limit(1);
if (!config) {
return jsonResponse({
data: { success: false, message: "No configuration found for this user" },
status: 404,
});
}
if (!config.githubConfig?.token) {
return jsonResponse({
data: { success: false, message: "GitHub token is missing in config" },
status: 400,
});
}
const token = getDecryptedGitHubToken(config);
const githubUsername = config.githubConfig?.owner || undefined;
const octokit = createGitHubClient(token, userId, githubUsername);
const lists = await getGithubStarredListNames({ octokit });
return jsonResponse({
data: {
success: true,
lists,
},
status: 200,
});
} catch (error) {
return createSecureErrorResponse(error, "starred lists fetch", 500);
}
};

View File

@@ -28,7 +28,6 @@ try {
activityData = jobs.flatMap((job: any) => {
// Check if log exists before parsing
if (!job.log) {
console.warn(`Job ${job.id} has no log data`);
return [];
}
try {
@@ -68,4 +67,4 @@ try {
<body>
<App page='dashboard' client:load />
</body>
</html>
</html>

View File

@@ -61,6 +61,7 @@ export interface GitHubConfig {
token: string;
privateRepositories: boolean;
mirrorStarred: boolean;
starredLists?: string[];
starredDuplicateStrategy?: DuplicateNameStrategy;
starredReposMode?: StarredReposMode;
}

50
www/pnpm-lock.yaml generated
View File

@@ -1199,8 +1199,8 @@ packages:
graceful-fs@4.2.11:
resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==}
h3@1.15.5:
resolution: {integrity: sha512-xEyq3rSl+dhGX2Lm0+eFQIAzlDN6Fs0EcC4f7BNUmzaRX/PTzeuM+Tr2lHB8FoXggsQIeXLj8EDVgs5ywxyxmg==}
h3@1.15.9:
resolution: {integrity: sha512-H7UPnyIupUOYUQu7f2x7ABVeMyF/IbJjqn20WSXpMdnQB260luADUkSgJU7QTWLutq8h3tUayMQ1DdbSYX5LkA==}
hast-util-from-html@2.0.3:
resolution: {integrity: sha512-CUSRHXyKjzHov8yKsQjGOElXy/3EKpyX56ELnkHH34vDVw1N1XSQ1ZcAvTyAPtGqLTuKP/uxM+aLkSPqF/EtMw==}
@@ -1655,12 +1655,12 @@ packages:
picocolors@1.1.1:
resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==}
picomatch@2.3.1:
resolution: {integrity: sha512-JU3teHTNjmE2VCGFzuY8EXzCDVwEqB2a8fsIvwaStHhAWJEeVd1o1QD80CU6+ZdEXXSLbSsuLwJjkCBWqRQUVA==}
picomatch@2.3.2:
resolution: {integrity: sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA==}
engines: {node: '>=8.6'}
picomatch@4.0.3:
resolution: {integrity: sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==}
picomatch@4.0.4:
resolution: {integrity: sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==}
engines: {node: '>=12'}
postcss@8.5.6:
@@ -1801,8 +1801,8 @@ packages:
sisteransi@1.0.5:
resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==}
smol-toml@1.6.0:
resolution: {integrity: sha512-4zemZi0HvTnYwLfrpk/CF9LOd9Lt87kAt50GnqhMpyF9U3poDAP2+iukq2bZsO/ufegbYehBkqINbsWxj4l4cw==}
smol-toml@1.6.1:
resolution: {integrity: sha512-dWUG8F5sIIARXih1DTaQAX4SsiTXhInKf1buxdY9DIg4ZYPZK5nGM1VRIYmEbDbsHt7USo99xSLFu5Q1IqTmsg==}
engines: {node: '>= 18'}
source-map-js@1.2.1:
@@ -2091,7 +2091,7 @@ snapshots:
'@astrojs/internal-helpers@0.8.0':
dependencies:
picomatch: 4.0.3
picomatch: 4.0.4
'@astrojs/markdown-remark@7.0.0':
dependencies:
@@ -2109,7 +2109,7 @@ snapshots:
remark-rehype: 11.1.2
remark-smartypants: 3.0.2
shiki: 4.0.2
smol-toml: 1.6.0
smol-toml: 1.6.1
unified: 11.0.5
unist-util-remove-position: 5.0.0
unist-util-visit: 5.1.0
@@ -2553,7 +2553,7 @@ snapshots:
dependencies:
'@types/estree': 1.0.8
estree-walker: 2.0.2
picomatch: 4.0.3
picomatch: 4.0.4
optionalDependencies:
rollup: 4.59.0
@@ -2844,7 +2844,7 @@ snapshots:
anymatch@3.1.3:
dependencies:
normalize-path: 3.0.0
picomatch: 2.3.1
picomatch: 2.3.2
argparse@2.0.1: {}
@@ -2891,11 +2891,11 @@ snapshots:
p-queue: 9.1.0
package-manager-detector: 1.6.0
piccolore: 0.1.3
picomatch: 4.0.3
picomatch: 4.0.4
rehype: 13.0.2
semver: 7.7.4
shiki: 4.0.2
smol-toml: 1.6.0
smol-toml: 1.6.1
svgo: 4.0.1
tinyclip: 0.1.12
tinyexec: 1.0.2
@@ -3181,9 +3181,9 @@ snapshots:
extend@3.0.2: {}
fdir@6.5.0(picomatch@4.0.3):
fdir@6.5.0(picomatch@4.0.4):
optionalDependencies:
picomatch: 4.0.3
picomatch: 4.0.4
flattie@1.1.1: {}
@@ -3204,7 +3204,7 @@ snapshots:
graceful-fs@4.2.11: {}
h3@1.15.5:
h3@1.15.9:
dependencies:
cookie-es: 1.2.2
crossws: 0.3.5
@@ -3987,9 +3987,9 @@ snapshots:
picocolors@1.1.1: {}
picomatch@2.3.1: {}
picomatch@2.3.2: {}
picomatch@4.0.3: {}
picomatch@4.0.4: {}
postcss@8.5.6:
dependencies:
@@ -4247,7 +4247,7 @@ snapshots:
sisteransi@1.0.5: {}
smol-toml@1.6.0: {}
smol-toml@1.6.1: {}
source-map-js@1.2.1: {}
@@ -4294,8 +4294,8 @@ snapshots:
tinyglobby@0.2.15:
dependencies:
fdir: 6.5.0(picomatch@4.0.3)
picomatch: 4.0.3
fdir: 6.5.0(picomatch@4.0.4)
picomatch: 4.0.4
trim-lines@3.0.1: {}
@@ -4389,7 +4389,7 @@ snapshots:
anymatch: 3.1.3
chokidar: 5.0.0
destr: 2.0.5
h3: 1.15.5
h3: 1.15.9
lru-cache: 11.2.6
node-fetch-native: 1.6.7
ofetch: 1.5.1
@@ -4419,8 +4419,8 @@ snapshots:
vite@7.3.1(@types/node@24.7.1)(jiti@2.6.1)(lightningcss@1.31.1):
dependencies:
esbuild: 0.27.3
fdir: 6.5.0(picomatch@4.0.3)
picomatch: 4.0.3
fdir: 6.5.0(picomatch@4.0.4)
picomatch: 4.0.4
postcss: 8.5.6
rollup: 4.59.0
tinyglobby: 0.2.15