Added Github API rate limiting

- Implemented comprehensive GitHub API rate limit handling:
    - Integrated @octokit/plugin-throttling for automatic retry with exponential backoff
    - Added RateLimitManager service to track and enforce rate limits
    - Store rate limit status in database for persistence across restarts
    - Automatic pause and resume when limits are exceeded
    - Proper user identification for 5000 req/hr authenticated limit (vs 60 unauthenticated)

  - Improved rate limit UI/UX:
    - Removed intrusive rate limit card from dashboard
    - Toast notifications only at critical thresholds (80% and 100% usage)
    - All rate limit events logged for debugging

  - Optimized for GitHub's API constraints:
    - Reduced default batch size from 10 to 5 repositories
    - Added documentation about GitHub's 100 concurrent request limit
    - Better handling of repositories with many issues/PRs
This commit is contained in:
Arunavo Ray
2025-09-09 11:14:43 +05:30
parent 89ca5abe7d
commit 37e5b68bd5
22 changed files with 2873 additions and 22 deletions

View File

@@ -260,11 +260,13 @@ async function recoverMirrorJob(job: any, remainingItemIds: string[]) {
throw new Error('GitHub token not found in configuration');
}
// Create GitHub client with error handling
// Create GitHub client with error handling and rate limit tracking
let octokit;
try {
const decryptedToken = getDecryptedGitHubToken(config);
octokit = createGitHubClient(decryptedToken);
const githubUsername = config.githubConfig?.owner || undefined;
const userId = config.userId || undefined;
octokit = createGitHubClient(decryptedToken, userId, githubUsername);
} catch (error) {
throw new Error(`Failed to create GitHub client: ${error instanceof Error ? error.message : String(error)}`);
}