Added Github API rate limiting

- Implemented comprehensive GitHub API rate limit handling:
    - Integrated @octokit/plugin-throttling for automatic retry with exponential backoff
    - Added RateLimitManager service to track and enforce rate limits
    - Store rate limit status in database for persistence across restarts
    - Automatic pause and resume when limits are exceeded
    - Proper user identification for 5000 req/hr authenticated limit (vs 60 unauthenticated)

  - Improved rate limit UI/UX:
    - Removed intrusive rate limit card from dashboard
    - Toast notifications only at critical thresholds (80% and 100% usage)
    - All rate limit events logged for debugging

  - Optimized for GitHub's API constraints:
    - Reduced default batch size from 10 to 5 repositories
    - Added documentation about GitHub's 100 concurrent request limit
    - Better handling of repositories with many issues/PRs
This commit is contained in:
Arunavo Ray
2025-09-09 11:14:43 +05:30
parent 89ca5abe7d
commit 37e5b68bd5
22 changed files with 2873 additions and 22 deletions

View File

@@ -71,12 +71,13 @@ export const POST: APIRoute = async ({ request }) => {
// Start background retry with parallel processing
setTimeout(async () => {
// Create a single Octokit instance to be reused if needed
// Create a single Octokit instance to be reused if needed with rate limit tracking
const decryptedToken = config.githubConfig.token
? getDecryptedGitHubToken(config)
: null;
const githubUsername = config.githubConfig?.owner || undefined;
const octokit = decryptedToken
? createGitHubClient(decryptedToken)
? createGitHubClient(decryptedToken, userId, githubUsername)
: null;
// Define the concurrency limit - adjust based on API rate limits