feat: enable multi-page stars' fetching for private vercel instances

This commit enables multi-page stars' support from fetching on private Vercel instances. This feature can be disabled on the public Vercel instance by adding the `FETCH_SINGLE_PAGE_STARS=true` as an env variable in the public Vercel instance. This variable will not be present when people deploy their own Vercel instance, causing the code to fetch multiple star pages.
anuraghazra · Oct 8, 2022 · 56c9aca · 56c9aca
1 parent 39535db
commit 56c9aca
Show file tree

Hide file tree

Showing 3 changed files with 84 additions and 33 deletions.
diff --git a/readme.md b/readme.md
@@ -80,6 +80,7 @@ Visit <https://indiafightscorona.giveindia.org> and make a small donation to hel
 
 # Features
 
+-   [Important Notice](#important-notice)
 -   [GitHub Stats Card](#github-stats-card)
 -   [GitHub Extra Pins](#github-extra-pins)
 -   [Top Languages Card](#top-languages-card)
@@ -93,6 +94,11 @@ Visit <https://indiafightscorona.giveindia.org> and make a small donation to hel
     -   [Wakatime Card Exclusive Option](#wakatime-card-exclusive-options)
 -   [Deploy Yourself](#deploy-on-your-own-vercel-instance)
 
+# Important Notice
+
+> **Warning**
+> Since the GitHub API only [allows 5k requests per hour per user account](https://docs.github.com/en/graphql/overview/resource-limitations), the public Vercel instance hosted on `https://github-readme-stats.vercel.app/api` could possibly hit the rate limiter (see #1471). Because of this, we have limited the public API to only fetch the first 100 repositories with stars and languages. As a result, for a very active GitHub account, the language, stars and commits results might be off when using the public API. These limits do not apply when you deploy [your own Vercel instance](#deploy-on-your-own-vercel-instance), so in that case, you do not have to worry about anything! :rocket:
+
 # GitHub Stats Card
 
 Copy-paste this into your markdown content, and that is it. Simple!
@@ -167,13 +173,13 @@ You can customize the appearance of your `Stats Card` or `Repo Card` however you
 -   `border_color` - Card's border color _(hex color)_. Default: `e4e2e2` (Does not apply when `hide_border` is enabled).
 -   `bg_color` - Card's background color _(hex color)_ **or** a gradient in the form of _angle,start,end_. Default: `fffefe`
 -   `hide_border` - Hides the card's border _(boolean)_. Default: `false`
--   `theme` - name of the theme, choose from [all available themes](./themes/README.md). Default: `default` theme. 
+-   `theme` - name of the theme, choose from [all available themes](./themes/README.md). Default: `default` theme.
 -   `cache_seconds` - set the cache header manually _(min: 7200, max: 86400)_. Default: `14400 seconds (4 hours)`.
 -   `locale` - set the language in the card _(e.g. cn, de, es, etc.)_. Default: `en`.
 -   `border_radius` - Corner rounding on the card. Default: `4.5`.
 
 > **Warning**
-> We use caching to decrease the load on our servers (see https://github.com/anuraghazra/github-readme-stats/issues/1471#issuecomment-1271551425). Our cards have a default cache of 4 hours (14400 seconds). Also, note that the cache is clamped to a minimum of 4 hours and a maximum of 24 hours.
+> We use caching to decrease the load on our servers (see <https://github.com/anuraghazra/github-readme-stats/issues/1471#issuecomment-1271551425>). Our cards have a default cache of 4 hours (14400 seconds). Also, note that the cache is clamped to a minimum of 4 hours and a maximum of 24 hours.
 
 ##### Gradient in bg_color
 
@@ -261,7 +267,7 @@ Use [show_owner](#customization) variable to include the repo's owner username
 The top languages card shows a GitHub user's most frequently used top language.
 
 > **Note**
-> Top Languages does not indicate my skill level or anything like that; it's a GitHub metric to determine which languages have the most code on GitHub. It is a new feature of github-readme-stats._
+> Top Languages does not indicate my skill level or anything like that; it's a GitHub metric to determine which languages have the most code on GitHub. It is a new feature of github-readme-stats.\_
 
 ### Usage
 
@@ -405,17 +411,14 @@ By default, GitHub does not lay out the cards side by side. To do that, you can
 
 ## Deploy on your own Vercel instance
 
-#### [Check Out Step By Step Video Tutorial By @codeSTACKr](https://youtu.be/n6d4KHSKqGk?t=107)
+As explained [above](#important-notice) the GitHub API only [allows 5k requests per hour per user account](https://docs.github.com/en/graphql/overview/resource-limitations), the public Vercel instance hosted on `https://github-readme-stats.vercel.app/api` could possibly hit the rate limiter (see #1471). Because of this, we have limited the public API to only fetch the first 100 repositories with stars and languages. As a result, for a very active GitHub account, the language, stars and commits results might be off when using the public API. These limits do not apply when you deploy [your own Vercel instance](#deploy-on-your-own-vercel-instance), so in that case, you do not have to worry about anything! Click on the deploy button and follow the video or step-to-step guides below to get started! :rocket:
+
+[![Deploy to Vercel](https://vercel.com/button)](https://vercel.com/import/project?template=https://github.com/anuraghazra/github-readme-stats)
 
 > **Warning**
 > If you are on the [hobby (i.e. free)](https://vercel.com/pricing) Vercel plan, please make sure you change the `maxDuration` parameter in the [vercel.json](https://github.com/anuraghazra/github-readme-stats/blob/master/vercel.json) file from `30` to `10` (see [#1416](https://github.com/anuraghazra/github-readme-stats/issues/1416#issuecomment-950275476) for more information).
 
-Since the GitHub API only allows 5k requests per hour, my `https://github-readme-stats.vercel.app/api` could possibly hit the rate limiter. If you host it on your own Vercel server, then you do not have to worry about anything. Click on the deploy button to get started!
-
-> **Note**
-> Since [#58](https://github.com/anuraghazra/github-readme-stats/pull/58), we should be able to handle more than 5k requests and have fewer issues with downtime :grin:.
-
-[![Deploy to Vercel](https://vercel.com/button)](https://vercel.com/import/project?template=https://github.com/anuraghazra/github-readme-stats)
+#### :film_projector: [Check Out Step By Step Video Tutorial By @codeSTACKr](https://youtu.be/n6d4KHSKqGk?t=107)
 
 <details>
  <summary><b>:hammer_and_wrench: Step-by-step guide on setting up your own Vercel instance</b></summary>

diff --git a/src/fetchers/stats-fetcher.js b/src/fetchers/stats-fetcher.js
@@ -168,10 +168,14 @@ const totalStarsFetcher = async (username, repoToHide) => {
       (node) => node.stargazers.totalCount !== 0,
     );
     nodes.push(...nodesWithStars);
-    // hasNextPage =
-    //   allNodes.length === nodesWithStars.length &&
-    //   res.data.data.user.repositories.pageInfo.hasNextPage;
-    hasNextPage = false; // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+
+    // Disable multi page fetching on public Vercel instance due to rate limits.
+    hasNextPage =
+      process.env.FETCH_SINGLE_PAGE_STARS === "true"
+        ? false
+        : allNodes.length === nodesWithStars.length &&
+          res.data.data.user.repositories.pageInfo.hasNextPage;
+
     endCursor = res.data.data.user.repositories.pageInfo.endCursor;
   }
 

diff --git a/tests/fetchStats.test.js b/tests/fetchStats.test.js
@@ -4,6 +4,7 @@ import MockAdapter from "axios-mock-adapter";
 import { calculateRank } from "../src/calculateRank.js";
 import { fetchStats } from "../src/fetchers/stats-fetcher.js";
 
+// Test parameters.
 const data = {
   data: {
     user: {
@@ -93,13 +94,14 @@ const error = {
 const mock = new MockAdapter(axios);
 
 beforeEach(() => {
+  process.env.FETCH_SINGLE_PAGE_STARS = "true"; // Set to true to fetch only one page of stars.
   mock
     .onPost("https://api.github.com/graphql")
     .replyOnce(200, data)
     .onPost("https://api.github.com/graphql")
-    .replyOnce(200, firstRepositoriesData);
-  // .onPost("https://api.github.com/graphql") // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-  // .replyOnce(200, secondRepositoriesData); // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+    .replyOnce(200, firstRepositoriesData)
+    .onPost("https://api.github.com/graphql")
+    .replyOnce(200, secondRepositoriesData);
 });
 
 afterEach(() => {
@@ -114,8 +116,7 @@ describe("Test fetchStats", () => {
       totalRepos: 5,
       followers: 100,
       contributions: 61,
-      // stargazers: 400, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      stargazers: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      stargazers: 300,
       prs: 300,
       issues: 200,
     });
@@ -126,8 +127,7 @@ describe("Test fetchStats", () => {
       totalCommits: 100,
       totalIssues: 200,
       totalPRs: 300,
-      // totalStars: 400, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      totalStars: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      totalStars: 300,
       rank,
     });
   });
@@ -178,8 +178,7 @@ describe("Test fetchStats", () => {
       totalRepos: 5,
       followers: 100,
       contributions: 61,
-      // stargazers: 400, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      stargazers: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      stargazers: 300,
       prs: 300,
       issues: 200,
     });
@@ -190,8 +189,7 @@ describe("Test fetchStats", () => {
       totalCommits: 150,
       totalIssues: 200,
       totalPRs: 300,
-      // totalStars: 400, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      totalStars: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      totalStars: 300,
       rank,
     });
   });
@@ -207,8 +205,7 @@ describe("Test fetchStats", () => {
       totalRepos: 5,
       followers: 100,
       contributions: 61,
-      // stargazers: 400, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      stargazers: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      stargazers: 300,
       prs: 300,
       issues: 200,
     });
@@ -219,8 +216,7 @@ describe("Test fetchStats", () => {
       totalCommits: 1050,
       totalIssues: 200,
       totalPRs: 300,
-      // totalStars: 400, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      totalStars: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      totalStars: 300,
       rank,
     });
   });
@@ -236,8 +232,7 @@ describe("Test fetchStats", () => {
       totalRepos: 5,
       followers: 100,
       contributions: 61,
-      // stargazers: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      stargazers: 200, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      stargazers: 200,
       prs: 300,
       issues: 200,
     });
@@ -248,8 +243,57 @@ describe("Test fetchStats", () => {
       totalCommits: 1050,
       totalIssues: 200,
       totalPRs: 300,
-      // totalStars: 300, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
-      totalStars: 200, // NOTE: Temporarily disable fetching of multiple pages. Done because of #2130.
+      totalStars: 200,
+      rank,
+    });
+  });
+
+  it("should fetch two pages of stars if 'FETCH_SINGLE_PAGE_STARS' env variable is not defined", async () => {
+    process.env.FETCH_SINGLE_PAGE_STARS = undefined;
+
+    let stats = await fetchStats("anuraghazra");
+    const rank = calculateRank({
+      totalCommits: 100,
+      totalRepos: 5,
+      followers: 100,
+      contributions: 61,
+      stargazers: 400,
+      prs: 300,
+      issues: 200,
+    });
+
+    expect(stats).toStrictEqual({
+      contributedTo: 61,
+      name: "Anurag Hazra",
+      totalCommits: 100,
+      totalIssues: 200,
+      totalPRs: 300,
+      totalStars: 400,
+      rank,
+    });
+  });
+
+  it("should fetch two pages of stars if 'FETCH_SINGLE_PAGE_STARS' env variable is set to `false`", async () => {
+    process.env.FETCH_SINGLE_PAGE_STARS = "false";
+
+    let stats = await fetchStats("anuraghazra");
+    const rank = calculateRank({
+      totalCommits: 100,
+      totalRepos: 5,
+      followers: 100,
+      contributions: 61,
+      stargazers: 400,
+      prs: 300,
+      issues: 200,
+    });
+
+    expect(stats).toStrictEqual({
+      contributedTo: 61,
+      name: "Anurag Hazra",
+      totalCommits: 100,
+      totalIssues: 200,
+      totalPRs: 300,
+      totalStars: 400,
       rank,
     });
   });