respect shallow/grafted nixpkgs repositories

[xfnw]

if you attempt to nixpkgs-review pr on a shallow copy of nixpkgs, it will try to download the entire ~4gb of git history

[Mic92]

I don't see a way around this. It currently uses git fetch to download the history of both master and the PR and than needs to perform a merge of one into the other

[SuperSandro2000]

it will try to download the entire ~4gb of git history

For me the entire history is just 1.3 GB big.

We could probably modify the fetch commands to only fetch what we really need but it is a pain to work with shallow repositories since certain things done with the history will just fail in some situations and executing multiple fetches for every user is probably slowing down things more.

[xfnw]

For me the entire history is just 1.3 GB big.

huh... do you have some sort of compression enabled? mine seems to be a lot larger

[xfnw@feesh:~]$ du -sh nixpkgs
3.7G	nixpkgs

[xfnw@feesh:~]$ du -sh nixpkgs/.git
3.6G	nixpkgs/.git

[SuperSandro2000]

Try

du -sh .git
git gc --aggressive
git gc
git repack -Ad
git prune
du -sh .git

[lolbinarycat]

@SuperSandro2000

We could probably modify the fetch commands to only fetch what we really need but it is a pain to work with shallow repositories since certain things done with the history will just fail in some situations and executing multiple fetches for every user is probably slowing down things more.

what about an arbitrarily large but still much smaller than the whole repository amount?

for example, fetching only the last 10000 commits?

waiting 10 minutes is still a lot better than waiting 2 hours, and people using a shallow clone are likely to have a a slow network connection anyways.

[azuwis]

Github offers a ref named pull/<PULL_REQUEST_NUMBER>/merge, point to the merge commit of the pull request and the current pull request base branch (current master, staging, staging-next, etc.).

I believe nixpkgs-review can make use of pull/<PULL_REQUEST_NUMBER>/merge, instead of doing the merge itself.

$ git init nixpkgs
$ cd nixpkgs
$ git fetch --depth=1 https://github.com/NixOS/nixpkgs pull/350787/merge
$ git checkout FETCH_HEAD
$ git log  
commit 154a1db8ff36a8c05ff0c2fd432b1a5b5f3f3e8c (grafted, HEAD)
Author: Weijia Wang <9713184+wegank@users.noreply.github.com>
Date:   Fri Oct 25 16:44:48 2024 +0800

    Merge acae4128d4fcc1f2d81e2acf3af50d24a6592758 into 31eafaeb6d1e29571e628c6855e68dc7cf5fb0a0
$ git ls-remote https://github.com/NixOS/nixpkgs heads/master
31eafaeb6d1e29571e628c6855e68dc7cf5fb0a0	refs/heads/master

https://github.com/actions/checkout uses this ref for pull request, https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_request

[azuwis]

Something like:

nixpkgs-review pr --checkout commit 350787 ->

$ git fetch --depth=1 https://github.com/NixOS/nixpkgs pull/350787/head:refs/nixpkgs-review/0
$ git worktree add /home/azuwis/.cache/nixpkgs-review/pr-350787-1/nixpkgs refs/nixpkgs-review/0

nixpkgs-review pr --checkout merge 350787 ->

$ git fetch --depth=1 https://github.com/NixOS/nixpkgs pull/350787/merge:refs/nixpkgs-review/1
$ git worktree add /home/azuwis/.cache/nixpkgs-review/pr-350787-2/nixpkgs refs/nixpkgs-review/1