chore: Use cargo-chef to build the hivetests docker image

[janjakubnanista]

Ref #14777

The docker build for the hive tests is now a bit lacking - it builds the binary outside the container, then just copies it in. Besides fragility, this also means longer buildtimes since the build is not utilizing cargo-chef.

This PR modifies the prepare-reth workflow and the hive tests Dockerfile to improve the speed and stability of the CI builds:

• Update .github/assets/hive/Dockerfile to use cargo-chef
• Exclude .git and dist from being copied along with the source code where not needed - this effectively breaks the docker layer caching. The --exclude flag is still in stable Dockerfile syntax and needs to be enabled. See the docs
• Drop the local rust cache since we're now building in docker

The caching has been tested in the forked repository (on ubuntu-latest so the times are not directly translatable):

• The initial build took ~7mins
• The subsequent build took ~37secs

The runs were performed on debug branch that only differs from main in the fact that it uses ubuntu-latest and removes the condition if: github.repository == 'paradigmxyz/reth'

[fgimenez]

@janjakubnanista thanks for your contribution! improving speed and stability sounds very good :)

i've forked this branch to a branch in this repo to execute the hive workflow with these changes. i get this error: https://github.com/paradigmxyz/reth/actions/runs/13705636252/job/38330111508. can you please take a look?

[janjakubnanista]

Hey @fgimenez! Thanks for that - I debugged the issue in my fork and all should be good now, already for a review. The cached runs seem super fast (see PR description)

[emhane]

amazing

[janjakubnanista]

@emhane or @fgimenez - i see some lints failed, some vulnerabilities in the lockfile - suspicious since i have not touched the lockfile. Any ideas?

[emhane]

@emhane or @fgimenez - i see some lints failed, some vulnerabilities in the lockfile - suspicious since i have not touched the lockfile. Any ideas?

merge in latest main might fix it

[mattsse]

ty!

pending @fgimenez

failing ci unrelated #14889

[fgimenez]

@janjakubnanista thx! the build succeeds now, but it takes much longer than before, this build is with the changes in this branch https://github.com/paradigmxyz/reth/actions/runs/13717164093, building the hivetests image took 19m23s, and this is the last regular build executed last night https://github.com/paradigmxyz/reth/actions/runs/13710819568 building the image took 8m46s, is this expected?

[emhane]

@janjakubnanista thx! the build succeeds now, but it takes much longer than before, this build is with the changes in this branch https://github.com/paradigmxyz/reth/actions/runs/13717164093, building the hivetests image took 19m23s, and this is the last regular build executed last night https://github.com/paradigmxyz/reth/actions/runs/13710819568 building the image took 8m46s, is this expected?

could this be because we are building loads of other branches in-between the hive tests, that don't make use of cargo chef? probs right way to test is running several hive jobs right after each other on that branch @fgimenez

[emhane]

yepp, just re-ran hive and reth built in 37s !! https://github.com/paradigmxyz/reth/actions/runs/13718282056

[fgimenez]

yepp, just re-ran hive and reth built in 37s !! https://github.com/paradigmxyz/reth/actions/runs/13718282056

awesome! maybe the first time it needs to build the cache. makes me wonder, if we run again from main, after some commits have been merged, will it need to rebuild the cache?

we can test @janjakubnanista if you rebase on top of current main or merge main CI would pass, and we can retrigger hive from a different commit

[fgimenez]

we can test @janjakubnanista if you rebase on top of current main or merge main CI would pass, and we can retrigger hive from a different commit

I've rebased the branch forked from this one on top of current main and run hive twice, the first one building the image took 23m https://github.com/paradigmxyz/reth/actions/runs/13726539897 and the second one 39s https://github.com/paradigmxyz/reth/actions/runs/13727275178

[mattsse]

failing lint tests are unrelated, good to merge?

[janjakubnanista]

@mattsse @emhane should i rebase on the newest main you think? to get rid of those red lints