Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add commmand for pruning states #4648

Closed
wants to merge 5 commits into from
Closed

Add commmand for pruning states #4648

wants to merge 5 commits into from

Conversation

michaelsproul
Copy link
Member

@michaelsproul michaelsproul commented Aug 22, 2023 •
edited
Loading

Issue Addressed

Closes #4481

Proposed Changes

  • Add lighthouse db prune-states
  • Make it work
  • Ensure block roots are handled correctly
  • Check perf on mainnet/Goerli/Gnosis (takes a few seconds max)
  • Add some tests
  • Update docs
  • Add --freezer flag and other improvements to lighthouse db inspect

@michaelsproul michaelsproul added work-in-progress PR is a work-in-progress database v4.5.0 ETA Q4 2023 labels Aug 22, 2023
@michaelsproul
Copy link
Member Author

michaelsproul commented Aug 24, 2023 •
edited
Loading

Just noting this here for later:

Fatal error: VectorChunkError(InconsistentGenesisValue { field: BeaconRandaoMixes, existing_value: "0x5ac670562dbf877a45039d65ec3c2e3402a40eda9b1dba931c2376ab7d0927c2", new_value: "0xeeea1373d4aa9e099d7c9deddb694db9aeb4577755ef83f9b6345ce4357d9abf" })

Just realising this was probably because I ran on Goerli but forgot --network goerli. Should definitely add a safeguard for this. Still seems a bit odd, because I would expect the previous value to be completely wiped.

@michaelsproul
Copy link
Member Author

Previous bug with InconsistentGenesisValue is doubly-fixed by:

  • Comparing network genesis state to genesis state from DB (only in CLI tool)
  • Properly deleting the 8-byte 0x00 keys in the chunked vector columns. There was a bug where the iteration was starting from 32-byte 0x00 key which was lexicographically greater than 8-bytes of zeroes, resulting in it being skipped.

@michaelsproul michaelsproul mentioned this pull request Sep 1, 2023
Open
@paulhauner paulhauner added v4.6.0 ETA Q1 2024 and removed v4.5.0 ETA Q4 2023 labels Sep 20, 2023
@jimmygchen jimmygchen mentioned this pull request Oct 12, 2023
Closed
8 tasks
@jimmygchen
Copy link
Member

Continuing this work on #4835

@jimmygchen jimmygchen closed this Oct 12, 2023
bors bot pushed a commit that referenced this pull request Nov 3, 2023
@jimmygchen @michaelsproul
Add commmand for pruning states (#4835)
36d8849 
## Issue Addressed

Closes #4481. 

(Continuation of #4648)

## Proposed Changes

- [x] Add `lighthouse db prune-states`
- [x] Make it work
- [x] Ensure block roots are handled correctly (to be addressed in 4735)
- [x] Check perf on mainnet/Goerli/Gnosis (takes a few seconds max)
- [x] Run block root healing logic (#4875 ) at the beginning
- [x] Add some tests
- [x] Update docs
- [x] Add `--freezer` flag and other improvements to `lighthouse db inspect`

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
Co-authored-by: Michael Sproul <micsproul@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
database v4.6.0 ETA Q1 2024 work-in-progress PR is a work-in-progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@michaelsproul @jimmygchen @paulhauner