Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tree): UpdateProof fixes #129

Merged
merged 2 commits into from
Sep 23, 2024
Merged

Conversation

distractedm1nd
Copy link
Contributor

@distractedm1nd distractedm1nd commented Sep 23, 2024

This PR reduces sp1 cycle count by ~10x by only validating the last signature in the hashchain.
To do this, we need to add some extra validation checks inside UpdateProof so that we can make the safe assumption that the existing value has been validated in a previous epoch.

New cycle counts:

 Instruction counts considering call graph
+------------------------------------------------------------------------------------------------------------------------+-------------------+
| Function Name                                                                                                          | Instruction Count |
| __start                                                                                                                | 34377752          |
| main                                                                                                                   | 34376876          |
| prism_common::hashchain::Hashchain::verify_last_entry                                                                  | 21666136          |
| prism_common::hashchain::Hashchain::verify_signature                                                                   | 21232162          |
| ed25519_dalek::verifying::VerifyingKey::verify                                                                         | 20963136          |
| prism_common::tree::UpdateProof::verify                                                                                | 20573203          |
| curve25519_dalek::edwards::EdwardsPoint::compress                                                                      | 15467300          |
| curve25519_dalek::field::<impl curve25519_dalek::backend::serial::u32::field::FieldElement2625>::pow22501              | 14800000          |
| curve25519_dalek::backend::serial::u32::field::FieldElement2625::pow2k                                                 | 13786600          |
| curve25519_dalek::backend::serial::u32::field::FieldElement2625::square_inner                                          | 10414000          |
| prism_common::tree::InsertProof::verify                                                                                | 8911014           |
| &mut bincode::de::Deserializer::deserialize_struct                                                                     | 4777238           |
| serde::de::impls::<impl serde::de::Deserialize for alloc::vec::Vec::visit_seq                                          | 4774739           |
| prism_common::tree::_::<impl serde::de::Deserialize for prism_common::tree::Proof>::deserialize::__Visitor::visit_enum | 4717067           |
| &mut bincode::de::Deserializer::deserialize_struct                                                                     | 3427356           |
| jmt::types::proof::definition::UpdateMerkleProof::verify_update                                                        | 3367122           |
| &mut bincode::de::Deserializer::deserialize_tuple                                                                      | 2960964           |
| jmt::types::proof::definition::SparseMerkleProof::root_hash                                                            | 2910400           |
| &mut bincode::de::Deserializer::deserialize_struct                                                                     | 2869669           |
| jmt::types::proof::definition::SparseMerkleProof::verify                                                               | 2660353           |
| memcpy                                                                                                                 | 2638756           |
| sha2::sha256::compress256                                                                                              | 2630202           |
| sha2::sha512::compress512                                                                                              | 2442200           |
| serde::de::impls::<impl serde::de::Deserialize for alloc::vec::Vec::visit_seq                                          | 2326699           |
| curve25519_dalek::edwards::EdwardsPoint::vartime_double_scalar_mul_basepoint                                           | 2273692           |
| curve25519_dalek::backend::vartime_double_base_mul                                                                     | 2272992           |
| jmt::types::proof::_::<impl serde::de::Deserialize for                                                                 | 2039808           |
| jmt::types::proof::SparseMerkleNode>::deserialize::__Visitor::visit_enum                                               |                   |
| prism_common::tree::Hasher::finalize                                                                                   | 1894267           |
| serde::de::SeqAccess::next_element                                                                                     | 1811808           |
| jmt::types::proof::SparseMerkleNode::hash                                                                              | 1717735           |
+------------------------------------------------------------------------------------------------------------------------+-------------------+

Summary by CodeRabbit

  • New Features

    • Enhanced verification process for hashchain updates, focusing on the last entry and inclusion proof for the old value.
    • Introduced a method for inserting new entries into the hash chain without validation.
  • Bug Fixes

    • Improved memory management by changing return types to references for public key handling.
  • Chores

    • Refined test utilities for hashchain handling, ensuring updates focus on the last element and incorporate mock signing keys.

Copy link

vercel bot commented Sep 23, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
prism ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 23, 2024 11:05am

Copy link
Contributor

coderabbitai bot commented Sep 23, 2024

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Files selected but had no reviewable changes (1)
  • elf/riscv32im-succinct-zkvm-elf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The changes across multiple files primarily focus on refining the hashchain verification process and updating related methods. Key modifications include renaming the validate method to verify_last_entry, adjusting return types to improve memory management, and enhancing the UpdateProof structure for better verification of hashchain updates. Additionally, test utilities have been updated to incorporate mock signing keys, ensuring a more streamlined approach to handling hashchains.

Changes

Files Change Summary
crates/common/src/hashchain.rs Renamed validate to verify_last_entry, integrated logic from validate_entry, and modified get_key_at_index return type. Added insert_unsafe method for unvalidated entry insertion.
crates/common/src/operation.rs Changed get_public_key return type from Option<PublicKey> to Option<&PublicKey>, allowing for reference returns instead of cloning.
crates/common/src/test_utils.rs Updated update and insert methods to focus on the last element of the hashchain and introduced mock signing keys in hashchain creation.
crates/common/src/tree.rs Modified UpdateProof struct to include old_value and new_entry, updated verification logic in verify method, and adjusted update method signatures to accept HashchainEntry instead of Hashchain.
crates/nova/src/update.rs Changed access to update_proof from self.update_proof.proof.proofs()[0] to self.update_proof.update_proof.proofs()[0], reflecting a structural change in the update_proof object.

Possibly related PRs

Poem

In the world of chains so fine,
A rabbit hops, with code to shine.
"Verify the last," it sings with glee,
"Mock keys and updates, all in harmony! 🐇✨
With every change, we leap and bound,
In the land of code, new joys are found!"


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    -- I pushed a fix in commit <commit_id>, please review it.
    -- Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    -- @coderabbitai generate unit testing code for this file.
    -- @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    -- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    -- @coderabbitai read src/utils.ts and generate unit testing code.
    -- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    -- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Outside diff range comments (5)
crates/nova/src/update.rs (2)

Line range hint 77-79: Consider addressing the TODO comment

There's a TODO comment indicating that the current implementation uses an inclusion proof of the node before the update, but it should create a new Merkle proof by hashing the new node. This might be an opportunity for further optimization or correctness improvement.

Would you like assistance in implementing this TODO? I can help draft the code for creating and verifying the new Merkle proof.


Line range hint 92-94: Improve error handling for update_proof.verify()

The verify method is called on update_proof without any specific error handling. Consider adding more detailed error handling to provide better diagnostics in case of verification failure.

Consider replacing:

self.update_proof
    .verify()
    .map_err(|_| SynthesisError::Unsatisfiable)?;

with:

self.update_proof
    .verify()
    .map_err(|e| {
        log::error!("UpdateProof verification failed: {:?}", e);
        SynthesisError::Unsatisfiable
    })?;

This will log the specific error before converting it to a SynthesisError, aiding in debugging if issues arise.

crates/common/src/operation.rs (3)

Line range hint 87-96: Consider optimizing get_signature_bundle method

The get_signature_bundle method currently clones the signature bundle. Consider returning a reference to avoid unnecessary cloning, similar to the optimization made in get_public_key.

Here's a suggested implementation:

pub fn get_signature_bundle(&self) -> Option<&SignatureBundle> {
    match self {
        Operation::AddKey(args) | Operation::RevokeKey(args) => Some(&args.signature),
        Operation::CreateAccount(args) => None, // or create a static SignatureBundle if needed
    }
}

Note that this change would require adjusting the return type and potentially updating the callers of this method.


Line range hint 98-111: Review serialization approach in insert_signature method

The insert_signature method serializes the entire operation before signing. For large operations, this could be inefficient.

Consider serializing only the essential parts of the operation that need to be signed, rather than the entire struct. This would require defining a canonical representation of the operation for signing purposes.

Example approach:

fn serialize_for_signing(&self) -> Result<Vec<u8>> {
    // Implement a custom serialization that includes only the necessary fields
    // This could be a subset of the full serialization
}

pub fn insert_signature(&mut self, signing_key: &SigningKey) -> Result<()> {
    let serialized = self.serialize_for_signing()?;
    let signature = signing_key.sign(&serialized);
    // ... rest of the method
}

Line range hint 139-180: Enhance validation in the validate method

The current validate method performs basic checks, but it could be more comprehensive.

Consider adding the following enhancements:

  1. Validate the format and length of the id field.
  2. Check if the PublicKey is in a valid format.
  3. Verify the length and format of signatures.
  4. Implement the commented-out challenge validation for CreateAccount.

Example:

pub fn validate(&self) -> Result<()> {
    match &self {
        Operation::AddKey(args) | Operation::RevokeKey(args) => {
            validate_id(&args.id)?;
            validate_public_key(&args.value)?;
            validate_signature(&args.signature.signature)?;
            Ok(())
        }
        Operation::CreateAccount(args) => {
            validate_id(&args.id)?;
            validate_public_key(&args.value)?;
            validate_signature(&args.signature)?;
            validate_challenge(&args.challenge)?;
            Ok(())
        }
    }
}

fn validate_id(id: &str) -> Result<()> {
    // Implement id validation logic
}

fn validate_public_key(key: &PublicKey) -> Result<()> {
    // Implement public key validation logic
}

fn validate_signature(signature: &[u8]) -> Result<()> {
    // Implement signature validation logic
}

fn validate_challenge(challenge: &ServiceChallengeInput) -> Result<()> {
    // Implement challenge validation logic
}
Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE

Commits

Files that changed from the base of the PR and between 9bfcaf6 and fdaf3fc.

Files selected for processing (5)
  • crates/common/src/hashchain.rs (2 hunks)
  • crates/common/src/operation.rs (1 hunks)
  • crates/common/src/test_utils.rs (3 hunks)
  • crates/common/src/tree.rs (6 hunks)
  • crates/nova/src/update.rs (1 hunks)
Additional comments not posted (13)
crates/nova/src/update.rs (2)

Line range hint 1-104: Summary of review

  1. The main change (line 83) aligns with the PR objectives of optimizing the UpdateProof function.
  2. There's a TODO comment that might be worth addressing for further optimization or correctness.
  3. Error handling for the update_proof.verify() call could be improved.

Overall, the changes look good and align with the PR objectives. The suggested improvements are not blocking issues but could enhance the code quality and maintainability.


83-83: LGTM! Verify consistency across the codebase.

The change from self.update_proof.proof.proofs()[0] to self.update_proof.update_proof.proofs()[0] appears to be in line with the PR objectives of modifying the UpdateProof function for performance improvements. This suggests a restructuring of the UpdateProof struct.

To ensure consistency across the codebase, please run the following script:

This will help identify if all instances have been updated consistently and if there are any other occurrences of the new pattern.

Verification successful

Consistency Verified Across Codebase.

The change from self.update_proof.proof.proofs()[0] to self.update_proof.update_proof.proofs()[0] has been consistently applied. The old access pattern .proof.proofs() is no longer present, and the new pattern exists only in crates/nova/src/update.rs at line 83 as intended.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any remaining instances of `.proof.proofs()` and new instances of `.update_proof.proofs()`.

echo "Checking for old access pattern:"
rg --type rust 'update_proof\.proof\.proofs\(\)'

echo "Checking for new access pattern:"
rg --type rust 'update_proof\.update_proof\.proofs\(\)'

Length of output: 356

crates/common/src/operation.rs (2)

Line range hint 1-214: Summary of review for crates/common/src/operation.rs

The main change to get_public_key is a good optimization that aligns with the PR's performance improvement goals. However, the changes in this file seem to focus more on memory efficiency than on the specific UpdateProof function mentioned in the PR summary.

Additional suggestions have been made to further optimize the code:

  1. Optimize the get_signature_bundle method to avoid cloning.
  2. Review the serialization approach in the insert_signature method for potential performance improvements.
  3. Enhance the validation logic in the validate method for more comprehensive checks.

These suggestions, if implemented, could contribute to the overall performance improvements targeted by this PR.


80-84: Approve the change with a suggestion for further verification

The modification to get_public_key method is a good optimization. By returning a reference instead of cloning the PublicKey, it improves performance and memory efficiency.

However, this change modifies the method's API, which might affect other parts of the codebase. Please run the following script to verify that all usages of get_public_key have been updated accordingly:

Review the output to ensure all usages are compatible with the new return type.

Verification successful

Verification of get_public_key usage is successful

No incompatible usages of get_public_key found in the codebase. All instances correctly handle the returned reference.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Find all usages of get_public_key and verify they handle the new return type correctly

# Search for get_public_key usages
rg --type rust "\.get_public_key\(\)" -C 3

# Look for potential issues with ownership or borrowing related to get_public_key
rg --type rust "let .+ = .+\.get_public_key\(\)" -C 3

Length of output: 1114

crates/common/src/test_utils.rs (2)

Line range hint 195-214: Approved addition of create_mock_hashchain function

The create_mock_hashchain function is a valuable addition that supports the changes made in create_random_insert. It provides a way to create a hashchain with a valid initial state, using the provided signing key to sign the CreateAccount operation.

This implementation aligns well with the PR objectives by enabling the creation of more realistic test data, which is crucial for thorough testing of the UpdateProof functionality.

The function is well-structured and correctly implements the creation of a mock hashchain. It's a good addition to the test utilities.


Line range hint 1-248: Overall assessment: Changes align well with PR objectives

The modifications to this file significantly enhance the test utilities to support the new UpdateProof functionality. Key improvements include:

  1. Consistent use of the last entry of the hashchain for updates, aligning with the PR's performance optimization goals.
  2. Introduction of more realistic test data through the use of actual signing keys and mock hashchains.
  3. Enhanced support for testing the UpdateProof functionality with more comprehensive test scenarios.

These changes collectively contribute to a more robust testing environment, which is crucial for validating the reported performance improvements in the UpdateProof function.

While the changes are generally well-implemented, consider addressing the minor suggestions for error handling and code structure in a follow-up commit to further improve the robustness and maintainability of the test utilities.

crates/common/src/hashchain.rs (1)

136-140: LGTM!

Returning a reference to PublicKey in get_key_at_index efficiently avoids unnecessary cloning.

crates/common/src/tree.rs (6)

14-14: Import HashchainEntry to support new functionality.

The addition of HashchainEntry to the import statement ensures that the struct is available for use in the updated methods.


208-208: Added verification of the last hashchain entry in InsertProof::verify.

Including self.value.verify_last_entry()?; enhances the security by ensuring that the last entry in the hashchain is valid before proceeding.


220-226: Updated UpdateProof struct with necessary fields for verification.

Adding old_value, new_entry, inclusion_proof, and update_proof to the UpdateProof struct provides all the required data to perform a comprehensive verification of updates to the hashchain.


231-243: Verification logic in UpdateProof::verify is correctly implemented.

The method properly verifies the existence of the old value, appends the new entry, validates the last entry of the updated hashchain, and ensures that the update proof corresponds to the new state.


340-343: Operation processing updated to utilize HashchainEntry.

The generation of a new HashchainEntry using perform_operation and passing it to the update method aligns with the new update logic and improves performance by focusing on the last entry.


Line range hint 430-446: update method correctly updates the hashchain and tree structure.

The implementation correctly serializes the new hashchain, updates the tree with the new value, and constructs the UpdateProof with all necessary components, ensuring the integrity of the update process.

crates/common/src/test_utils.rs Show resolved Hide resolved
crates/common/src/test_utils.rs Show resolved Hide resolved
crates/common/src/test_utils.rs Show resolved Hide resolved
crates/common/src/hashchain.rs Show resolved Hide resolved
crates/common/src/hashchain.rs Show resolved Hide resolved
crates/common/src/hashchain.rs Show resolved Hide resolved
crates/common/src/tree.rs Show resolved Hide resolved
crates/common/src/tree.rs Show resolved Hide resolved
Copy link
Contributor

@sebasti810 sebasti810 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@distractedm1nd distractedm1nd merged commit b358e65 into main Sep 23, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants