Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update honk ultra_recursive_verifier to do aggregation #7582

Merged
merged 43 commits into from
Aug 13, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
d024c54
honk recursion constraint tests in dsl_tests pass
lucasxia01 Jul 12, 2024
8a2a33e
fix verify_honk_proof inputs to take in agg obj
lucasxia01 Jul 15, 2024
47d2806
Merge branch 'master' into lx/reinstate-aggregation
lucasxia01 Jul 15, 2024
83cfed3
change to 409 in rollup, renable agg obj in a couple ts files
lucasxia01 Jul 15, 2024
dd7daea
Merge branch 'lx/reinstate-aggregation' of github.com:AztecProtocol/a…
lucasxia01 Jul 15, 2024
ae4ccd7
forgot to fix the tube function
lucasxia01 Jul 15, 2024
2153d8b
bb prover fix
lucasxia01 Jul 18, 2024
f67f529
added fake agg obj to tube circuit to pass full test
lucasxia01 Jul 22, 2024
ab0b2bb
initialize interface change
lucasxia01 Jul 23, 2024
c3be434
updated function interfaces, bb compilies
lucasxia01 Jul 23, 2024
010062b
remove public_inputs and proof_witness_indices fields of aggregation_…
lucasxia01 Jul 25, 2024
25df38f
Merge remote-tracking branch 'origin/master' into lx/update-honk-agg-…
lucasxia01 Jul 29, 2024
e1116ce
Merge remote-tracking branch 'origin/master' into lx/update-honk-agg-…
lucasxia01 Jul 29, 2024
ea47e7a
fix weird merge changes
lucasxia01 Jul 29, 2024
2c07750
redo something merge erased
lucasxia01 Jul 29, 2024
ff25f3c
fix write_vk flow to have default agg obj for dummy key
lucasxia01 Jul 29, 2024
70c414d
do aggregation in ultra_recursive_verifier
lucasxia01 Jul 30, 2024
f074801
enabling/refactoring honk_recursion_constraint to use agg objs
lucasxia01 Jul 30, 2024
603ecd1
fixed write_vk flow to correctly set the vk recursive_proof_public_in…
lucasxia01 Jul 30, 2024
11231d3
refactor to use shared functionality in aggregation_state.hpp
lucasxia01 Jul 30, 2024
20f254f
update tests to pass in a default or handle an agg obj
lucasxia01 Jul 30, 2024
5c626b1
added a normalize call and fixed constructing a biggroup in aggregati…
lucasxia01 Jul 31, 2024
4f2a80b
just wildly add aggregation objects everywhere
lucasxia01 Jul 31, 2024
6ce1558
add simulator function to agg state
lucasxia01 Aug 1, 2024
674bcca
try increasing some client ivc gate numbers
lucasxia01 Aug 1, 2024
9c4e62b
clean up PR
lucasxia01 Aug 2, 2024
553fb2d
fix tests
lucasxia01 Aug 2, 2024
aed087e
fixed problems?
lucasxia01 Aug 6, 2024
5b9920f
try reverting change to ultra rec verifier
lucasxia01 Aug 7, 2024
02e17ac
undo moving add_recursive_proof change
lucasxia01 Aug 8, 2024
3305c90
oops, fix to previous commit
lucasxia01 Aug 8, 2024
32a11ac
try running CI with debug statements
lucasxia01 Aug 9, 2024
3ff3fb2
try a fix for prove_tube (there's two agg objs)
lucasxia01 Aug 12, 2024
e9479a5
Merge remote-tracking branch 'origin/master' into lx/update-honk-agg-…
lucasxia01 Aug 12, 2024
fe46e5f
add agg obj to aztec ivc
lucasxia01 Aug 12, 2024
cea4738
remove extra agg object in client ivc test
lucasxia01 Aug 12, 2024
156aba5
just throw an error when you try to add multiple agg objs
lucasxia01 Aug 12, 2024
4c50c74
refactor honk recursion constraint (moved dummy stuff to a function)
lucasxia01 Aug 12, 2024
45f0329
added assert_is_in_field to agg_obj function to properly constrain it
lucasxia01 Aug 12, 2024
37c8392
undo full test change
lucasxia01 Aug 12, 2024
9245975
removing some print statements DISABLE_CI
lucasxia01 Aug 12, 2024
e9035dd
Merge remote-tracking branch 'origin/master' into lx/update-honk-agg-…
lucasxia01 Aug 12, 2024
eeda4a1
remove throw error and replace with assert
lucasxia01 Aug 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions barretenberg/cpp/src/barretenberg/bb/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -576,6 +576,37 @@ void prove_tube(const std::string& output_path)
ClientIVC verifier{ builder, input };

verifier.verify(proof);

std::array<uint32_t, acir_format::HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE> current_aggregation_object = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
};
fq x0("0x031e97a575e9d05a107acb64952ecab75c020998797da7842ab5d6d1986846cf");
fq y0("0x178cbf4206471d722669117f9758a4c410db10a01750aebb5666547acf8bd5a4");
fq x1("0x0f94656a2ca489889939f81e9c74027fd51009034b3357f0e91b8a11e7842c38");
fq y1("0x1b52c2020d7464a0c80c0da527a08193fe27776f50224bd6fb128b46c1ddb67f");
std::vector<fq> aggregation_object_fq_values = { x0, y0, x1, y1 };
size_t agg_obj_indices_idx = 0;
for (fq val : aggregation_object_fq_values) {
const uint256_t x = val;
std::array<fr, acir_format::fq_ct::NUM_LIMBS> val_limbs = {
x.slice(0, acir_format::fq_ct::NUM_LIMB_BITS),
x.slice(acir_format::fq_ct::NUM_LIMB_BITS, acir_format::fq_ct::NUM_LIMB_BITS * 2),
x.slice(acir_format::fq_ct::NUM_LIMB_BITS * 2, acir_format::fq_ct::NUM_LIMB_BITS * 3),
x.slice(acir_format::fq_ct::NUM_LIMB_BITS * 3, stdlib::field_conversion::TOTAL_BITS)
};
for (size_t i = 0; i < acir_format::fq_ct::NUM_LIMBS; ++i) {
uint32_t idx = builder->add_variable(val_limbs[i]);
builder->set_public_input(idx);
current_aggregation_object[agg_obj_indices_idx] = idx;
agg_obj_indices_idx++;
}
}
// Make sure the verification key records the public input indices of the
// final recursion output.
std::vector<uint32_t> proof_output_witness_indices(current_aggregation_object.begin(),
current_aggregation_object.end());
builder->set_recursive_proof(proof_output_witness_indices);

info("num gates in tube circuit: ", builder->get_num_gates());
using Prover = UltraProver_<UltraFlavor>;
using Verifier = UltraVerifier_<UltraFlavor>;
Expand Down
108 changes: 52 additions & 56 deletions barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -348,20 +348,19 @@ void build_constraints(Builder& builder,
// constants set by keeping the nested aggregation object attached to the proof as public inputs.
std::array<uint32_t, HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE> nested_aggregation_object = {};
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
// for (size_t i = 0; i < HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE; ++i) {
// // Set the nested aggregation object indices to witness indices from the proof
// nested_aggregation_object[i] =
// static_cast<uint32_t>(constraint.proof[HonkRecursionConstraint::inner_public_input_offset + i]);
// // Adding the nested aggregation object to the constraint's public inputs
// constraint.public_inputs.emplace_back(nested_aggregation_object[i]);
// }
for (size_t i = 0; i < HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE; ++i) {
// Set the nested aggregation object indices to witness indices from the proof
nested_aggregation_object[i] =
static_cast<uint32_t>(constraint.proof[HonkRecursionConstraint::inner_public_input_offset + i]);
// Adding the nested aggregation object to the constraint's public inputs
constraint.public_inputs.emplace_back(nested_aggregation_object[i]);
}
// Remove the aggregation object so that they can be handled as normal public inputs
// in they way that the recursion constraint expects
// constraint.proof.erase(constraint.proof.begin() + HonkRecursionConstraint::inner_public_input_offset,
// constraint.proof.begin() +
// static_cast<std::ptrdiff_t>(HonkRecursionConstraint::inner_public_input_offset
// +
// HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE));
constraint.proof.erase(constraint.proof.begin() + HonkRecursionConstraint::inner_public_input_offset,
constraint.proof.begin() +
static_cast<std::ptrdiff_t>(HonkRecursionConstraint::inner_public_input_offset +
HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE));
current_aggregation_object = create_honk_recursion_constraints(builder,
constraint,
current_aggregation_object,
Expand All @@ -379,51 +378,48 @@ void build_constraints(Builder& builder,
// Set the indices as public inputs because they are no longer being
// created in ACIR
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
// for (const auto& idx : current_aggregation_object) {
// builder.set_public_input(idx);
// }

// // Make sure the verification key records the public input indices of the
// // final recursion output.
// std::vector<uint32_t> proof_output_witness_indices(current_aggregation_object.begin(),
// current_aggregation_object.end());
// builder.set_recursive_proof(proof_output_witness_indices);
for (const auto& idx : current_aggregation_object) {
builder.set_public_input(idx);
}

// Make sure the verification key records the public input indices of the
// final recursion output.
std::vector<uint32_t> proof_output_witness_indices(current_aggregation_object.begin(),
current_aggregation_object.end());
builder.set_recursive_proof(proof_output_witness_indices);
} else if (honk_recursion &&
builder.is_recursive_circuit) { // Set a default aggregation object if we don't have one.
// TODO(https://github.com/AztecProtocol/barretenberg/issues/911): These are pairing points extracted
// from a valid proof. This is a workaround because we can't represent the point at infinity in biggroup
// yet.
fq x0("0x031e97a575e9d05a107acb64952ecab75c020998797da7842ab5d6d1986846cf");
fq y0("0x178cbf4206471d722669117f9758a4c410db10a01750aebb5666547acf8bd5a4");

fq x1("0x0f94656a2ca489889939f81e9c74027fd51009034b3357f0e91b8a11e7842c38");
fq y1("0x1b52c2020d7464a0c80c0da527a08193fe27776f50224bd6fb128b46c1ddb67f");
std::vector<fq> aggregation_object_fq_values = { x0, y0, x1, y1 };
size_t agg_obj_indices_idx = 0;
for (fq val : aggregation_object_fq_values) {
const uint256_t x = val;
std::array<fr, fq_ct::NUM_LIMBS> val_limbs = {
x.slice(0, fq_ct::NUM_LIMB_BITS),
x.slice(fq_ct::NUM_LIMB_BITS, fq_ct::NUM_LIMB_BITS * 2),
x.slice(fq_ct::NUM_LIMB_BITS * 2, fq_ct::NUM_LIMB_BITS * 3),
x.slice(fq_ct::NUM_LIMB_BITS * 3, stdlib::field_conversion::TOTAL_BITS)
};
for (size_t i = 0; i < fq_ct::NUM_LIMBS; ++i) {
uint32_t idx = builder.add_variable(val_limbs[i]);
builder.set_public_input(idx);
current_aggregation_object[agg_obj_indices_idx] = idx;
agg_obj_indices_idx++;
}
}
// Make sure the verification key records the public input indices of the
// final recursion output.
std::vector<uint32_t> proof_output_witness_indices(current_aggregation_object.begin(),
current_aggregation_object.end());
builder.set_recursive_proof(proof_output_witness_indices);
}
static_cast<void>(honk_recursion);
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
// else if (honk_recursion &&
// builder.is_recursive_circuit) { // Set a default aggregation object if we don't have one.
// // TODO(https://github.com/AztecProtocol/barretenberg/issues/911): These are pairing points extracted
// from
// // a valid proof. This is a workaround because we can't represent the point at infinity in biggroup
// yet. fq x0("0x031e97a575e9d05a107acb64952ecab75c020998797da7842ab5d6d1986846cf"); fq
// y0("0x178cbf4206471d722669117f9758a4c410db10a01750aebb5666547acf8bd5a4");

// fq x1("0x0f94656a2ca489889939f81e9c74027fd51009034b3357f0e91b8a11e7842c38");
// fq y1("0x1b52c2020d7464a0c80c0da527a08193fe27776f50224bd6fb128b46c1ddb67f");
// std::vector<fq> aggregation_object_fq_values = { x0, y0, x1, y1 };
// size_t agg_obj_indices_idx = 0;
// for (fq val : aggregation_object_fq_values) {
// const uint256_t x = val;
// std::array<fr, fq_ct::NUM_LIMBS> val_limbs = {
// x.slice(0, fq_ct::NUM_LIMB_BITS),
// x.slice(fq_ct::NUM_LIMB_BITS, fq_ct::NUM_LIMB_BITS * 2),
// x.slice(fq_ct::NUM_LIMB_BITS * 2, fq_ct::NUM_LIMB_BITS * 3),
// x.slice(fq_ct::NUM_LIMB_BITS * 3, stdlib::field_conversion::TOTAL_BITS)
// };
// for (size_t i = 0; i < fq_ct::NUM_LIMBS; ++i) {
// uint32_t idx = builder.add_variable(val_limbs[i]);
// builder.set_public_input(idx);
// current_aggregation_object[agg_obj_indices_idx] = idx;
// agg_obj_indices_idx++;
// }
// }
// // Make sure the verification key records the public input indices of the
// // final recursion output.
// std::vector<uint32_t> proof_output_witness_indices(current_aggregation_object.begin(),
// current_aggregation_object.end());
// builder.set_recursive_proof(proof_output_witness_indices);
// }
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -211,13 +211,13 @@ std::array<uint32_t, HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE> create_ho
// Recursively verify the proof
auto vkey = std::make_shared<RecursiveVerificationKey>(builder, key_fields);
RecursiveVerifier verifier(&builder, vkey);
std::array<typename Flavor::GroupElement, 2> pairing_points = verifier.verify_proof(proof_fields);
aggregation_state_ct pairing_points = verifier.verify_proof(proof_fields);

// Aggregate the current aggregation object with these pairing points from verify_proof
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
aggregation_state_ct cur_aggregation_object;
cur_aggregation_object.P0 = pairing_points[0]; // * recursion_separator;
cur_aggregation_object.P1 = pairing_points[1]; // * recursion_separator;
cur_aggregation_object.P0 = pairing_points.P0; // * recursion_separator;
cur_aggregation_object.P1 = pairing_points.P1; // * recursion_separator;

std::vector<uint32_t> proof_witness_indices = {
cur_aggregation_object.P0.x.binary_basis_limbs[0].element.normalize().witness_index,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,13 +162,15 @@ class AcirHonkRecursionConstraint : public ::testing::Test {
std::vector<fr> inner_public_input_values(
proof_witnesses.begin() + static_cast<std::ptrdiff_t>(inner_public_input_offset),
proof_witnesses.begin() +
static_cast<std::ptrdiff_t>(inner_public_input_offset + num_inner_public_inputs));
static_cast<std::ptrdiff_t>(inner_public_input_offset + num_inner_public_inputs -
RecursionConstraint::AGGREGATION_OBJECT_SIZE));

// We want to make sure that we do not remove the nested aggregation object.
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
proof_witnesses.erase(proof_witnesses.begin() + static_cast<std::ptrdiff_t>(inner_public_input_offset),
proof_witnesses.begin() +
static_cast<std::ptrdiff_t>(inner_public_input_offset + num_inner_public_inputs));
static_cast<std::ptrdiff_t>(inner_public_input_offset + num_inner_public_inputs -
RecursionConstraint::AGGREGATION_OBJECT_SIZE));

std::vector<bb::fr> key_witnesses = verification_key->to_field_elements();

Expand All @@ -179,9 +181,9 @@ class AcirHonkRecursionConstraint : public ::testing::Test {
const uint32_t public_input_start_idx =
static_cast<uint32_t>(inner_public_input_offset + witness_offset); // points to public_input_0
const uint32_t proof_indices_start_idx =
static_cast<uint32_t>(public_input_start_idx + num_inner_public_inputs);
static_cast<uint32_t>(public_input_start_idx + num_inner_public_inputs -
RecursionConstraint::AGGREGATION_OBJECT_SIZE); // points to agg_obj_0
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
// - RecursionConstraint::AGGREGATION_OBJECT_SIZE); // points to agg_obj_0
const uint32_t key_indices_start_idx =
static_cast<uint32_t>(proof_indices_start_idx + proof_witnesses.size() -
inner_public_input_offset); // would point to vkey_3 without the -
Expand All @@ -205,7 +207,7 @@ class AcirHonkRecursionConstraint : public ::testing::Test {
// thus we do not explicitly have to keep the public inputs while setting up the initial recursion
// constraint. They will later be attached as public inputs when creating the circuit.
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
for (size_t i = 0; i < num_inner_public_inputs; ++i) {
for (size_t i = 0; i < num_inner_public_inputs - RecursionConstraint::AGGREGATION_OBJECT_SIZE; ++i) {
inner_public_inputs.push_back(static_cast<uint32_t>(i + public_input_start_idx));
}

Expand Down Expand Up @@ -245,7 +247,7 @@ class AcirHonkRecursionConstraint : public ::testing::Test {
// We once again have to check whether we have a nested proof, because if we do have one
// then we could get a segmentation fault as `inner_public_inputs` was never filled with values.
// TODO(https://github.com/AztecProtocol/barretenberg/issues/1044): Reinstate aggregation
for (size_t i = 0; i < num_inner_public_inputs; ++i) {
for (size_t i = 0; i < num_inner_public_inputs - RecursionConstraint::AGGREGATION_OBJECT_SIZE; ++i) {
witness[inner_public_inputs[i]] = inner_public_input_values[i];
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,19 +24,20 @@ UltraRecursiveVerifier_<Flavor>::UltraRecursiveVerifier_(Builder* builder, const
*
*/
template <typename Flavor>
std::array<typename Flavor::GroupElement, 2> UltraRecursiveVerifier_<Flavor>::verify_proof(const HonkProof& proof)
UltraRecursiveVerifier_<Flavor>::PairingPoints UltraRecursiveVerifier_<Flavor>::verify_proof(
const HonkProof& proof, const aggregation_state<typename Flavor::Curve>& agg_obj)
{
StdlibProof<Builder> stdlib_proof = bb::convert_proof_to_witness(builder, proof);
return verify_proof(stdlib_proof);
return verify_proof(stdlib_proof, agg_obj);
}

/**
* @brief This function constructs a recursive verifier circuit for a native Ultra Honk proof of a given flavor.
*
*/
template <typename Flavor>
std::array<typename Flavor::GroupElement, 2> UltraRecursiveVerifier_<Flavor>::verify_proof(
const StdlibProof<Builder>& proof)
UltraRecursiveVerifier_<Flavor>::PairingPoints UltraRecursiveVerifier_<Flavor>::verify_proof(
const StdlibProof<Builder>& proof, const aggregation_state<typename Flavor::Curve>& agg_obj)
{
using Sumcheck = ::bb::SumcheckVerifier<Flavor>;
using PCS = typename Flavor::PCS;
Expand Down Expand Up @@ -67,6 +68,7 @@ std::array<typename Flavor::GroupElement, 2> UltraRecursiveVerifier_<Flavor>::ve
for (size_t i = 0; i < key->num_public_inputs; ++i) {
public_inputs.emplace_back(transcript->template receive_from_prover<FF>("public_input_" + std::to_string(i)));
}
// WORKTODO: parse out the aggregation object here

// Get commitments to first three wire polynomials
commitments.w_l = transcript->template receive_from_prover<Commitment>(commitment_labels.w_l);
Expand Down Expand Up @@ -154,7 +156,9 @@ std::array<typename Flavor::GroupElement, 2> UltraRecursiveVerifier_<Flavor>::ve
transcript);
auto pairing_points = PCS::reduce_verify(opening_claim, transcript);

return pairing_points;
// WORKTODO: aggregate here
static_cast<void>(pairing_points);
return agg_obj;
}

template class UltraRecursiveVerifier_<bb::UltraRecursiveFlavor_<UltraCircuitBuilder>>;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#pragma once
#include "barretenberg/honk/proof_system/types/proof.hpp"
#include "barretenberg/stdlib/honk_recursion/transcript/transcript.hpp"
#include "barretenberg/stdlib/plonk_recursion/aggregation_state/aggregation_state.hpp"
#include "barretenberg/stdlib_circuit_builders/mega_recursive_flavor.hpp"
#include "barretenberg/stdlib_circuit_builders/ultra_recursive_flavor.hpp"
#include "barretenberg/sumcheck/sumcheck.hpp"
Expand All @@ -16,7 +17,7 @@ template <typename Flavor> class UltraRecursiveVerifier_ {
using VerifierCommitmentKey = typename Flavor::VerifierCommitmentKey;
using Builder = typename Flavor::CircuitBuilder;
using RelationSeparator = typename Flavor::RelationSeparator;
using PairingPoints = std::array<GroupElement, 2>;
using PairingPoints = aggregation_state<typename Flavor::Curve>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably rename all these pairing points to AggregationObject

using Transcript = bb::BaseTranscript<bb::stdlib::recursion::honk::StdlibTranscriptParams<Builder>>;

explicit UltraRecursiveVerifier_(Builder* builder,
Expand All @@ -25,8 +26,12 @@ template <typename Flavor> class UltraRecursiveVerifier_ {

// TODO(luke): Eventually this will return something like aggregation_state but I'm simplifying for now until we
// determine the exact interface. Simply returns the two pairing points.
PairingPoints verify_proof(const HonkProof& proof);
PairingPoints verify_proof(const StdlibProof<Builder>& proof);
PairingPoints verify_proof(
const HonkProof& proof,
const aggregation_state<typename Flavor::Curve>& previous_output = aggregation_state<typename Flavor::Curve>());
PairingPoints verify_proof(
const StdlibProof<Builder>& proof,
const aggregation_state<typename Flavor::Curve>& previous_output = aggregation_state<typename Flavor::Curve>());

std::shared_ptr<VerificationKey> key;
std::map<std::string, Commitment> commitments;
Expand Down
Loading
Loading