Add headers first validation to simpa #364
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1ac721c to
9c37179
Compare
michaelsutton
approved these changes
Dec 26, 2023
michaelsutton
added a commit
to michaelsutton/rusty-kaspa
that referenced
this pull request
Dec 26, 2023
commit a4ba784 Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:22:36 2023 +0200 Add headers first validation to simpa (kaspanet#364) commit fb1aa5f Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:07:19 2023 +0200 Move to mimalloc (kaspanet#361) * mimalloc * mimalloc disable purge_decommits * Add to simpa and integration tests * remove redundant unix and replace with win/linux/mac * Add comment * Add comment * Sort out features * Get rid of ctor --------- Co-authored-by: Michael Sutton <msutton@cs.huji.ac.il> commit b6f7e09 Author: aspect <anton.yemelyanov@gmail.com> Date: Tue Dec 26 13:38:24 2023 +0200 Cumulative PR for Wallet & Metrics Refactoring (kaspanet#360) * external access to kaspad core rpc services API * rename wallet event members for serde serialization misc wallet functionality update * wip - event renaming * wip - event serialization tuneup * remove default wrpc listening ports on kaspad Args + fix port mapping bug for automatic JSON RPC port mapping. * Box wallet events * introduce RpcCtl + isolate wRPC from DynRpcApi * debug trait for secret * de-couple wRPC from the runtiem wallet making RPC bindings generic * convert wallet_list to return Vec<WalletDescriptor> * user-friendly error message on decryption error * UtxoProcessor - ignore task shutdown if not running * wallet storage data versioning * wallet - propagate wallet title * sort traits on WalletDescriptor * wallet - use precise decimal conversion when parsing strings to sompi * display kaspa on get-balance-by-address-rpc * wallet - rename settings fns to have try_ prefixes * rename library names "kaspad" and "kaspa_cli" to "kaspad_lib" and "kaspa_cli_lib" to prevent build collisions * no-unsafe-eval feature * allow for chrome-extension schema in the RPC url * wallet - expose bip32 wordlist for external access * wallet - delegate rpc ownership to utxo_processor; reorder rpc disconnect to occur before utxo_processor shutdown; * wallet - migrate wallet settings loading to cli * rename RpcCtlOp (Open, Close) to RpcState (Opened, Closed) * remove automatic RPC disconnect from wallet stop() to allow retention of the RPC channel after closing the wallet * allow UtxoProcessor to bind to an already existing/connected RPC channel, allow retention of a connected RPC channel past UtxoProcessor shutdown. (API centric defensive functionality) * code formatting * fix clippy warnings * migrate chrome-extension to use ws:// schema by default * wallet list formatting * staging for WRS 0.7.0 * add bip32 word list iterator * update WRS to 0.7.0 * fix rustdoc errors * wip - initial WalletApi trait implementation * wallet - macros and account descriptors * wallet - ping_call * cleanup * wip - wallet api cleanup & testing * kaspa-metrics as a separate crate * wip - metrics cleanup * cargo.lock update * async/sync api changes for storage * merge * master merge * wip * wip * wip * wallet - (wip) changes to AccountDescriptor and Wallet API * metrics - make sink future return as optional * wip - refactor wallet open; improve wallet event ordering; cleanup * wip (debugging UtxoChanged notifications) - fix issuance during failed tx * adding workflow-rs as github dependency (allows CI) * revert github deps for WRS * metrics * refactor metrics to include bdi counts and peer counts * wallet api updates refactoring transaction access interface misc updates to wallet APIs refactor wallet open sequence refactor legacy accounts * wallet framework - fix a but resulting in the early issuane of UTXO maturity events. Refactor handling of outgoing transaction UTXOs (sending outgoing tx ACKs on UtxoChange removal), minor fixes in the Transaction file-system storage. Add "outgoing" property to account balances (to track in-flight transactions being submitted until acceptance). Introduce new Events::Change that gets fired specifically when receiving change of the currently outgoing transaction. Initial steps toward Daa property wrapper. * npm 0.13.0 * metrics - cleanup after RPC refactoring * rename rust-side of wasm bindings as naming conventions interfere with rust use (make fns more explicitly named) * make metrics data formatting fns as pub * expose log levels in kaspa-core::log * rename metrics & improve metrics fetch task * add API calls to pre-configure wallet framework stroage folders * cumulative updates for Cargo.lock * Improve address error messages for human readability * PrvKeyData enumeration, initial account transfer support (wip), wallet descriptor handling * wallet - change error to retain PrvKeyDataId instead of a String * propagate bip39 flag to runtime account and it's descriptor * wip - keydata refactoring * metrics refactoring (bandwidth) * wip (cleanup) * misc improvements to metrics * code formatting * wRPC bandwidth monitoring + scaffolding for global bandwidth metrics * bump WRS to 0.9.0 * metrics - add mempool_size and convert some items to u32/f32 * metrics cleanup * wallet and account rename, remove "title" from accounts * add missing mempool size metric * UTXO "stasis" mode for new coinbase transactions * remove useless Option from the rpc url parser * fix daa score in maturity state check * add name_or_id to PrvKeyDataInfo * wallet RPC Connect/Disconnect and GetStatus API calls * cleanup * support for 12-word bip39 mnemonics * add a hidden --daemon flag to kaspad * refactor get_daa_score_timestamp helper to return Vec<u64> instead of Response struct * Add Scan event * implement support for unixtime in transactions * kaspa value formatting functions * fix a deadlock due to re-entrant mutex lock when loading multiple accounts from storage * introduce utxo stasis mode + make balance carry utxo counts as well as the totals * refactor wallet events, replace reset() with close() where appropriate * manual debug impl for tx components (display script as hex, address as address etc) * fix incorrect gas assignment in WASM bindings * updates for wallet event handling * refactor sign_with_multiple to return Err(tx) when partially signed * refactor Storage and Wallet descriptors * fix account derivation index handling in DerivationCapable * track acceptance DAA stamp in outgoing transaction record * temporary harness for embedding script engine directly into tx generator (used for debugging) * convert PendingTransaction Utxo entries into HashSet * rename pending tx fns * outgoing tx wrapper around pending tx * refactor notification events during wallet ops * as_ref() for PrvKeyData * wip - initial RpcCoreMocks for wallet framework tests * wallet api messages update for StorageDescriptor usage * update AddressDerivationMeta to track addresses from index 0 * add AHash and async_channel to wallet_core crate * allow Args to be used externally for daemon argument pre-validation * impl Hash for UtxoEntryReference and rename "fake()" to "simulated()" * update wallet events * expose tx validator check_scripts() as pub * update wallet wasm fns for wallet and storage descriptors * rename kaspa-metrics to kaspa-metrics-core * code formatting * code formatting * migrate transaction console formatting to cli/extensions * refactor wallet events * use ahash for faster processing in wallet hash maps (utxo context) * wallet - add SendError handling for error::Error * wallet event refactoring, scan/discovery process refactoring, wallet change secret api handling * refactoring: transfer and deep derivation scan * transaction record refactoring * wallet - support for cross-account transfers in generator * wallet - outgoing transaction impl * wallet - refactor maturity handling * wallet/utxo - initial tests scaffolding * minor optimization of the discovery/scan process * wallet - utxo processor outgoing tx and discovery timestamp unixtime handling * refactor utxo context to use output transactions, refactor transfers, compound and change notifications * code formatting + Cargo.lock update * Signed enum to represent Fully or Partially signed tx * bind p2p and grpc bandwidth metrics * misc clippy fixes * cleanup * remove --daemon as Args::parse() offers external mutations * remove previously implemented mnemonic variant in favor of client-side mnemonic creation * remove AccessContextT, convert to wallet_secret, wip WalletApi traits, refactor account args * code formatting * separate runtime wallet into multiple modules * wallet - revert create_accounts to process a single account at a time * cleanup * wallet - JSON import and export + preliminary docs * address - AddressList - fix TryFrom deadlock caused by recursive trait invocation * wallet - api rustdoc + cleanup, transaction calls * wallet docs * change address debug formatting * adding cli rpc addpeer * wip - wallet restructuring * wip - restructure wallet modules * wip - wallet restructuring - cleanup * wip - AccountKind refactoring * wallet - refactoring storage serialization & encryption * fold in PR kaspanet#349 * wallet framework - misc helper fns and cleanup * storage unit tests + prv key data module restructuring * wallet - account factory registrations * Remove panics caused by sudden loss of connection while submitting transactions. * cleanup * cleanup * cleanup + docs * docs * adjust metrics formatting * cleanup * bump workflow-rs to 0.10.1 * fix wallet storage unit tests * fix generator unit tests * convert missing listener_id panic to error * docs * code formatting * lints * misc updates following github reviews * refactor AccountKind to use fixedstr and implement Copy * refactor AccountKind to use fixedstr (str64) and re-implement Copy, removing clone() where necessary * change js-bound fn names from *as* to *to* to follow Rust conventions * replace AssocPrvKeyDataIds iter().any() to contains() * remove vec![] in MetricsGroup iterator fn; add TODO comment on Metric::group() * scope magic numbers within their associated structs * add try_fully_signed() and try_partially_signed() fns to Signed that allow recovery of the tx * refactor local storage cache to use RwLock instead of a Mutex * refactor serialization as per github feedback (partial trait impl) * Optimize AssocPrvKeyDataIds to use Either in the IntoIter impl, removing allocations. * Refactor to bypass aux buffer creation when storing wallet on native platforms. * impl custom Debug for storage structs to display data in hex * adding misc SAFETY comments on unsafe ops * TODO comments for later SignerT+multisig refactoring * Change panic!() to unreachable!() to make it clearer that a state should not be reached. * rename TransactionRecord::unixtime to unixtime_msec * fix wrong references in comments * Fix min_cosigner_index handling in multisig fns * add comments to UtxoContext::insert() error handling * bind AccountStorable:Storable for AccountStorage, migrate Storable into a separate module, add comments to MAGIC values in some locations * make framework init functions unsafe * update workflow-rs to 0.10.2 * fix comments * fix data cloning when getting collection counts * adding TODO notes to wasm macros and KAS value decimal parsing --------- Co-authored-by: Surinder Singh Matoo <surinder83singh@gmail.com> commit 10d0ebf Author: D-Stacks <78099568+D-Stacks@users.noreply.github.com> Date: Tue Dec 26 10:43:59 2023 +0100 make step 3. of linux installation not fail copy and paste (kaspanet#363) remove the space to escape the newline instead.
someone235
pushed a commit
that referenced
this pull request
Dec 28, 2023
… data sizes (#351) * cache inner struct: prep for tracking size * initial cache policy and mem size estimator * use new cache policies * fix storage calcs * update TN11 genesis * temp: limit max block parents in virtual processor * use more conservative constants (wip) * ensure sufficient cache allocs for high level stores * increase caches related to daa window * Squashed commit of the following: commit a4ba784 Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:22:36 2023 +0200 Add headers first validation to simpa (#364) commit fb1aa5f Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:07:19 2023 +0200 Move to mimalloc (#361) * mimalloc * mimalloc disable purge_decommits * Add to simpa and integration tests * remove redundant unix and replace with win/linux/mac * Add comment * Add comment * Sort out features * Get rid of ctor --------- Co-authored-by: Michael Sutton <msutton@cs.huji.ac.il> commit b6f7e09 Author: aspect <anton.yemelyanov@gmail.com> Date: Tue Dec 26 13:38:24 2023 +0200 Cumulative PR for Wallet & Metrics Refactoring (#360) * external access to kaspad core rpc services API * rename wallet event members for serde serialization misc wallet functionality update * wip - event renaming * wip - event serialization tuneup * remove default wrpc listening ports on kaspad Args + fix port mapping bug for automatic JSON RPC port mapping. * Box wallet events * introduce RpcCtl + isolate wRPC from DynRpcApi * debug trait for secret * de-couple wRPC from the runtiem wallet making RPC bindings generic * convert wallet_list to return Vec<WalletDescriptor> * user-friendly error message on decryption error * UtxoProcessor - ignore task shutdown if not running * wallet storage data versioning * wallet - propagate wallet title * sort traits on WalletDescriptor * wallet - use precise decimal conversion when parsing strings to sompi * display kaspa on get-balance-by-address-rpc * wallet - rename settings fns to have try_ prefixes * rename library names "kaspad" and "kaspa_cli" to "kaspad_lib" and "kaspa_cli_lib" to prevent build collisions * no-unsafe-eval feature * allow for chrome-extension schema in the RPC url * wallet - expose bip32 wordlist for external access * wallet - delegate rpc ownership to utxo_processor; reorder rpc disconnect to occur before utxo_processor shutdown; * wallet - migrate wallet settings loading to cli * rename RpcCtlOp (Open, Close) to RpcState (Opened, Closed) * remove automatic RPC disconnect from wallet stop() to allow retention of the RPC channel after closing the wallet * allow UtxoProcessor to bind to an already existing/connected RPC channel, allow retention of a connected RPC channel past UtxoProcessor shutdown. (API centric defensive functionality) * code formatting * fix clippy warnings * migrate chrome-extension to use ws:// schema by default * wallet list formatting * staging for WRS 0.7.0 * add bip32 word list iterator * update WRS to 0.7.0 * fix rustdoc errors * wip - initial WalletApi trait implementation * wallet - macros and account descriptors * wallet - ping_call * cleanup * wip - wallet api cleanup & testing * kaspa-metrics as a separate crate * wip - metrics cleanup * cargo.lock update * async/sync api changes for storage * merge * master merge * wip * wip * wip * wallet - (wip) changes to AccountDescriptor and Wallet API * metrics - make sink future return as optional * wip - refactor wallet open; improve wallet event ordering; cleanup * wip (debugging UtxoChanged notifications) - fix issuance during failed tx * adding workflow-rs as github dependency (allows CI) * revert github deps for WRS * metrics * refactor metrics to include bdi counts and peer counts * wallet api updates refactoring transaction access interface misc updates to wallet APIs refactor wallet open sequence refactor legacy accounts * wallet framework - fix a but resulting in the early issuane of UTXO maturity events. Refactor handling of outgoing transaction UTXOs (sending outgoing tx ACKs on UtxoChange removal), minor fixes in the Transaction file-system storage. Add "outgoing" property to account balances (to track in-flight transactions being submitted until acceptance). Introduce new Events::Change that gets fired specifically when receiving change of the currently outgoing transaction. Initial steps toward Daa property wrapper. * npm 0.13.0 * metrics - cleanup after RPC refactoring * rename rust-side of wasm bindings as naming conventions interfere with rust use (make fns more explicitly named) * make metrics data formatting fns as pub * expose log levels in kaspa-core::log * rename metrics & improve metrics fetch task * add API calls to pre-configure wallet framework stroage folders * cumulative updates for Cargo.lock * Improve address error messages for human readability * PrvKeyData enumeration, initial account transfer support (wip), wallet descriptor handling * wallet - change error to retain PrvKeyDataId instead of a String * propagate bip39 flag to runtime account and it's descriptor * wip - keydata refactoring * metrics refactoring (bandwidth) * wip (cleanup) * misc improvements to metrics * code formatting * wRPC bandwidth monitoring + scaffolding for global bandwidth metrics * bump WRS to 0.9.0 * metrics - add mempool_size and convert some items to u32/f32 * metrics cleanup * wallet and account rename, remove "title" from accounts * add missing mempool size metric * UTXO "stasis" mode for new coinbase transactions * remove useless Option from the rpc url parser * fix daa score in maturity state check * add name_or_id to PrvKeyDataInfo * wallet RPC Connect/Disconnect and GetStatus API calls * cleanup * support for 12-word bip39 mnemonics * add a hidden --daemon flag to kaspad * refactor get_daa_score_timestamp helper to return Vec<u64> instead of Response struct * Add Scan event * implement support for unixtime in transactions * kaspa value formatting functions * fix a deadlock due to re-entrant mutex lock when loading multiple accounts from storage * introduce utxo stasis mode + make balance carry utxo counts as well as the totals * refactor wallet events, replace reset() with close() where appropriate * manual debug impl for tx components (display script as hex, address as address etc) * fix incorrect gas assignment in WASM bindings * updates for wallet event handling * refactor sign_with_multiple to return Err(tx) when partially signed * refactor Storage and Wallet descriptors * fix account derivation index handling in DerivationCapable * track acceptance DAA stamp in outgoing transaction record * temporary harness for embedding script engine directly into tx generator (used for debugging) * convert PendingTransaction Utxo entries into HashSet * rename pending tx fns * outgoing tx wrapper around pending tx * refactor notification events during wallet ops * as_ref() for PrvKeyData * wip - initial RpcCoreMocks for wallet framework tests * wallet api messages update for StorageDescriptor usage * update AddressDerivationMeta to track addresses from index 0 * add AHash and async_channel to wallet_core crate * allow Args to be used externally for daemon argument pre-validation * impl Hash for UtxoEntryReference and rename "fake()" to "simulated()" * update wallet events * expose tx validator check_scripts() as pub * update wallet wasm fns for wallet and storage descriptors * rename kaspa-metrics to kaspa-metrics-core * code formatting * code formatting * migrate transaction console formatting to cli/extensions * refactor wallet events * use ahash for faster processing in wallet hash maps (utxo context) * wallet - add SendError handling for error::Error * wallet event refactoring, scan/discovery process refactoring, wallet change secret api handling * refactoring: transfer and deep derivation scan * transaction record refactoring * wallet - support for cross-account transfers in generator * wallet - outgoing transaction impl * wallet - refactor maturity handling * wallet/utxo - initial tests scaffolding * minor optimization of the discovery/scan process * wallet - utxo processor outgoing tx and discovery timestamp unixtime handling * refactor utxo context to use output transactions, refactor transfers, compound and change notifications * code formatting + Cargo.lock update * Signed enum to represent Fully or Partially signed tx * bind p2p and grpc bandwidth metrics * misc clippy fixes * cleanup * remove --daemon as Args::parse() offers external mutations * remove previously implemented mnemonic variant in favor of client-side mnemonic creation * remove AccessContextT, convert to wallet_secret, wip WalletApi traits, refactor account args * code formatting * separate runtime wallet into multiple modules * wallet - revert create_accounts to process a single account at a time * cleanup * wallet - JSON import and export + preliminary docs * address - AddressList - fix TryFrom deadlock caused by recursive trait invocation * wallet - api rustdoc + cleanup, transaction calls * wallet docs * change address debug formatting * adding cli rpc addpeer * wip - wallet restructuring * wip - restructure wallet modules * wip - wallet restructuring - cleanup * wip - AccountKind refactoring * wallet - refactoring storage serialization & encryption * fold in PR #349 * wallet framework - misc helper fns and cleanup * storage unit tests + prv key data module restructuring * wallet - account factory registrations * Remove panics caused by sudden loss of connection while submitting transactions. * cleanup * cleanup * cleanup + docs * docs * adjust metrics formatting * cleanup * bump workflow-rs to 0.10.1 * fix wallet storage unit tests * fix generator unit tests * convert missing listener_id panic to error * docs * code formatting * lints * misc updates following github reviews * refactor AccountKind to use fixedstr and implement Copy * refactor AccountKind to use fixedstr (str64) and re-implement Copy, removing clone() where necessary * change js-bound fn names from *as* to *to* to follow Rust conventions * replace AssocPrvKeyDataIds iter().any() to contains() * remove vec![] in MetricsGroup iterator fn; add TODO comment on Metric::group() * scope magic numbers within their associated structs * add try_fully_signed() and try_partially_signed() fns to Signed that allow recovery of the tx * refactor local storage cache to use RwLock instead of a Mutex * refactor serialization as per github feedback (partial trait impl) * Optimize AssocPrvKeyDataIds to use Either in the IntoIter impl, removing allocations. * Refactor to bypass aux buffer creation when storing wallet on native platforms. * impl custom Debug for storage structs to display data in hex * adding misc SAFETY comments on unsafe ops * TODO comments for later SignerT+multisig refactoring * Change panic!() to unreachable!() to make it clearer that a state should not be reached. * rename TransactionRecord::unixtime to unixtime_msec * fix wrong references in comments * Fix min_cosigner_index handling in multisig fns * add comments to UtxoContext::insert() error handling * bind AccountStorable:Storable for AccountStorage, migrate Storable into a separate module, add comments to MAGIC values in some locations * make framework init functions unsafe * update workflow-rs to 0.10.2 * fix comments * fix data cloning when getting collection counts * adding TODO notes to wasm macros and KAS value decimal parsing --------- Co-authored-by: Surinder Singh Matoo <surinder83singh@gmail.com> commit 10d0ebf Author: D-Stacks <78099568+D-Stacks@users.noreply.github.com> Date: Tue Dec 26 10:43:59 2023 +0100 make step 3. of linux installation not fail copy and paste (#363) remove the space to escape the newline instead. * reduce children cache policy * reduce caches 25% * increase tx cache * fix gd alloc and slightly increase other numbers * fix a cache alloc bug by redesigning the cache and adding a min units option + fix bps constants * fix simpa short cmd op name colision * move impls to inner struct * comments, docs and technical refactoring * refactor MemSizeEstimator * refactor bool to MemMode * memsize docs * remove unneeded min limits * comments and name enhancements * rename `CachePolicy::Units` to `CachePolicy::Count` to avoid confusion with `MemMode` * use estimated num of unique headers in validate pruning point headers cache * cache policy builder * add todos * median window has different size + comments * read ghostdag data from compact access if possible * remove wrong upper bound for reachability sets
KashProtocol
pushed a commit
to Kash-Protocol/rusty-kash
that referenced
this pull request
Jan 5, 2024
KashProtocol
pushed a commit
to Kash-Protocol/rusty-kash
that referenced
this pull request
Jan 8, 2024
KashProtocol
pushed a commit
to Kash-Protocol/rusty-kash
that referenced
this pull request
Jan 9, 2024
D-Stacks
pushed a commit
to D-Stacks/rusty-kaspa
that referenced
this pull request
Jan 23, 2024
D-Stacks
pushed a commit
to D-Stacks/rusty-kaspa
that referenced
this pull request
Jan 23, 2024
… data sizes (kaspanet#351) * cache inner struct: prep for tracking size * initial cache policy and mem size estimator * use new cache policies * fix storage calcs * update TN11 genesis * temp: limit max block parents in virtual processor * use more conservative constants (wip) * ensure sufficient cache allocs for high level stores * increase caches related to daa window * Squashed commit of the following: commit a4ba784 Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:22:36 2023 +0200 Add headers first validation to simpa (kaspanet#364) commit fb1aa5f Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:07:19 2023 +0200 Move to mimalloc (kaspanet#361) * mimalloc * mimalloc disable purge_decommits * Add to simpa and integration tests * remove redundant unix and replace with win/linux/mac * Add comment * Add comment * Sort out features * Get rid of ctor --------- Co-authored-by: Michael Sutton <msutton@cs.huji.ac.il> commit b6f7e09 Author: aspect <anton.yemelyanov@gmail.com> Date: Tue Dec 26 13:38:24 2023 +0200 Cumulative PR for Wallet & Metrics Refactoring (kaspanet#360) * external access to kaspad core rpc services API * rename wallet event members for serde serialization misc wallet functionality update * wip - event renaming * wip - event serialization tuneup * remove default wrpc listening ports on kaspad Args + fix port mapping bug for automatic JSON RPC port mapping. * Box wallet events * introduce RpcCtl + isolate wRPC from DynRpcApi * debug trait for secret * de-couple wRPC from the runtiem wallet making RPC bindings generic * convert wallet_list to return Vec<WalletDescriptor> * user-friendly error message on decryption error * UtxoProcessor - ignore task shutdown if not running * wallet storage data versioning * wallet - propagate wallet title * sort traits on WalletDescriptor * wallet - use precise decimal conversion when parsing strings to sompi * display kaspa on get-balance-by-address-rpc * wallet - rename settings fns to have try_ prefixes * rename library names "kaspad" and "kaspa_cli" to "kaspad_lib" and "kaspa_cli_lib" to prevent build collisions * no-unsafe-eval feature * allow for chrome-extension schema in the RPC url * wallet - expose bip32 wordlist for external access * wallet - delegate rpc ownership to utxo_processor; reorder rpc disconnect to occur before utxo_processor shutdown; * wallet - migrate wallet settings loading to cli * rename RpcCtlOp (Open, Close) to RpcState (Opened, Closed) * remove automatic RPC disconnect from wallet stop() to allow retention of the RPC channel after closing the wallet * allow UtxoProcessor to bind to an already existing/connected RPC channel, allow retention of a connected RPC channel past UtxoProcessor shutdown. (API centric defensive functionality) * code formatting * fix clippy warnings * migrate chrome-extension to use ws:// schema by default * wallet list formatting * staging for WRS 0.7.0 * add bip32 word list iterator * update WRS to 0.7.0 * fix rustdoc errors * wip - initial WalletApi trait implementation * wallet - macros and account descriptors * wallet - ping_call * cleanup * wip - wallet api cleanup & testing * kaspa-metrics as a separate crate * wip - metrics cleanup * cargo.lock update * async/sync api changes for storage * merge * master merge * wip * wip * wip * wallet - (wip) changes to AccountDescriptor and Wallet API * metrics - make sink future return as optional * wip - refactor wallet open; improve wallet event ordering; cleanup * wip (debugging UtxoChanged notifications) - fix issuance during failed tx * adding workflow-rs as github dependency (allows CI) * revert github deps for WRS * metrics * refactor metrics to include bdi counts and peer counts * wallet api updates refactoring transaction access interface misc updates to wallet APIs refactor wallet open sequence refactor legacy accounts * wallet framework - fix a but resulting in the early issuane of UTXO maturity events. Refactor handling of outgoing transaction UTXOs (sending outgoing tx ACKs on UtxoChange removal), minor fixes in the Transaction file-system storage. Add "outgoing" property to account balances (to track in-flight transactions being submitted until acceptance). Introduce new Events::Change that gets fired specifically when receiving change of the currently outgoing transaction. Initial steps toward Daa property wrapper. * npm 0.13.0 * metrics - cleanup after RPC refactoring * rename rust-side of wasm bindings as naming conventions interfere with rust use (make fns more explicitly named) * make metrics data formatting fns as pub * expose log levels in kaspa-core::log * rename metrics & improve metrics fetch task * add API calls to pre-configure wallet framework stroage folders * cumulative updates for Cargo.lock * Improve address error messages for human readability * PrvKeyData enumeration, initial account transfer support (wip), wallet descriptor handling * wallet - change error to retain PrvKeyDataId instead of a String * propagate bip39 flag to runtime account and it's descriptor * wip - keydata refactoring * metrics refactoring (bandwidth) * wip (cleanup) * misc improvements to metrics * code formatting * wRPC bandwidth monitoring + scaffolding for global bandwidth metrics * bump WRS to 0.9.0 * metrics - add mempool_size and convert some items to u32/f32 * metrics cleanup * wallet and account rename, remove "title" from accounts * add missing mempool size metric * UTXO "stasis" mode for new coinbase transactions * remove useless Option from the rpc url parser * fix daa score in maturity state check * add name_or_id to PrvKeyDataInfo * wallet RPC Connect/Disconnect and GetStatus API calls * cleanup * support for 12-word bip39 mnemonics * add a hidden --daemon flag to kaspad * refactor get_daa_score_timestamp helper to return Vec<u64> instead of Response struct * Add Scan event * implement support for unixtime in transactions * kaspa value formatting functions * fix a deadlock due to re-entrant mutex lock when loading multiple accounts from storage * introduce utxo stasis mode + make balance carry utxo counts as well as the totals * refactor wallet events, replace reset() with close() where appropriate * manual debug impl for tx components (display script as hex, address as address etc) * fix incorrect gas assignment in WASM bindings * updates for wallet event handling * refactor sign_with_multiple to return Err(tx) when partially signed * refactor Storage and Wallet descriptors * fix account derivation index handling in DerivationCapable * track acceptance DAA stamp in outgoing transaction record * temporary harness for embedding script engine directly into tx generator (used for debugging) * convert PendingTransaction Utxo entries into HashSet * rename pending tx fns * outgoing tx wrapper around pending tx * refactor notification events during wallet ops * as_ref() for PrvKeyData * wip - initial RpcCoreMocks for wallet framework tests * wallet api messages update for StorageDescriptor usage * update AddressDerivationMeta to track addresses from index 0 * add AHash and async_channel to wallet_core crate * allow Args to be used externally for daemon argument pre-validation * impl Hash for UtxoEntryReference and rename "fake()" to "simulated()" * update wallet events * expose tx validator check_scripts() as pub * update wallet wasm fns for wallet and storage descriptors * rename kaspa-metrics to kaspa-metrics-core * code formatting * code formatting * migrate transaction console formatting to cli/extensions * refactor wallet events * use ahash for faster processing in wallet hash maps (utxo context) * wallet - add SendError handling for error::Error * wallet event refactoring, scan/discovery process refactoring, wallet change secret api handling * refactoring: transfer and deep derivation scan * transaction record refactoring * wallet - support for cross-account transfers in generator * wallet - outgoing transaction impl * wallet - refactor maturity handling * wallet/utxo - initial tests scaffolding * minor optimization of the discovery/scan process * wallet - utxo processor outgoing tx and discovery timestamp unixtime handling * refactor utxo context to use output transactions, refactor transfers, compound and change notifications * code formatting + Cargo.lock update * Signed enum to represent Fully or Partially signed tx * bind p2p and grpc bandwidth metrics * misc clippy fixes * cleanup * remove --daemon as Args::parse() offers external mutations * remove previously implemented mnemonic variant in favor of client-side mnemonic creation * remove AccessContextT, convert to wallet_secret, wip WalletApi traits, refactor account args * code formatting * separate runtime wallet into multiple modules * wallet - revert create_accounts to process a single account at a time * cleanup * wallet - JSON import and export + preliminary docs * address - AddressList - fix TryFrom deadlock caused by recursive trait invocation * wallet - api rustdoc + cleanup, transaction calls * wallet docs * change address debug formatting * adding cli rpc addpeer * wip - wallet restructuring * wip - restructure wallet modules * wip - wallet restructuring - cleanup * wip - AccountKind refactoring * wallet - refactoring storage serialization & encryption * fold in PR kaspanet#349 * wallet framework - misc helper fns and cleanup * storage unit tests + prv key data module restructuring * wallet - account factory registrations * Remove panics caused by sudden loss of connection while submitting transactions. * cleanup * cleanup * cleanup + docs * docs * adjust metrics formatting * cleanup * bump workflow-rs to 0.10.1 * fix wallet storage unit tests * fix generator unit tests * convert missing listener_id panic to error * docs * code formatting * lints * misc updates following github reviews * refactor AccountKind to use fixedstr and implement Copy * refactor AccountKind to use fixedstr (str64) and re-implement Copy, removing clone() where necessary * change js-bound fn names from *as* to *to* to follow Rust conventions * replace AssocPrvKeyDataIds iter().any() to contains() * remove vec![] in MetricsGroup iterator fn; add TODO comment on Metric::group() * scope magic numbers within their associated structs * add try_fully_signed() and try_partially_signed() fns to Signed that allow recovery of the tx * refactor local storage cache to use RwLock instead of a Mutex * refactor serialization as per github feedback (partial trait impl) * Optimize AssocPrvKeyDataIds to use Either in the IntoIter impl, removing allocations. * Refactor to bypass aux buffer creation when storing wallet on native platforms. * impl custom Debug for storage structs to display data in hex * adding misc SAFETY comments on unsafe ops * TODO comments for later SignerT+multisig refactoring * Change panic!() to unreachable!() to make it clearer that a state should not be reached. * rename TransactionRecord::unixtime to unixtime_msec * fix wrong references in comments * Fix min_cosigner_index handling in multisig fns * add comments to UtxoContext::insert() error handling * bind AccountStorable:Storable for AccountStorage, migrate Storable into a separate module, add comments to MAGIC values in some locations * make framework init functions unsafe * update workflow-rs to 0.10.2 * fix comments * fix data cloning when getting collection counts * adding TODO notes to wasm macros and KAS value decimal parsing --------- Co-authored-by: Surinder Singh Matoo <surinder83singh@gmail.com> commit 10d0ebf Author: D-Stacks <78099568+D-Stacks@users.noreply.github.com> Date: Tue Dec 26 10:43:59 2023 +0100 make step 3. of linux installation not fail copy and paste (kaspanet#363) remove the space to escape the newline instead. * reduce children cache policy * reduce caches 25% * increase tx cache * fix gd alloc and slightly increase other numbers * fix a cache alloc bug by redesigning the cache and adding a min units option + fix bps constants * fix simpa short cmd op name colision * move impls to inner struct * comments, docs and technical refactoring * refactor MemSizeEstimator * refactor bool to MemMode * memsize docs * remove unneeded min limits * comments and name enhancements * rename `CachePolicy::Units` to `CachePolicy::Count` to avoid confusion with `MemMode` * use estimated num of unique headers in validate pruning point headers cache * cache policy builder * add todos * median window has different size + comments * read ghostdag data from compact access if possible * remove wrong upper bound for reachability sets
smartgoo
pushed a commit
to smartgoo/rusty-kaspa
that referenced
this pull request
Jun 18, 2024
smartgoo
pushed a commit
to smartgoo/rusty-kaspa
that referenced
this pull request
Jun 18, 2024
… data sizes (kaspanet#351) * cache inner struct: prep for tracking size * initial cache policy and mem size estimator * use new cache policies * fix storage calcs * update TN11 genesis * temp: limit max block parents in virtual processor * use more conservative constants (wip) * ensure sufficient cache allocs for high level stores * increase caches related to daa window * Squashed commit of the following: commit a4ba784 Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:22:36 2023 +0200 Add headers first validation to simpa (kaspanet#364) commit fb1aa5f Author: Ori Newman <orinewman1@gmail.com> Date: Tue Dec 26 16:07:19 2023 +0200 Move to mimalloc (kaspanet#361) * mimalloc * mimalloc disable purge_decommits * Add to simpa and integration tests * remove redundant unix and replace with win/linux/mac * Add comment * Add comment * Sort out features * Get rid of ctor --------- Co-authored-by: Michael Sutton <msutton@cs.huji.ac.il> commit b6f7e09 Author: aspect <anton.yemelyanov@gmail.com> Date: Tue Dec 26 13:38:24 2023 +0200 Cumulative PR for Wallet & Metrics Refactoring (kaspanet#360) * external access to kaspad core rpc services API * rename wallet event members for serde serialization misc wallet functionality update * wip - event renaming * wip - event serialization tuneup * remove default wrpc listening ports on kaspad Args + fix port mapping bug for automatic JSON RPC port mapping. * Box wallet events * introduce RpcCtl + isolate wRPC from DynRpcApi * debug trait for secret * de-couple wRPC from the runtiem wallet making RPC bindings generic * convert wallet_list to return Vec<WalletDescriptor> * user-friendly error message on decryption error * UtxoProcessor - ignore task shutdown if not running * wallet storage data versioning * wallet - propagate wallet title * sort traits on WalletDescriptor * wallet - use precise decimal conversion when parsing strings to sompi * display kaspa on get-balance-by-address-rpc * wallet - rename settings fns to have try_ prefixes * rename library names "kaspad" and "kaspa_cli" to "kaspad_lib" and "kaspa_cli_lib" to prevent build collisions * no-unsafe-eval feature * allow for chrome-extension schema in the RPC url * wallet - expose bip32 wordlist for external access * wallet - delegate rpc ownership to utxo_processor; reorder rpc disconnect to occur before utxo_processor shutdown; * wallet - migrate wallet settings loading to cli * rename RpcCtlOp (Open, Close) to RpcState (Opened, Closed) * remove automatic RPC disconnect from wallet stop() to allow retention of the RPC channel after closing the wallet * allow UtxoProcessor to bind to an already existing/connected RPC channel, allow retention of a connected RPC channel past UtxoProcessor shutdown. (API centric defensive functionality) * code formatting * fix clippy warnings * migrate chrome-extension to use ws:// schema by default * wallet list formatting * staging for WRS 0.7.0 * add bip32 word list iterator * update WRS to 0.7.0 * fix rustdoc errors * wip - initial WalletApi trait implementation * wallet - macros and account descriptors * wallet - ping_call * cleanup * wip - wallet api cleanup & testing * kaspa-metrics as a separate crate * wip - metrics cleanup * cargo.lock update * async/sync api changes for storage * merge * master merge * wip * wip * wip * wallet - (wip) changes to AccountDescriptor and Wallet API * metrics - make sink future return as optional * wip - refactor wallet open; improve wallet event ordering; cleanup * wip (debugging UtxoChanged notifications) - fix issuance during failed tx * adding workflow-rs as github dependency (allows CI) * revert github deps for WRS * metrics * refactor metrics to include bdi counts and peer counts * wallet api updates refactoring transaction access interface misc updates to wallet APIs refactor wallet open sequence refactor legacy accounts * wallet framework - fix a but resulting in the early issuane of UTXO maturity events. Refactor handling of outgoing transaction UTXOs (sending outgoing tx ACKs on UtxoChange removal), minor fixes in the Transaction file-system storage. Add "outgoing" property to account balances (to track in-flight transactions being submitted until acceptance). Introduce new Events::Change that gets fired specifically when receiving change of the currently outgoing transaction. Initial steps toward Daa property wrapper. * npm 0.13.0 * metrics - cleanup after RPC refactoring * rename rust-side of wasm bindings as naming conventions interfere with rust use (make fns more explicitly named) * make metrics data formatting fns as pub * expose log levels in kaspa-core::log * rename metrics & improve metrics fetch task * add API calls to pre-configure wallet framework stroage folders * cumulative updates for Cargo.lock * Improve address error messages for human readability * PrvKeyData enumeration, initial account transfer support (wip), wallet descriptor handling * wallet - change error to retain PrvKeyDataId instead of a String * propagate bip39 flag to runtime account and it's descriptor * wip - keydata refactoring * metrics refactoring (bandwidth) * wip (cleanup) * misc improvements to metrics * code formatting * wRPC bandwidth monitoring + scaffolding for global bandwidth metrics * bump WRS to 0.9.0 * metrics - add mempool_size and convert some items to u32/f32 * metrics cleanup * wallet and account rename, remove "title" from accounts * add missing mempool size metric * UTXO "stasis" mode for new coinbase transactions * remove useless Option from the rpc url parser * fix daa score in maturity state check * add name_or_id to PrvKeyDataInfo * wallet RPC Connect/Disconnect and GetStatus API calls * cleanup * support for 12-word bip39 mnemonics * add a hidden --daemon flag to kaspad * refactor get_daa_score_timestamp helper to return Vec<u64> instead of Response struct * Add Scan event * implement support for unixtime in transactions * kaspa value formatting functions * fix a deadlock due to re-entrant mutex lock when loading multiple accounts from storage * introduce utxo stasis mode + make balance carry utxo counts as well as the totals * refactor wallet events, replace reset() with close() where appropriate * manual debug impl for tx components (display script as hex, address as address etc) * fix incorrect gas assignment in WASM bindings * updates for wallet event handling * refactor sign_with_multiple to return Err(tx) when partially signed * refactor Storage and Wallet descriptors * fix account derivation index handling in DerivationCapable * track acceptance DAA stamp in outgoing transaction record * temporary harness for embedding script engine directly into tx generator (used for debugging) * convert PendingTransaction Utxo entries into HashSet * rename pending tx fns * outgoing tx wrapper around pending tx * refactor notification events during wallet ops * as_ref() for PrvKeyData * wip - initial RpcCoreMocks for wallet framework tests * wallet api messages update for StorageDescriptor usage * update AddressDerivationMeta to track addresses from index 0 * add AHash and async_channel to wallet_core crate * allow Args to be used externally for daemon argument pre-validation * impl Hash for UtxoEntryReference and rename "fake()" to "simulated()" * update wallet events * expose tx validator check_scripts() as pub * update wallet wasm fns for wallet and storage descriptors * rename kaspa-metrics to kaspa-metrics-core * code formatting * code formatting * migrate transaction console formatting to cli/extensions * refactor wallet events * use ahash for faster processing in wallet hash maps (utxo context) * wallet - add SendError handling for error::Error * wallet event refactoring, scan/discovery process refactoring, wallet change secret api handling * refactoring: transfer and deep derivation scan * transaction record refactoring * wallet - support for cross-account transfers in generator * wallet - outgoing transaction impl * wallet - refactor maturity handling * wallet/utxo - initial tests scaffolding * minor optimization of the discovery/scan process * wallet - utxo processor outgoing tx and discovery timestamp unixtime handling * refactor utxo context to use output transactions, refactor transfers, compound and change notifications * code formatting + Cargo.lock update * Signed enum to represent Fully or Partially signed tx * bind p2p and grpc bandwidth metrics * misc clippy fixes * cleanup * remove --daemon as Args::parse() offers external mutations * remove previously implemented mnemonic variant in favor of client-side mnemonic creation * remove AccessContextT, convert to wallet_secret, wip WalletApi traits, refactor account args * code formatting * separate runtime wallet into multiple modules * wallet - revert create_accounts to process a single account at a time * cleanup * wallet - JSON import and export + preliminary docs * address - AddressList - fix TryFrom deadlock caused by recursive trait invocation * wallet - api rustdoc + cleanup, transaction calls * wallet docs * change address debug formatting * adding cli rpc addpeer * wip - wallet restructuring * wip - restructure wallet modules * wip - wallet restructuring - cleanup * wip - AccountKind refactoring * wallet - refactoring storage serialization & encryption * fold in PR kaspanet#349 * wallet framework - misc helper fns and cleanup * storage unit tests + prv key data module restructuring * wallet - account factory registrations * Remove panics caused by sudden loss of connection while submitting transactions. * cleanup * cleanup * cleanup + docs * docs * adjust metrics formatting * cleanup * bump workflow-rs to 0.10.1 * fix wallet storage unit tests * fix generator unit tests * convert missing listener_id panic to error * docs * code formatting * lints * misc updates following github reviews * refactor AccountKind to use fixedstr and implement Copy * refactor AccountKind to use fixedstr (str64) and re-implement Copy, removing clone() where necessary * change js-bound fn names from *as* to *to* to follow Rust conventions * replace AssocPrvKeyDataIds iter().any() to contains() * remove vec![] in MetricsGroup iterator fn; add TODO comment on Metric::group() * scope magic numbers within their associated structs * add try_fully_signed() and try_partially_signed() fns to Signed that allow recovery of the tx * refactor local storage cache to use RwLock instead of a Mutex * refactor serialization as per github feedback (partial trait impl) * Optimize AssocPrvKeyDataIds to use Either in the IntoIter impl, removing allocations. * Refactor to bypass aux buffer creation when storing wallet on native platforms. * impl custom Debug for storage structs to display data in hex * adding misc SAFETY comments on unsafe ops * TODO comments for later SignerT+multisig refactoring * Change panic!() to unreachable!() to make it clearer that a state should not be reached. * rename TransactionRecord::unixtime to unixtime_msec * fix wrong references in comments * Fix min_cosigner_index handling in multisig fns * add comments to UtxoContext::insert() error handling * bind AccountStorable:Storable for AccountStorage, migrate Storable into a separate module, add comments to MAGIC values in some locations * make framework init functions unsafe * update workflow-rs to 0.10.2 * fix comments * fix data cloning when getting collection counts * adding TODO notes to wasm macros and KAS value decimal parsing --------- Co-authored-by: Surinder Singh Matoo <surinder83singh@gmail.com> commit 10d0ebf Author: D-Stacks <78099568+D-Stacks@users.noreply.github.com> Date: Tue Dec 26 10:43:59 2023 +0100 make step 3. of linux installation not fail copy and paste (kaspanet#363) remove the space to escape the newline instead. * reduce children cache policy * reduce caches 25% * increase tx cache * fix gd alloc and slightly increase other numbers * fix a cache alloc bug by redesigning the cache and adding a min units option + fix bps constants * fix simpa short cmd op name colision * move impls to inner struct * comments, docs and technical refactoring * refactor MemSizeEstimator * refactor bool to MemMode * memsize docs * remove unneeded min limits * comments and name enhancements * rename `CachePolicy::Units` to `CachePolicy::Count` to avoid confusion with `MemMode` * use estimated num of unique headers in validate pruning point headers cache * cache policy builder * add todos * median window has different size + comments * read ghostdag data from compact access if possible * remove wrong upper bound for reachability sets
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.