Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Platform SDK 0.7.0-beta.1, add payer records to state for deterministic duplicate classification #397

Merged
merged 46 commits into from
Aug 7, 2020

Conversation

tinker-michaelj
Copy link
Collaborator

@tinker-michaelj tinker-michaelj commented Aug 6, 2020

Issue(s):

Summary of the change:

  • Upgrade to Platform 0.7.0-beta.1.
    • Ensure a newly-copied ServicesState is mutable.
    • Replace all direct references to the primitive FCMaps with suppliers, e.g. in FCMapBackingAccounts (necessary because the first ServicesState injected into the ServicesContext will become immutable as soon as the platform calls copy).
    • Similarly, ensure the midnightRates are always retrieved from the current state when file 0.0.112 is updated.
    • Reduce FCMap touches by returning all MerkleAccount refs from FCMapBackingAccounts via getForModify; restore the mutable MerkleAccounts at the end of each transaction.
    • Improve performance by building the TransfersList imperatively instead of declaratively in HederaLedger.
  • Upon finalizing a txn record, the FeeChargingRecordHistorian now adds it to the payerRecords FCQueue of the transaction's effective payer.
  • The ExpiryManager tracks these records and purges them using the HederaLedger at the appropriate time.
  • A new TxnIdRecentHistory process object summarizes the "recent history" of a given TransactionID.
    • These relationships are kept in a txnHistories map in the context.
    • handleTransaction uses this history to classify the duplicate status of each txn as either BELIEVED_UNIQUE, DUPLICATE, or NODE_DUPLICATE; where the last implies the same node resubmitted a txn, and leads to the node being charge the network fee for the txn.
    • When the ExpiryManager purges a record, it updates the recent history accordingly.
    • Similarly, when the ExpiryManager rebuilds its expiry queues from a saved state, it reconstructs the recent history via the payerRecords in the state.
  • Fix a regression allowing contracts with admin keys to be used in a CryptoTransfer.
  • Add copyright headers to hapi-proto/*.proto files.
  • Isolate bootstrap-only and (most) global/static properties in a new bootstrap.properties resource; expose them through the StandardizedPropertySource.

tinker-michaelj added 30 commits July 21, 2020 00:03
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
…id tech debt when refactoring NodeAccountsCreation

Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
tinker-michaelj added 8 commits August 5, 2020 20:32
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
@codecov
Copy link

codecov bot commented Aug 6, 2020

Codecov Report

Merging #397 into master will increase coverage by 0.23%.
The diff coverage is 88.70%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #397      +/-   ##
============================================
+ Coverage     74.82%   75.05%   +0.23%     
- Complexity     3341     3364      +23     
============================================
  Files           290      293       +3     
  Lines         11789    11835      +46     
  Branches       1240     1255      +15     
============================================
+ Hits           8821     8883      +62     
+ Misses         2637     2605      -32     
- Partials        331      347      +16     
Impacted Files Coverage Δ Complexity Δ
...hedera/services/fees/calculation/FeeCalcUtils.java 85.71% <ø> (-3.76%) 5.00 <0.00> (-3.00)
...a/services/legacy/config/SyncPropertiesObject.java 78.66% <ø> (-2.82%) 14.00 <0.00> (-7.00)
...a/com/hedera/services/legacy/core/jproto/JKey.java 77.50% <ø> (+0.93%) 33.00 <0.00> (ø)
...era/services/legacy/service/FreezeServiceImpl.java 70.00% <0.00%> (+4.09%) 8.00 <1.00> (ø)
...vices/legacy/service/SmartContractServiceImpl.java 2.69% <0.00%> (ø) 3.00 <0.00> (ø)
...s/context/properties/DefaultPropertySanitizer.java 33.33% <0.00%> (-6.67%) 2.00 <0.00> (ø)
...vices/legacy/services/state/AwareProcessLogic.java 2.95% <0.00%> (-0.02%) 4.00 <0.00> (ø)
...vices/legacy/utils/TransactionValidationUtils.java 30.34% <0.00%> (ø) 22.00 <0.00> (ø)
...s/metadata/lookups/RetryingFCMapAccountLookup.java 95.55% <ø> (ø) 12.00 <0.00> (ø)
...a/services/sigs/verification/PrecheckVerifier.java 79.16% <ø> (-0.84%) 6.00 <0.00> (ø)
... and 114 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c18e47f...565e1d2. Read the comment docs.

@tinker-michaelj tinker-michaelj self-assigned this Aug 6, 2020
@tinker-michaelj tinker-michaelj added this to the Hedera 0.7.0 milestone Aug 6, 2020
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
ljianghedera and others added 3 commits August 7, 2020 10:18
Signed-off-by: ljianghedera <leo.jiang@hedera.com>
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
…dera-services into 0347-M-UpgradeToSdk070beta1
tinker-michaelj added 2 commits August 7, 2020 12:05
Signed-off-by: tinker-michaelj <michael.tinker@hedera.com>
…dera-services into 0347-M-UpgradeToSdk070beta1
@@ -157,7 +158,6 @@ private boolean isTreasury() {
try {
return Optional.of(ExchangeRateSet.parseFrom(data));
} catch (InvalidProtocolBufferException ignore) {
log.warn(ignore.getMessage());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I added this just for solving sonar cube errors.

@tinker-michaelj tinker-michaelj merged commit 9c2edfa into master Aug 7, 2020
@tinker-michaelj tinker-michaelj deleted the 0347-M-UpgradeToSdk070beta1 branch August 7, 2020 21:05
triggers:
- schedule:
cron: "0 5,9,13 * * *"
cron: "0 16,21 * * *"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When merging, this changes need to be restored to original.

filters:
branches:
only:
- master
- 0347-M-UpgradeToSdk070beta1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here

@@ -1356,6 +1356,7 @@ workflows:
ignore:
- /.*-PERF/
- /.*-REGRESSION/
- 0347-M-UpgradeToSdk070beta1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants