-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
consumed_by_tx_id
has invalid value for byron tx_out entries
#1821
Comments
consumed_by_tx_id
has invalid value for some byron tx IDsconsumed_by_tx_id
has invalid value for byron tx_out entries
Indeed, that's a bug that probablly affects all Byron consumed entries. We can prepare a tag that fixes it. |
Thanks for the quick turnaround and fix , and getting a tag or minor release - I am happy with the change and we can resync from genesis (as 1-time resync and then snapshot to replicate to others is simple enough). Just noting from tag/release notes pov that this is breaking and not [atleast yet] handled by migration, mentioning this since I have seen responses to queries that dbsync does not require resync, while there have been re-sync required between the tags - atleast if not using config options. I think we had one at 13.2.x , and ever since - have stuck to re-synching from genesis, unless the delta is quite small or new data is inserted w/o changes to existing columns. |
This has been fixed and tag 13.4.0.2 doesn't have the problem. It require a resync to fix older data as no migration is provided In general the numbering of the releases follow changes to the default schema. If an upgrade caused wrong old values, that was probably due to a bug in the migration process. |
Tested the version of fix with migration: #1827 |
This fix is now 13.4.1.0.
|
I guess this is the warning message which initially I wanted to report but then realized that it is expected in case of stopping in the middle of fixing and starting service again: [db-sync-node:Info:95] [2024-08-27 10:09:33.79 UTC] Fixed 2003511/2065090 entries
[db-sync-node:Warning:95] [2024-08-27 10:10:33.85 UTC] While fixing block ByronBlock {byronBlockRaw = ABOBBlock (ABlock {blockHeader = AHeader {aHeaderProtocolMagicId = Annotated {unAnnotated = ProtocolMagicId {unProtocolMagicId = 764824073}, annotation = "\SUB-\150J\t"}, aHeaderPrevHash = Annotated {unAnnotated = 2aa5103f79c14d5d2cb704223965f7a3f23bdfeb0105737fffda54ad422d1965, annotation = "X *\165\DLE?y\193M],\183\EOT\"9e\247\163\242;\223\235\SOH\ENQs\DEL\255\218T\173B-\EMe"}, aHeaderSlot = Annotated {unAnnotated = SlotNumber {unSlotNumber = 2541752}, annotation = "\130\CANu\EM8\216"}, aHeaderDifficulty = Annotated {unAnnotated = ChainDifficulty {unChainDifficulty = 2540244}, annotation = "\129\SUB\NUL&\194\212"}, headerProtocolVersion = 0.2.0, headerSoftwareVersion = cardano-sl:1, aHeaderProof = Annotated {unAnnotated = Proof {proofUTxO = TxProof {txpNumber = 1, txpRoot = MerkleRoot {getMerkleRoot = f5d2d9331f61f100e486295d5a2adc8fccb39f467e20f07915e49af65b9e1e9a}, txpWitnessesHash = 7a469f952e7eaa5c5337794680e0565b3d47629f9efbea98cdfd6d7868136a81}, proofSsc = SscProof, proofDelegation = afc0da64183bf2664f3d4eec7238d524ba607faeeab24fc100eb861dba69971b, proofUpdate = 4e66280cd94d591072349bec0a3090a53aa945562efb6d08d56e53654b0e4098}, annotation = "\132\131\SOHX \245\210\217\&3\USa\241\NUL\228\134)]Z*\220\143\204\179\159F~ \240y\NAK\228\154\246[\158\RS\154X zF\159\149.~\170\\S7yF\128\224V[=Gb\159\158\251\234\152\205\253mxh\DC3j\129\130\ETXX \211j&\EM\166rIF\EOT\225\ESC\180G\203\207R1\233\242\186%\194\SYN\145w\237\201A\189P\173lX \175\192\218d\CAN;\242fO=N\236r8\213$\186`\DEL\174\234\178O\193\NUL\235\134\GS\186i\151\ESCX Nf(\f\217MY\DLEr4\155\236\n0\144\165:\169EV.\251m\b\213nSeK\SO@\152"}, headerGenesisKey = VerificationKey {unVerificationKey = XPub {xpubPublicKey = "\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169", xpubChaincode = ChainCode "\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216"}}, headerSignature = ABlockSignature {delegationCertificate = UnsafeACertificate {aEpoch = Annotated {unAnnotated = EpochNumber {getEpochNumber = 0}, annotation = "\NUL"}, issuerVK = VerificationKey {unVerificationKey = XPub {xpubPublicKey = "\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169", xpubChaincode = ChainCode "\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216"}}, delegateVK = VerificationKey {unVerificationKey = XPub {xpubPublicKey = "_\221\238\218\222'\DC4\214\219/\158\DC1\EOTt=-\141\129\142\205\220\&0n\ETBa\b\219\DC4\202\173\212A", xpubChaincode = ChainCode "\180W\213\132\f`\248\132\v\153\200\247\140)\n\226)\212\248C\RSg\139\167\165E\195V\a\185M\219"}}, signature = Signature (XSignature {unXSignature = "U'A\247(\EMnb\242\CAN\EOT{\148K$\206M7C\NUL\208K\155(\DC4&\245Z\160\NUL\213=\237f\152\154\213\234\t\b\230\255d\146\NUL\US\241\142\206lp@\169\&4\ACK\aY\233\174\t\134;\242\ETX"}), annotation = "\132\NULX@\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216X@_\221\238\218\222'\DC4\214\219/\158\DC1\EOTt=-\141\129\142\205\220\&0n\ETBa\b\219\DC4\202\173\212A\180W\213\132\f`\248\132\v\153\200\247\140)\n\226)\212\248C\RSg\139\167\165E\195V\a\185M\219X@U'A\247(\EMnb\242\CAN\EOT{\148K$\206M7C\NUL\208K\155(\DC4&\245Z\160\NUL\213=\237f\152\154\213\234\t\b\230\255d\146\NUL\US\241\142\206lp@\169\&4\ACK\aY\233\174\t\134;\242\ETX"}, signature = Signature (XSignature {unXSignature = "\187\171\157S*\255\228\194\148\234\173\CAN^\129\155\204\156#\ESC\215\157C\169L\242O\SI!\n\210\SOa/\182\SO\212\ENQ\191x\226\160\158<H\218L1i\129\170\172\166\GS\138\210P\160@\233I\128\t\FS\ACK"})}, headerAnnotation = "\133\SUB-\150J\tX *\165\DLE?y\193M],\183\EOT\"9e\247\163\242;\223\235\SOH\ENQs\DEL\255\218T\173B-\EMe\132\131\SOHX \245\210\217\&3\USa\241\NUL\228\134)]Z*\220\143\204\179\159F~ \240y\NAK\228\154\246[\158\RS\154X zF\159\149.~\170\\S7yF\128\224V[=Gb\159\158\251\234\152\205\253mxh\DC3j\129\130\ETXX \211j&\EM\166rIF\EOT\225\ESC\180G\203\207R1\233\242\186%\194\SYN\145w\237\201A\189P\173lX \175\192\218d\CAN;\242fO=N\236r8\213$\186`\DEL\174\234\178O\193\NUL\235\134\GS\186i\151\ESCX Nf(\f\217MY\DLEr4\155\236\n0\144\165:\169EV.\251m\b\213nSeK\SO@\152\132\130\CANu\EM8\216X@\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216\129\SUB\NUL&\194\212\130\STX\130\132\NULX@\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216X@_\221\238\218\222'\DC4\214\219/\158\DC1\EOTt=-\141\129\142\205\220\&0n\ETBa\b\219\DC4\202\173\212A\180W\213\132\f`\248\132\v\153\200\247\140)\n\226)\212\248C\RSg\139\167\165E\195V\a\185M\219X@U'A\247(\EMnb\242\CAN\EOT{\148K$\206M7C\NUL\208K\155(\DC4&\245Z\160\NUL\213=\237f\152\154\213\234\t\b\230\255d\146\NUL\US\241\142\206lp@\169\&4\ACK\aY\233\174\t\134;\242\ETXX@\187\171\157S*\255\228\194\148\234\173\CAN^\129\155\204\156#\ESC\215\157C\169L\242O\SI!\n\210\SOa/\182\SO\212\ENQ\191x\226\160\158<H\218L1i\129\170\172\166\GS\138\210P\160@\233I\128\t\FS\ACK\132\131\NUL\STX\NUL\130jcardano-sl\SOH\160X K\169*\163 \198\n\204\154\215\185\166O.\218U\196\210\236(\230\EOT\250\241\134p\139O\fN\142\223", headerExtraAnnotation = "\132\131\NUL\STX\NUL\130jcardano-sl\SOH\160X K\169*\163 \198\n\204\154\215\185\166O.\218U\196\210\236(\230\EOT\250\241\134p\139O\fN\142\223"}, blockBody = ABody {bodyTxPayload = ATxPayload {aUnTxPayload = [ATxAux {aTaTx = Annotated {unAnnotated = UnsafeTx {txInputs = TxInUtxo 637279e61188c847f991743c5530c6d83e167f47a57000bdf52f3076367c2491 0 :| [], txOutputs = TxOut {txOutAddress = Address {addrRoot = 782ed82aad1439d33a20c8c739425b52b1944ea8b5d91cc8a3cf900f, addrAttributes = Attributes { data_ = AddrAttributes {aaVKDerivationPath = Just (HDAddressPayload {getHDAddressPayload = "\202>U<\156c\197\195\STX\140\212C\128H\190\193t\133\159\134\DEL\US\ETB\206\143g\164e"}), aaNetworkMagic = NetworkMainOrStage} }, addrType = ATVerKey}, txOutValue = Lovelace 2783038327581} :| [TxOut {txOutAddress = Address {addrRoot = 6789efcd83371bc0eb2fcaaddf53f78535c6e2bcbbcb40f2af8d455e, addrAttributes = Attributes { data_ = AddrAttributes {aaVKDerivationPath = Just (HDAddressPayload {getHDAddressPayload = "\ENQ\182\164\201\FS\236\242\r.\139\232o\238\227\239O5\132\253\DEL/\194w\242\235\221\186\230"}), aaNetworkMagic = NetworkMainOrStage} }, addrType = ATVerKey}, txOutValue = Lovelace 290903095}], txAttributes = Attributes { data_ = () }}, annotation = "\131\159\130\NUL\216\CANX$\130X cry\230\DC1\136\200G\249\145t<U0\198\216>\SYN\DELG\165p\NUL\189\245/0v6|$\145\NUL\255\159\130\130\216\CANXB\131X\FSx.\216*\173\DC49\211: \200\199\&9B[R\177\148N\168\181\217\FS\200\163\207\144\SI\161\SOHX\RSX\FS\202>U<\156c\197\195\STX\140\212C\128H\190\193t\133\159\134\DEL\US\ETB\206\143g\164e\NUL\SUB\182\DC3\ETB\n\ESC\NUL\NUL\STX\135\250\STX\203\GS\130\130\216\CANXB\131X\FSg\137\239\205\131\&7\ESC\192\235/\202\173\223S\247\133\&5\198\226\188\187\203@\242\175\141E^\161\SOHX\RSX\FS\ENQ\182\164\201\FS\236\242\r.\139\232o\238\227\239O5\132\253\DEL/\194w\242\235\221\186\230\NUL\SUB\146\217^\175\SUB\DC1V\212\&7\255\160"}, aTaWitness = Annotated {unAnnotated = [VKWitness (VerificationKey {unVerificationKey = XPub {xpubPublicKey = "\ENQ\237rlwj>\157\206SIB-\162XO\203\149\165\t,\162\153k\NAK+:\161j\239\RS\DC3", xpubChaincode = ChainCode "\162\ESC=uW\243\&6\154\204\216/\204\r(\FS-O\210\149\186,\192eGuj\205\&6<\189\b\195"}}) (Signature (XSignature {unXSignature = "\155\141\r\162_+\182\215\ETX\ETX\177\241\219\174\147\221\f\SO\178\230(\153:q\233\189-!/\219H\249G\199i\DC3\147\247$k\181\201\175\142\&2\168Km]RF\CAN\225\214#\189\193\STX~H\235\147\227\ENQ"}))], annotation = "\129\130\NUL\216\CANX\133\130X@\ENQ\237rlwj>\157\206SIB-\162XO\203\149\165\t,\162\153k\NAK+:\161j\239\RS\DC3\162\ESC=uW\243\&6\154\204\216/\204\r(\FS-O\210\149\186,\192eGuj\205\&6<\189\b\195X@\155\141\r\162_+\182\215\ETX\ETX\177\241\219\174\147\221\f\SO\178\230(\153:q\233\189-!/\219H\249G\199i\DC3\147\247$k\181\201\175\142\&2\168Km]RF\CAN\225\214#\189\193\STX~H\235\147\227\ENQ"}, aTaAnnotation = "\130\131\159\130\NUL\216\CANX$\130X cry\230\DC1\136\200G\249\145t<U0\198\216>\SYN\DELG\165p\NUL\189\245/0v6|$\145\NUL\255\159\130\130\216\CANXB\131X\FSx.\216*\173\DC49\211: \200\199\&9B[R\177\148N\168\181\217\FS\200\163\207\144\SI\161\SOHX\RSX\FS\202>U<\156c\197\195\STX\140\212C\128H\190\193t\133\159\134\DEL\US\ETB\206\143g\164e\NUL\SUB\182\DC3\ETB\n\ESC\NUL\NUL\STX\135\250\STX\203\GS\130\130\216\CANXB\131X\FSg\137\239\205\131\&7\ESC\192\235/\202\173\223S\247\133\&5\198\226\188\187\203@\242\175\141E^\161\SOHX\RSX\FS\ENQ\182\164\201\FS\236\242\r.\139\232o\238\227\239O5\132\253\DEL/\194w\242\235\221\186\230\NUL\SUB\146\217^\175\SUB\DC1V\212\&7\255\160\129\130\NUL\216\CANX\133\130X@\ENQ\237rlwj>\157\206SIB-\162XO\203\149\165\t,\162\153k\NAK+:\161j\239\RS\DC3\162\ESC=uW\243\&6\154\204\216/\204\r(\FS-O\210\149\186,\192eGuj\205\&6<\189\b\195X@\155\141\r\162_+\182\215\ETX\ETX\177\241\219\174\147\221\f\SO\178\230(\153:q\233\189-!/\219H\249G\199i\DC3\147\247$k\181\201\175\142\&2\168Km]RF\CAN\225\214#\189\193\STX~H\235\147\227\ENQ"}]}, bodySscPayload = SscPayload, bodyDlgPayload = UnsafeAPayload {getPayload = [], getAnnotation = "\159\255"}, bodyUpdatePayload = APayload {payloadProposal = Nothing, payloadVotes = [], payloadAnnotation = "\130\128\159\255"}}, blockAnnotation = "\131\133\SUB-\150J\tX *\165\DLE?y\193M],\183\EOT\"9e\247\163\242;\223\235\SOH\ENQs\DEL\255\218T\173B-\EMe\132\131\SOHX \245\210\217\&3\USa\241\NUL\228\134)]Z*\220\143\204\179\159F~ \240y\NAK\228\154\246[\158\RS\154X zF\159\149.~\170\\S7yF\128\224V[=Gb\159\158\251\234\152\205\253mxh\DC3j\129\130\ETXX \211j&\EM\166rIF\EOT\225\ESC\180G\203\207R1\233\242\186%\194\SYN\145w\237\201A\189P\173lX \175\192\218d\CAN;\242fO=N\236r8\213$\186`\DEL\174\234\178O\193\NUL\235\134\GS\186i\151\ESCX Nf(\f\217MY\DLEr4\155\236\n0\144\165:\169EV.\251m\b\213nSeK\SO@\152\132\130\CANu\EM8\216X@\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216\129\SUB\NUL&\194\212\130\STX\130\132\NULX@\v\219\US^\243\217\148\ETXu\147\242&bU\241\&4\165de\139\178\223\129K;\156\239\185m\163O\169\200\136Y\FS\133\183p\253\&6rm_=\153\FSf\136(\175\252{\190\br\253i\145\&6\230d\217\216X@_\221\238\218\222'\DC4\214\219/\158\DC1\EOTt=-\141\129\142\205\220\&0n\ETBa\b\219\DC4\202\173\212A\180W\213\132\f`\248\132\v\153\200\247\140)\n\226)\212\248C\RSg\139\167\165E\195V\a\185M\219X@U'A\247(\EMnb\242\CAN\EOT{\148K$\206M7C\NUL\208K\155(\DC4&\245Z\160\NUL\213=\237f\152\154\213\234\t\b\230\255d\146\NUL\US\241\142\206lp@\169\&4\ACK\aY\233\174\t\134;\242\ETXX@\187\171\157S*\255\228\194\148\234\173\CAN^\129\155\204\156#\ESC\215\157C\169L\242O\SI!\n\210\SOa/\182\SO\212\ENQ\191x\226\160\158<H\218L1i\129\170\172\166\GS\138\210P\160@\233I\128\t\FS\ACK\132\131\NUL\STX\NUL\130jcardano-sl\SOH\160X K\169*\163 \198\n\204\154\215\185\166O.\218U\196\210\236(\230\EOT\250\241\134p\139O\fN\142\223\132\159\130\131\159\130\NUL\216\CANX$\130X cry\230\DC1\136\200G\249\145t<U0\198\216>\SYN\DELG\165p\NUL\189\245/0v6|$\145\NUL\255\159\130\130\216\CANXB\131X\FSx.\216*\173\DC49\211: \200\199\&9B[R\177\148N\168\181\217\FS\200\163\207\144\SI\161\SOHX\RSX\FS\202>U<\156c\197\195\STX\140\212C\128H\190\193t\133\159\134\DEL\US\ETB\206\143g\164e\NUL\SUB\182\DC3\ETB\n\ESC\NUL\NUL\STX\135\250\STX\203\GS\130\130\216\CANXB\131X\FSg\137\239\205\131\&7\ESC\192\235/\202\173\223S\247\133\&5\198\226\188\187\203@\242\175\141E^\161\SOHX\RSX\FS\ENQ\182\164\201\FS\236\242\r.\139\232o\238\227\239O5\132\253\DEL/\194w\242\235\221\186\230\NUL\SUB\146\217^\175\SUB\DC1V\212\&7\255\160\129\130\NUL\216\CANX\133\130X@\ENQ\237rlwj>\157\206SIB-\162XO\203\149\165\t,\162\153k\NAK+:\161j\239\RS\DC3\162\ESC=uW\243\&6\154\204\216/\204\r(\FS-O\210\149\186,\192eGuj\205\&6<\189\b\195X@\155\141\r\162_+\182\215\ETX\ETX\177\241\219\174\147\221\f\SO\178\230(\153:q\233\189-!/\219H\249G\199i\DC3\147\247$k\181\201\175\142\&2\168Km]RF\CAN\225\214#\189\193\STX~H\235\147\227\ENQ\255\130\ETX\217\SOH\STX\128\159\255\130\128\159\255\129\160"}), byronBlockSlotNo = SlotNo 2541752, byronBlockHash = ByronHash {unByronHash = eae3d3f8a772d17f2e3e543874b5a411f008682e859407b490ad08e10997d3ce}}, encountered error Error SNErrDefault: "resolving tx tx hash \"cf4b6c26a260cae1235d8025128598eba06a52089cc97c62942c2c783bfa2b4a\""
[db-sync-node:Info:95] [2024-08-27 10:10:34.01 UTC] Fixed (2065090,Nothing) consumed_by_tx_id wrong entries What was initially worrying was the end of warning message:
but after starting service and checking data things looked OK. Maybe Information about those edge cases of the fix should be added to the changelog of pre-release ? |
OS
Your OS: Linux (Debian 12)
Versions
The
db-sync
version (egcardano-db-sync --version
): cardano-db-sync 13.4.0.1PostgreSQL version: 16.3
Build/Install Method
The method you use to build or install
cardano-db-sync
: cabalRun method
The method you used to run
cardano-db-sync
(eg Nix/Docker/systemd/none): systemdAdditional context
We are using these config options for dbsync , particularly
tx_out
hasconsumed
value set. These are applied across Koios instances (i.e. ~17 core instances + community ones), and we share snapshots with these options hosted ourselves. Thus, for the data integrity issue below, we're hoping to fix the values and re-host a snapshot with correct history.Problem Report
For Byron tx_out entries, the
tx_out
=>consumed_by_tx_id
is incorrect. Consider transactions for addressDdzFFzCqrhsvgjTLbbZNqacNLQpacoVnHM3AB5r8MiCHSnHx9ebH7Gjizg1KCt8Ba7hDmjA9q1gexs4yq787AQ2kCznVVqyAZNLxaMHr
(notice how consumed_by and tx_id are same):Thus, we're not able to check where this address is used as input:
For those who are not using
consumed
flag, thetx_in
did contain right transactions (eg: see adastat showing 4 transactions here ).Ideally, we'd want this tagged (even if not released) so that the binary built can be used consistently across instances and snapshots we end up creating with fixed data are compatible.
We verified that this is true only for Byron-era:
Thanks to @hodlonaut for testing and providing query results
The text was updated successfully, but these errors were encountered: