Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

latest firehose release crashing in engine_newPayloadV2 #8

Closed
sduchesneau opened this issue Sep 5, 2023 · 2 comments
Closed

latest firehose release crashing in engine_newPayloadV2 #8

sduchesneau opened this issue Sep 5, 2023 · 2 comments

Comments

@sduchesneau
Copy link

System information

Geth version: release v1.12.1-fh2.3
OS & Version: Linux

Logs

ERROR[09-05|10:50:45.166] RPC method engine_newPayloadV2 crashed: should have been call in a transaction, something is deeply wrong
goroutine 7642 [running]:
github.com/ethereum/go-ethereum/rpc.(*callback).call.func1()
	github.com/ethereum/go-ethereum/rpc/service.go:199 +0x89
panic({0x17ebf60, 0x209b720})
	runtime/panic.go:884 +0x213
github.com/ethereum/go-ethereum/firehose.(*Context).callIndex(0x7fb252cfcf18?)
	github.com/ethereum/go-ethereum/firehose/context.go:512 +0x4c
github.com/ethereum/go-ethereum/firehose.(*Context).RecordBalanceChange(0xc0bfb3c240, {0xb9, 0xd7, 0x93, 0x48, 0x78, 0xb5, 0xfb, 0x96, 0x10, ...}, ...)
	github.com/ethereum/go-ethereum/firehose/context.go:685 +0x91
github.com/ethereum/go-ethereum/core/state.(*stateObject).SetBalance(0xc03d550fc0, 0xc05bb854e0, 0x8f1e445e8afeb310?, {0x1ad8511?, 0xc0bf4dda40?})
	github.com/ethereum/go-ethereum/core/state/state_object.go:433 +0x88
github.com/ethereum/go-ethereum/core/state.(*stateObject).AddBalance(0xc03d550fc0, 0xc0bfcf5890, 0xc03f297ead?, {0x1ad8511, 0xa})
	github.com/ethereum/go-ethereum/core/state/state_object.go:419 +0x12c
github.com/ethereum/go-ethereum/core/state.(*StateDB).AddBalance(0xf0d2c4?, {0xb9, 0xd7, 0x93, 0x48, 0x78, 0xb5, 0xfb, 0x96, 0x10, ...}, ...)
	github.com/ethereum/go-ethereum/core/state/statedb.go:423 +0x6f
github.com/ethereum/go-ethereum/consensus/beacon.(*Beacon).Finalize(0xc0002b8130, {0x20b2740, 0xc01f571000}, 0xc0bee50b40, 0xc0bfcf5a60?, {0xc01df81680, 0x89, 0x89}, {0x2de5310, 0x0, ...}, ...)
	github.com/ethereum/go-ethereum/consensus/beacon/consensus.go:355 +0x2a7
github.com/ethereum/go-ethereum/core.(*StateProcessor).Process(0xc0002ab3a0, 0xc0bbb52e60, 0xc0bf4dda40, {{0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
	github.com/ethereum/go-ethereum/core/state_processor.go:141 +0x103a
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0xc01f571000, {0xc027ef9950?, 0x1, 0x1}, 0x0)
	github.com/ethereum/go-ethereum/core/blockchain.go:1813 +0x20f0
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertBlockWithoutSetHead(0xc01f571000, 0xc0bbb52e60)
	github.com/ethereum/go-ethereum/core/blockchain.go:2338 +0xcd
github.com/ethereum/go-ethereum/eth/catalyst.(*ConsensusAPI).newPayload(_, {{0xbc, 0x94, 0x46, 0x9e, 0x52, 0x8b, 0x13, 0x95, 0x1f, ...}, ...}, ...)
	github.com/ethereum/go-ethereum/eth/catalyst/api.go:555 +0x9ee
github.com/ethereum/go-ethereum/eth/catalyst.(*ConsensusAPI).NewPayloadV2(_, {{0xbc, 0x94, 0x46, 0x9e, 0x52, 0x8b, 0x13, 0x95, 0x1f, ...}, ...})
	github.com/ethereum/go-ethereum/eth/catalyst/api.go:455 +0x347
reflect.Value.call({0xc01697cc00?, 0xc0169dc298?, 0x7fb1e07cceb8?}, {0x1acf1e1, 0x4}, {0xc0bd823b80, 0x2, 0x0?})
	reflect/value.go:586 +0xb0b
reflect.Value.Call({0xc01697cc00?, 0xc0169dc298?, 0x1?}, {0xc0bd823b80?, 0x1?, 0x16?})
	reflect/value.go:370 +0xbc
github.com/ethereum/go-ethereum/rpc.(*callback).call(0xc0169e2cc0, {0x20aebd8?, 0xc0bd823a90}, {0xc058271ed8, 0x13}, {0xc0bf4c5d40, 0x1, 0x4e8bb7?})
	github.com/ethereum/go-ethereum/rpc/service.go:205 +0x3c5
github.com/ethereum/go-ethereum/rpc.(*handler).runMethod(0xc0c0400000?, {0x20aebd8?, 0xc0bd823a90?}, 0xc0bf4d8540, 0x1?, {0xc0bf4c5d40?, 0x42b9d0?, 0x18b5ec0?})
	github.com/ethereum/go-ethereum/rpc/handler.go:565 +0x45
github.com/ethereum/go-ethereum/rpc.(*handler).handleCall(0xc0bbb52c80, 0xc0bf4fc0f0, 0xc0bf4d8540)
	github.com/ethereum/go-ethereum/rpc/handler.go:512 +0x239
github.com/ethereum/go-ethereum/rpc.(*handler).handleCallMsg(0xc0bbb52c80, 0xc0bf4fc150?, 0xc0bf4d8540)
	github.com/ethereum/go-ethereum/rpc/handler.go:470 +0x237
github.com/ethereum/go-ethereum/rpc.(*handler).handleNonBatchCall(0xc0bbb52c80, 0xc0bf4fc0f0, 0xc0bf4d8540)
	github.com/ethereum/go-ethereum/rpc/handler.go:296 +0x1ae
github.com/ethereum/go-ethereum/rpc.(*handler).handleMsg.func1.1(0x20aebd8?)
	github.com/ethereum/go-ethereum/rpc/handler.go:269 +0x27
github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc.func1()
	github.com/ethereum/go-ethereum/rpc/handler.go:387 +0xc5
created by github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc
	github.com/ethereum/go-ethereum/rpc/handler.go:383 +0x8d

WARN [09-05|10:50:45.166] Served engine_newPayloadV2               conn=10.1.144.7:44968 reqid=1 duration=106.636873ms err="method handler crashed"
@sduchesneau
Copy link
Author

INFO [09-05|11:02:49.089] Importing sidechain segment              start=18,071,013 end=18,071,014
goroutine 4609 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
	runtime/debug/stack.go:16 +0x19
github.com/ethereum/go-ethereum/firehose.(*Context).callIndex(0x7ff257428f18?)
	github.com/ethereum/go-ethereum/firehose/context.go:511 +0x39
github.com/ethereum/go-ethereum/firehose.(*Context).RecordBalanceChange(0xc0138f2a20, {0xb9, 0xd7, 0x93, 0x48, 0x78, 0xb5, 0xfb, 0x96, 0x10, ...}, ...)
	github.com/ethereum/go-ethereum/firehose/context.go:685 +0x91
github.com/ethereum/go-ethereum/core/state.(*stateObject).SetBalance(0xc01de45180, 0xc01e9b10e0, 0x8f1e445e8afeb310?, {0x1ad8511?, 0xc012b72a80?})
	github.com/ethereum/go-ethereum/core/state/state_object.go:433 +0x88
github.com/ethereum/go-ethereum/core/state.(*stateObject).AddBalance(0xc01de45180, 0xc01709f118, 0xc03f297ead?, {0x1ad8511, 0xa})
	github.com/ethereum/go-ethereum/core/state/state_object.go:419 +0x12c
github.com/ethereum/go-ethereum/core/state.(*StateDB).AddBalance(0xf20fd5?, {0xb9, 0xd7, 0x93, 0x48, 0x78, 0xb5, 0xfb, 0x96, 0x10, ...}, ...)
	github.com/ethereum/go-ethereum/core/state/statedb.go:423 +0x6f
github.com/ethereum/go-ethereum/consensus/beacon.(*Beacon).Finalize(0xc018252260, {0x20b2740, 0xc019020800}, 0xc0138fa480, 0xc01709f2e8?, {0xc012702f00, 0x99, 0x99}, {0x2de5310, 0x0, ...}, ...)
	github.com/ethereum/go-ethereum/consensus/beacon/consensus.go:355 +0x2a7
github.com/ethereum/go-ethereum/core.(*StateProcessor).Process(0xc0009af4e0, 0xc012717040, 0xc012b72a80, {{0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
	github.com/ethereum/go-ethereum/core/state_processor.go:141 +0x103a
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0xc019020800, {0xc01364c410?, 0x2, 0x2}, 0x1)
	github.com/ethereum/go-ethereum/core/blockchain.go:1813 +0x20f0
github.com/ethereum/go-ethereum/core.(*BlockChain).insertSideChain(0xc019020800, 0x1b111e3?, 0xc01249d3a0)
	github.com/ethereum/go-ethereum/core/blockchain.go:2086 +0x19f3
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0xc019020800, {0xc0135fa4e0?, 0x1, 0x1}, 0x1)
	github.com/ethereum/go-ethereum/core/blockchain.go:1666 +0x98e
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertChain(0xc019020800, {0xc0135fa4e0?, 0x1, 0x1})
	github.com/ethereum/go-ethereum/core/blockchain.go:1568 +0xb0e
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).importBlockResults(0xc004d7bba0, {0xc0135fa4d8, 0x1, 0x30?})
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1559 +0x54e
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFullSyncContent(0xc004d7bba0, 0x0, 0x1)
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1526 +0x17c
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func9()
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:643 +0x25
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1()
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:655 +0x70
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:655 +0x68
panic: should have been call in a transaction, something is deeply wrong

goroutine 4609 [running]:
github.com/ethereum/go-ethereum/firehose.(*Context).callIndex(0x7ff257428f18?)
	github.com/ethereum/go-ethereum/firehose/context.go:512 +0x4c
github.com/ethereum/go-ethereum/firehose.(*Context).RecordBalanceChange(0xc0138f2a20, {0xb9, 0xd7, 0x93, 0x48, 0x78, 0xb5, 0xfb, 0x96, 0x10, ...}, ...)
	github.com/ethereum/go-ethereum/firehose/context.go:685 +0x91
github.com/ethereum/go-ethereum/core/state.(*stateObject).SetBalance(0xc01de45180, 0xc01e9b10e0, 0x8f1e445e8afeb310?, {0x1ad8511?, 0xc012b72a80?})
	github.com/ethereum/go-ethereum/core/state/state_object.go:433 +0x88
github.com/ethereum/go-ethereum/core/state.(*stateObject).AddBalance(0xc01de45180, 0xc01709f118, 0xc03f297ead?, {0x1ad8511, 0xa})
	github.com/ethereum/go-ethereum/core/state/state_object.go:419 +0x12c
github.com/ethereum/go-ethereum/core/state.(*StateDB).AddBalance(0xf20fd5?, {0xb9, 0xd7, 0x93, 0x48, 0x78, 0xb5, 0xfb, 0x96, 0x10, ...}, ...)
	github.com/ethereum/go-ethereum/core/state/statedb.go:423 +0x6f
github.com/ethereum/go-ethereum/consensus/beacon.(*Beacon).Finalize(0xc018252260, {0x20b2740, 0xc019020800}, 0xc0138fa480, 0xc01709f2e8?, {0xc012702f00, 0x99, 0x99}, {0x2de5310, 0x0, ...}, ...)
	github.com/ethereum/go-ethereum/consensus/beacon/consensus.go:355 +0x2a7
github.com/ethereum/go-ethereum/core.(*StateProcessor).Process(0xc0009af4e0, 0xc012717040, 0xc012b72a80, {{0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
	github.com/ethereum/go-ethereum/core/state_processor.go:141 +0x103a
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0xc019020800, {0xc01364c410?, 0x2, 0x2}, 0x1)
	github.com/ethereum/go-ethereum/core/blockchain.go:1813 +0x20f0
github.com/ethereum/go-ethereum/core.(*BlockChain).insertSideChain(0xc019020800, 0x1b111e3?, 0xc01249d3a0)
	github.com/ethereum/go-ethereum/core/blockchain.go:2086 +0x19f3
github.com/ethereum/go-ethereum/core.(*BlockChain).insertChain(0xc019020800, {0xc0135fa4e0?, 0x1, 0x1}, 0x1)
	github.com/ethereum/go-ethereum/core/blockchain.go:1666 +0x98e
github.com/ethereum/go-ethereum/core.(*BlockChain).InsertChain(0xc019020800, {0xc0135fa4e0?, 0x1, 0x1})
	github.com/ethereum/go-ethereum/core/blockchain.go:1568 +0xb0e
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).importBlockResults(0xc004d7bba0, {0xc0135fa4d8, 0x1, 0x30?})
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1559 +0x54e
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFullSyncContent(0xc004d7bba0, 0x0, 0x1)
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1526 +0x17c
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func9()
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:643 +0x25
github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1()
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:655 +0x70
created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
	github.com/ethereum/go-ethereum/eth/downloader/downloader.go:655 +0x68

@sduchesneau
Copy link
Author

fixed by #9

maoueh pushed a commit that referenced this issue Apr 29, 2024
This commit squashes the op-geth fork history into a more maintainable
diff for rebasing upon upstream geth.

reference-optimistic-geth changes (origins of op-geth in early Bedrock
development stage):
- Deposit TX Type
- Enable deposit tx in EVM/tx pool
- Change deposit nonce to not be the max nonce
- Extend PayloadAttributesV1 with a Transactions field
- Force deposits at the start of each L2 block
- Fix height check
- noTxPool flag, reproduce block in verifier mode without tx pool interference
- Fix RPC json marshalling (ref op-geth PR 4)
- Deposit txs block height check in block body validation (ref op-geth PR 5)
- core: do not try to reinject deposit txs into tx-pool (ref-op-geth PR 6)
- deposit source hash field instead of L2 block height and tx index combination
- Include invalid deposits, rewind state, but always persist mint (#10)
- Provide gas to Call/Create in deposit transactions (#12)
- Add docker builds (ref-op-geth PR 16, 17)
- Don't panic on deposit transaction signature values or chain ID (ref-op-geth PR 18)
- core: Add version to DepositTx (ref-op-geth PR 19)
- Enable Geth build/lint/test in CircleCI (ref-op-geth PR 23)
- core: Include guaranteed gas in the gas pool (ref-op-geth PR 21)
- core: handle base fee, l1 availability fee, tx fee (ref-op-geth PR 27)
- fix: deposit tx hash
- fix l1 fee cache, rpc, tracing and tx pool
- core: remove deposit-tx sub-type (a.k.a. deposit version byte)
- eth/catalyst: allow engine user to reorg own chain
- miner: restore ability to reorg deep as block builder
- params: print Optimism consensus type in banner
- core/types: remove unused protected() method, see upstream PR 23376
- core: do not mutate original balance value in tx pool l1 cost adjustment
- core: subtract deposit gas from pool, so other txs do not use the same gas. And fail tx processing if deposits reach gas limit
- core/types: deposits do not tip, avoid basefee subtraction
- Unmeter the L1 Attributes Transaction
- miner: handle force tx errors as critical, clean up diff
- ci: Switch branch
- eth,miner: return STATUS_INVALID when failing to process forced transactions in request (ref-op-geth PR 40)
- verifier: forward tx to sequencer based on flag
- txpool: add flag to disable tx gossip (ref-op-geth PR 42)
- Add op-geth version in addition to geth version (ref-op-geth PR 43)
- ci: CircleCI improvements (ref-op-geth PR 44)
- Rename to op-geth
- Build latest tag on optimism branch

op-geth changes:
- Expose cache config in simulated backend (#2)
- Add EIP-1559 parameters
- eth/catalyst: update payload id computation (#1)
- make eip1559 configurable (#4)
- post-merge network should not log warnings about missing transition information (#5)
- Make the simulator more configurable (#6)
- fix OPB-6 - IsDepositTx check instead of artificial nonce value check (#7)
- Simulated backend - enable proof of stake consensus type and fix performance issue (#8)
- accounts: simulated backend consensus engine option and immediate tx indexing
- consensus/beacon: recognize all blocks as reached TTD with 0 TTD in chain config
- Add --rollup.historicalhttp CLI flag and fix backend iface
- Flags and interfaces for historical RPC requests (#12)
- Redirect historical RPC requests (ethereum#13)
- Use the pre-existing ethereum.NotFound error (ethereum#18)
- Add historical endpoint to TraceBlockByNumber and TraceBlockByHash (ethereum#19)
- Add historical endpoint to TraceTransaction (ethereum#20)
- Add historical endpoint to TraceCall (ethereum#21)
- optimism: fee params from info txi, update l1 cost func GPO params read (ethereum#15)
- add hardcoded addresses for fee payouts (ethereum#23)
- dynamic gas limit via engine API (ethereum#22)

Co-authored-by: Matthew Slipper <me@matthewslipper.com>
Co-authored-by: Joshua Gutow <jgutow@oplabs.co>
Co-authored-by: protolambda <proto@protolambda.com>
Co-authored-by: Mark Tyneway <mark.tyneway@gmail.com>
Co-authored-by: Maurelian <maurelian@protonmail.ch>
sduchesneau pushed a commit that referenced this issue Jun 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant