remember recent transactions we've received and don't ask for them again

[arvidn]

Purpose:

Avoid requesting the same mempool transactions multiple time, especially ones that have failed validation.

Current Behavior:

New Behavior:

Testing Notes:

[emlowe]

can you explain the 2 minutes aspect of this here

[arvidn]

this was an arbitrary timeout. it might make sense to make it longer. It seemed unnecessary to keep old transactions around indefinitely, that's all.

[arvidn]

I bumped it to 10 minutes

[arvidn]

I can't reproduce the symptom that this is supposed to address. I'm having second thoughts about including this PR. I'm tempted to rebase it on top of main and possibly explore it further there.
This patch also appear to be somewhat overlapping with the existing seen_bundle_hashes.

[OverActiveBladderSystem]

I can't reproduce the symptom that this is supposed to address. I'm having second thoughts about including this PR. I'm tempted to rebase it on top of main and possibly explore it further there. This patch also appear to be somewhat overlapping with the existing seen_bundle_hashes.

I looked at a 30 second log in debug mode from yesterday and pulled out the lines which had double spend transactions, then I looked for duplicates among those transactions and can show a brief sample of numerous duplicates of the same transaction.

While I'm not certain this is the sole cause of my heavy CPU utilization, it shows reprocessing of data sometimes 15 seconds apart from the first time it was done, the double spends seemed to be present most during my high CPU utilization periods.

Note, I set MEMPOOL_BLOCK_BUFFER: 1 in the config yaml during this particular log period in case that has any impact.

Windows 10
Only Full_Node process running in CLI mode (farmer/harvester on a different machine)

processing-duplicate-transactions-results.txt

adding the unedited 30 second log below just in case you want to see other things within it for those timestamps
time-204640-to-204710.txt

[arvidn]

@OverActiveBladderSystem there's a built-in CPU profiler that can be enabled in config.yaml, under full_node: there's enable_profiler: False, you you set that to true and restart the node, it will dump a CPU profile once per second (into the ~/.chia/mainnet/profile-node.

To look at the CPU usage over time, assuming you have the chia-blockchain source, python -m chia.util.profiler ~/.chia/mainnet/profile-node | less -r. It will print a graph of CPU usage over time along with timestamps (seconds).

To focus on a specific spike, run:

python -m chia.util.profiler ~/.chia/mainnet/profile-node <timestamp>

or

python -m chia.util.profiler ~/.chia/mainnet/profile-node <first-timestamp> <last-timestamp>

This will generate a gprof2dot graph and save it in the current directory. It requires pprof2dot and graphwiz (dot) be installed.

[arvidn]

My understanding of the cost of having these invalid transactions in your peers' mempools is that it's proportional to your peer churn. Only when you connect to a peer do we request mempool items, we send our BIP158 filter of the transactions we have in our mempool (it's like a bloom filter) and the peer responds with its 100 "best" transactions that we don't have. Best meaning high fee-per-cost, so in this case it will be a random 100 invalid transactions.

According to your log, we validate each transaction in ~3 milliseconds (for 100 of them, that's 0.3 seconds). But then we'll never get any more bad transactions from that peer again, unless we disconnect and re-connect. During steady state, transactions are only propagated as we learn about new ones and we validate them and add them to our own mempool.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.