Cluster with zero addresses and txes

[ssiang1627]

Please provide a clear and concise description of what the problem is.

Expected Result

After clustering, I found there are many cluster with no addresses and txes and have zero size, but the type_equiv_size is not zero. In addition, I clustering with height 563981 and compare it to the default cluster in AMI(/home/ubuntu/bitcoin/clusters) .

The Result:

// cluster with chain height 563981
cm = blocksci.cluster.ClusterManager.create_clustering("/home/ubuntu/bitcoin/clusters_563981", chain)

clusters = cm.clusters()
clusters[1000].size() // return 0
clusters[1000].addresses.all // return []
clusters[1000].txes() // return []
clusters[1000].type_equiv_size // return 1

clusters[204830300].size() // return 0
clusters[204830300].addresses.all // return []
clusters[204830300].txes() // return []
clusters[204830300].type_equiv_size // return 1

//original one in AMI
cm = blocksci.cluster.ClusterManager("/home/ubuntu/bitcoin/clusters", chain)
clusters = cm.clusters()
clusters[1000].size() // return 0
clusters[1000].addresses.all // return []
clusters[1000].txes() // return []
clusters[1000].type_equiv_size // return 1

clusters[204830300].size() // return 1
clusters[204830300].addresses.all // return [NulldataAddressData()]
clusters[204830300].txes()
 // return [Tx(len(txins)=1, len(txouts)=3, size_bytes=257, block_height=521639, tx_index=314888215)]
clusters[204830300].type_equiv_size // return 1

So I wonder why there are clusters with no addresses? and why the results are different above? Moreover I also check the cluster with address, the result is also weird.

//with height 563901
address = chain.address_from_string("19Hu8jch1pz3N62npSxUkeS5U2M7t26ftu")
cluster = cm.cluster_with_address(address)
for c in cluster.addresses.all:
    print(c.address_string) // return 3JP5be5CoAaHZjCyRbue88Ae8K9XdvN1Eu

address.address_string //return 19Hu8jch1pz3N62npSxUkeS5U2M7t26ftu

Thanks for your help!!

System Information

BlockSci version: 0.5
Using AMI: yes
Total memory:60

[geofurb]

Should reference #219 on this.

[maltemoeser]

Clusters with cluster.size() == 0 and cluster.type_equiv_size > 0 are an artifact of BlockSci's deduplication. For example, you could have a multisig address with three pubkeys. BlockSci keeps track of the three pubkeys independently of their combined use as a multisig address.

During clustering, each of these four addresses (1 multisig, three pubkey) starts in their own cluster. If the individual pubkeys are never used on their own, they'll remain in their single-address cluster (type_equiv_size == 1). When calling .size(), .transactions() etc., BlockSci will then check whether the addresses in the cluster have actually been used. If one of the individual pubkeys that was part of a multisig has never been used individually you'll get .size() == 0, .txes() == [].

Cluster indexes are not kept consistent across clusterings that are done with different states of the blockchain. If you want to track how clusters change you'll need to identify them based on the addresses within them.

[jiagengliu]

Additional reference at #408: duplicated addresses in .addresses.