CommP CID generation (official & correct!)

[rvagg]

/cc @hannahhoward @pfac

multicodec table now has new entries for Filecoin, wrapup @ multiformats/multicodec#161 (comment) so the older CommP->CID generation methods should probably be removed (or at least updated) as they are not generating using official codes.

multiformats/go-multihash now has SHA2_256_TRUNC254_PADDED @ multiformats/go-multihash#128, although this is not yet released, so you need to grab the HEAD.

ipfs/go-cid has a pending PR for the two codecs, including FilecoinCommitmentUnsealed @ ipfs/go-cid#104, so a fork and redirect is needed for now.

In practice, given a commp faeba569cafd5c144328f8dbb1873c6c0da4ce3a2136a8e2e9b1966bcaa68f0e:

$ ./commp-cid faeba569cafd5c144328f8dbb1873c6c0da4ce3a2136a8e2e9b1966bcaa68f0e
baga6ea4seaqpv25fnhfp2xauimuprw5rq46gydnezy5ccnvi4lu3dftlzkti6dq

Then verify:

$ node
> mf = require('@ipld/multiformats')
> c = new mf.CID('baga6ea4seaqpv25fnhfp2xauimuprw5rq46gydnezy5ccnvi4lu3dftlzkti6dq')
CID(baga6ea4seaqpv25fnhfp2xauimuprw5rq46gydnezy5ccnvi4lu3dftlzkti6dq)
> Buffer.from(mf.multihash.decode(c.multihash).digest).toString('hex')
'faeba569cafd5c144328f8dbb1873c6c0da4ce3a2136a8e2e9b1966bcaa68f0e'
> c.code.toString(16)
'f101'
> mf.multihash.decode(c.multihash).code.toString(16)
'1012'

So we can extract the original commp, the codec for fil-commitment-unsealed: 0xf101 and the multihash for sha2-256-trunc254-padded: 0x1012.

[ribasushi]

@rvagg this can now be replaced entirely with (untested)

Suggested change

	mh, err := multihash.Encode(commp, multihash.SHA2_256_TRUNC254_PADDED)
	if err != nil {
	panic(err)
	}
	ccid := cid.NewCidV1(cid.FilCommitmentUnsealed, mh)
	println(ccid.String())
	ccid, err := commcid.DataCommitmentV1ToCID(commp)
	if err != nil {
	panic(err)
	}
	println(ccid.String())

[rvagg]

I think I'll leave this to sit and just be for the record as it's not got a practical use at the moment. https://github.com/rvagg/rust-fil-commp-generate is the one that we ended up using (for Lambda and commandline) and it's getting cleanup, tests and docs. So that's probably where people need to go for an up to date version. This library and PR may be helpful to someone trying to grok it from Go.

[ribasushi]

I think I'll leave this to sit

No worries at all: it simply came up during the reverse dependency walk when I was fixing up fil-commcid

[ribasushi]

( dropping a link for cross-reference )
There is now also a rust/ffi-free version in go: https://pkg.go.dev/github.com/filecoin-project/go-fil-commp-hashhash