[stateless_validation] Top contracts storage

[Longarithm]

Consider putting contracts code into separate storage to avoid including them into state witness.

Discussion: https://near.zulipchat.com/#narrow/stream/407237-pagoda.2Fcore.2Fstateless-validation/topic/witness.20size/near/417345742

Proposal

• Introduce separate rocksdb column for top-N contract codes indexed by (epoch_id|epoch_height, account_id).
• Add a new step to StateSync: together with state for tracked shards, if node is a chunk validator for the next epoch, it must download top-N contracts from the whole state, alongside with state proofs for the contracts. Chunk producers are responsible for providing these.
  • The supported request could be ContractCodeRequest(sender_id, contract_id) and response ContractCodeResponse(contract_id, proof).
• When chunk producer makes state witness, it is allowed to leave only contract code hash in the state transition proof, if the code is in top-N and wasn't updated.
• When chunk validator receives state witness and only the contract code hash is given, it must verify that contract is in top-N, and if this is the case, take code from its storage.

If N=100, given contract code length limit of 4MB, it will be only 400MB to download.

Alternatives

• We can put these contract codes on GCS if this is simpler.
• We can make the storage static if we believe that top-N contracts won't be updated in a quarter and work on proper solution later.
• Chunk validators can track contracts on their own. To do so, it's probably enough to get only DeployContract receipts from chain which existence can be proven alongside with outcomes and update contracts accordingly.

Open questions

• If contract is updated via StateSync, the optimisation won't be applied during next 2 epochs - how much of an impact will it be?

Given that multiple code changes are needed, including protocol changes, I estimate LoE as 8 weeks.

[Longarithm]

Discussed on weekly - Robin’s investigation show that mainnet state witnesses are small anyway and bounding by compute costs is enough for now.

[walnut-the-cat]

Will be tracked as a part of #10780

[walnut-the-cat]

Duplicate of #10780