feat: abstract over consumption of different RPC flavors

[harrysolovay]

Heavily WIP

Introduces Consumers, which provide a standard layer on top of different RPC APIs (the current and the experimental). This will allow us to decouple the fluent API from RPC specifics (and therefore ease the transition to Smoldot). The end DX may look as follows.

import { ParachainRune } from "@capi/parachain"
import { ExperimentalRpc, SmoldotConnection } from "capi"
import { parachainSpec, relayChainSpec } from "./chain_specs.ts"

const api = ParachainRune.from(
  new ExperimentalRpc(
    SmoldotConnection.bind({ parachainSpec, relayChainSpec }),
    signal,
  ),
)

The ExperimentalRpc (a Consumer) wraps the Connect itself. The consumer is then passed into the chain rune factory, so that the chain rune can operate with an RPC-unspecific API.

---

For now, the ExperimentalRpc API accepts an additional argument: a non-smoldot Connect. This is necessary for retrieving historical block info. It seems the archive-prefixed methods of the new JSON RPC spec are yet to be implemented for any chains (?). Seems a bit far-fetched... but a GitHub-wide search revealed no such clues.

[harrysolovay]

TODO: get rid of this spec before merging

[kratico]

This is great!!!

When I try smoldot, I noticed that

• the first chainHead_unstable_storage calls yielded a few inaccessible before giving a successful value.
• chainHead_unstable_follow stop could happen at any time, and it will require to get a new followId before attempting a new chainHead_unstable_storage

[kratico]

I think we will need to get this using chainHead_storage and query for system.account(AccountId32)

[kratico]

There a note in the docs for chainHead_unstable_call

Note: This can be used as a replacement for the legacy state_getMetadata, system_accountNextIndex, and payment_queryInfo functions.

https://paritytech.github.io/json-rpc-interface-spec/api/chainHead_unstable_call.html

[kratico]

I'm not sure if the docs are outdated or if the polkadot/substrate API implementation is not complete.

Passing a single hex encoded key works.

Something like

      this.subscription<{ event: "items"; items: { key: string; value: string }[] }>(
        "chainHead_unstable_storage",
        "chainHead_unstable_stopStorage",
        [followId, blockHash, items[0]?.key[0], null],
        (message) => {

And the response is something like

<<< {
  message: {
    jsonrpc: "2.0",
    method: "chainHead_unstable_storage",
    params: {
      subscription: "x1sMLqhwCWeg7QBJ",
      result: {
        event: "done",
        result: "0x000000000000000001000000000000004035323d3400000000000000000000000000000000000000000000000000000000"... 62 more characters
      }
    }
  }
}

It seems that the chainHead_unstable_storage is working for single key reads

[kratico]

Investigated this API spec updates and the multiple item support was documented 3 weeks ago
See paritytech/json-rpc-interface-spec@bea1a60

Smoldot hasn't been updated as it supports a single item request
See https://github.com/paritytech/smoldot/blob/938055a638ec201c022f680c8e8cbd0349e70ed1/bin/light-base/src/json_rpc_service/chain_head.rs#L874

And I guess substrate is in the same state

[kratico]

there are other events that we need to think how to handle

• waiting-for-continue, tricky because .valuesPending can be resolved only once
• inaccessible, retry a few times and reject?
• error, reject
• disjoint, reject