The performance will be very poor when using apiAt heavily.

[ntduan]

This is the test code：

const { WsProvider, ApiPromise } = require("@polkadot/api");

(async () => {
  const api = new ApiPromise({
    provider: new WsProvider("wss://rpc.polkadot.io/"),
  });

  await api.isReady;

  const blocks = 70;

  const blockHashes = await Promise.all(
    [...new Array(blocks)].map((_, index) =>
      api.rpc.chain.getBlockHash(index + 1)
    )
  );

  console.time("apiAt:");
  await Promise.all(blockHashes.map((blockHash) => api.at(blockHash)));
  console.timeEnd("apiAt:");

  process.exit(0);
})();

For 70 consecutive blocks, each block seems to fetch the metadata and create a new api object. Here's the result I got：

2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s
2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s
2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s
2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s
2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s
2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s
2022-03-31 16:00:52        RPC-CORE: getMetadata(at?: BlockHash): Metadata:: No response received from RPC endpoint in 30s

Even if I reduce the number of concurrent requests, it will take several seconds.

Although I know that getting the metadata and decoding the metadata can be very time taking. But the fact is that the metadata for these blocks should be the same.

Wouldn't it be better to cache the api according to runtimeversion?

[jacogr]

It should check the runtime version first and find a match of exiting metadata. If not there is an issue.

api/packages/api/src/base/Init.ts

Line 156 in a98e954

// get the runtime version, either on-chain or via an known upgrade history

retrieval happens there. It could very well be related to the fact that it goes in parallel, ie. It doesn’t have the cache as of yet.

[jacogr]

Looking at the above, while JS is single-threaded, it is indeed due to the parallel nature. One after the other and the above would indeed kick-in. There is actually 2 useful enhancements here to cater for this specific use:

1. Allow other chains to provide an upgradeHistory, e.g. for Kusama/Polkadot there is actually a shortcut for historic blocks that is maintained, e.g.

   api/packages/types-known/src/upgrades/kusama.ts

   Line 6 in a98e954

   const upgrades: ChainUpgradesRaw = [

   (kicks in on the line just above the one linked above)

2. The second part is more tricky - basically track open requests for a specific version and turn it into a promise. Basically if you know there is an open request for metadata for a specific version, then point to the same. It basically mirrors what happens here for the same blockHash into the version as well -

   api/packages/api/src/base/Init.ts

   Line 207 in a98e954

   if (isUndefined(waiting)) {

Overall though, the logic above works well to not duplicate when used in a serial fashion, but all at once would need a bit of love since requests don't cross-track (and quite a bit of testing to get right)

[ntduan]

thanks

[polkadot-js-bot]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.