Node Public Interfaces Team Weekly Update - 2021-08-27

[frol]

The update from the last week is here: #4683

Let us know in the comments about the most exciting update these weeks.

Brief Summary

• nearcore 1.21.0-rc.1 has been released and deployed on testnet on 2021-08-18! If you use Indexer Framework on testnet, 2021-08-19 was the date when the new protocol version has become effective, and thus your non-upgraded Indexer would not be able to continue following the network
  [indexer] Get ready for nearcore 1.21.0 #4689
• nearcore 1.21.0 release on mainnet is scheduled for 2021-09-06. We will provide all the Indexer Framework upgrade instructions in Discussions
  [indexer] Get ready for nearcore 1.21.0 #4689
• Indexer Framework 0.10.0 has received some quality of life improvements (reduced RAM consumption and extended boot-up logging)
• Indexer-as-a-Service concept has been proposed by @khorolets and @frol
  https://gov.near.org/t/indexer-as-a-service-proof-of-concept/4825

Results

• explorer:

  • Merge Validators and Proposals and Pools into single tab (@shelegdmitriy)
    feat(frontend): Expose whole pools list near-explorer#661
  • Improved dashboard page loading time (@shelegdmitriy)
    refactor(frontend): Split fetching transaction stats into separate requests near-explorer#723
  • Use Rust implementation (from Indexer) of computing circulating supply (@telezhnaya)
    feat(backend): use circulating supply computation from indexer near-explorer#720
  • Update to Next.js 11 (@shelegdmitriy)
    refactor(frontend): Upgrade to Next.js 11 near-explorer#695
  • Changed FOSSA to run only on master to avoid blocking and external contributors failing PRs (@chefsale)
    feat(fossa): run fossa only on master to unbreak external contributors near-explorer#718
  • Fixed near balance on Stats page (@shelegdmitriy)
    Fix near ballance on stats page near-explorer#709

• docs

  • Documented an exhaustive list of potential errors for all the JSON RPC methods (@khorolets)
    Add possible errors info to JSON RPC methods docs#811
  • Proof of concept: Indexer as a service (@khorolets)
    https://gov.near.org/t/indexer-as-a-service-proof-of-concept/4825

• indexer-for-explorer:

  • Released indexer-for-explorer 0.9.0 and 0.9.1 compatible with new 1.21.0-rc.1 nearcore release (@khorolets)
    Upgrade indexer-for-explorer to be compatible with nearcore 1.21.0 near-indexer-for-explorer#150
    Bump version to 0.9.1 (nearcore 1.21.0.rc1 near-indexer-for-explorer#151
  • Use anyhow errors in circulating supply, prepare the project for implementing other aggregated computations (@telezhnaya)
    fix: cleanup in circulating supply near-indexer-for-explorer#156
    fix: use anyhow errors in circulating supply near-indexer-for-explorer#157
  • Changed FOSSA to run only on master to avoid blocking and external contributors failing PRs (@chefsale)
    fix(fossa): run fossa only on master to gather signal instead of bloc… near-indexer-for-explorer#152

• infrastructure

  • Split internal and public endpoints (@mhalambek)
  • Added loadtest monitoring config for prometheus (@mhalambek)
  • Documented how to update prometheus configuration (@mhalambek)
  • Cleaned up unused RPC instances, downsize archival nodes (@mhalambek)
  • Land the load testing changes for betanet (machine setup, reviews, mocknet discussions, monitoring) (@nikurt @chefsale)
  • Setup a separate project for Nayduck and figure out the GCP project splitting, now Nayduck is starting the migration to GCP (@mina86 @mhalambek @chefsale)
  • Shut down Mocknet to cut our infrastructure costs (@chefsale @nikurt)
  • Added ability to name RPC machines managed by terraform to your liking instead of the generated names (@chefsale https://github.com/near/near-ops/commit/66ce4de28a1f22f8e7c9f08250de618000610c88)
  • Updated the near-node package to be able to support custom binaries when creating new canaries through the automation (@chefsale https://github.com/near/near-ops/commit/85fd50c675121bd52a4613fa681adcc2c576aec1)
  • Setup of a mainnet-canary0 machine (@chefsale https://github.com/near/near-ops/commit/006c7f178d025b4ab9a8643ee5b7214b01eba894)
  • Fixed naming and updated documentation in for the canary, now it uses neard service unit instead of canary to be consistent across all machines (@chefsale https://github.com/near/near-ops/commit/6e1307552ff88af036d51917f754fd2a147b3509)
  • Added mainnet canary to existing automation, monitoring and alerting (@chefsale https://github.com/near/near-ops/commit/affc6b2aa60ea43352ce12ac3befd6bf252f961c, https://github.com/near/near-ops/commit/3ada49bb78e242b4e656645bbb652d8429a669cc)
  • Cleaned up some of the obsolete scripts (@chefsale https://github.com/near/near-ops/commit/4401b03573e1757167ff69f2e31c36fbd8c62b84)

• nearcore

  • After reviews, iterative revisions and a feature dropped in near-network-primitives, we've further reduced the dependency tree of near-jsonrpc-primitives by another ~10% (@miraclx)
    refactor(chain): expunge rocksdb from near-jsonrpc-primitives dependency tree #4651
  • Fixed an issue with subaccount checks. AccountId::is_sub_account_of() should only mark subaccounts that are a direct subaccount of the account in question. E.g alice can't directly create an account with x.y.alice but can create one with x.alice. (@miraclx)
    fix(account-id): subaccounts are valid one level deep only #4694
  • Fixed the broken tests after deprecation of the near binary build in nearcore (@chefsale)
    fix(bug): fix the upgradable.py and backwards_compatible.py tests aft… #4693

• near-api-rs

  • Initialized a repository for our near-api-rs efforts and included a first pass with straightforward documentation. It's currently unfit for non-trivial applications, so do not use! Heavy revisions would ensue https://github.com/near/near-api-rs (@miraclx)
  • Implemented exhaustive error handling with strict typing (@miraclx)
    https://github.com/near/near-api-rs/blob/a29105289add9f8a67225c1cb651d53d66bf0b42/src/lib.rs#L18-L51
  • Implemented helpers for the rest of the RPC methods, including sandbox methods and adversarial controls (@miraclx)

Challenges & Blockers

• We need more feedback on Indexer-as-a-Service concept: https://gov.near.org/t/indexer-as-a-service-proof-of-concept/4825
• In order to include the most relevant highlights on Explorer Dashboard, we want to hear ideas as to what gives or would give you an idea that NEAR network is legit and alive. (Leave the comments under this post)

Action Items

• indexer-for-explorer:

  • Work on better FT & NFT support in Indexer for Explorer (@telezhnaya)
    feat: add ft, nft aggregated tables near-indexer-for-explorer#153

• explorer

  • Resolve the issue with non-existent accounts being inaccurately represented on Account Details page (@shelegdmitriy)
    Non-existent accounts inaccurately represented on Account Details page near-explorer#715

• infrastructure

  • Fix public backups (@mhalambek)
  • Migrate buildkite automation to provide better naming of the pipelines (@chefsale)
  • Figure out a cost-effective way forward for mocknet (@chefsale)

• nearcore

  • Ensure to implement Deserialize for all Rpc*Response variants to allow individual method interfaces in near-api-rs be more specialized and better defined. (@miraclx)
  • Revise the struct naming on EXPERIMENTAL_changes and EXPERIMENTAL_changes_in_block in near-jsonrpc to clear up confusion (@miraclx)
  • Look into Query ViewState with with non-existing account id and invalid prefix does not return error as expected #2793, since strict AccountId's introduction addresses this, cleanup the cases and close linked issues (@miraclx)

• near-api-rs

  • Utilize a trait, implemented on all Rpc*Request variants from near-jsonrpc to alleviate the use of enums. (@miraclx)