Node Public Interfaces Team Weekly Update - 2021-09-24

[frol]

The previous update is #4846. Also, I want to bring it to attention that there are global engineering updates posted on gov.near.org, and here is the most recent one: https://gov.near.org/t/engineering-weekly-update-2021-09-20/6078

Brief Summary

• Indexer for Explorer infrastructure has been improved to reduce incidents reaction time and recovery time (Kudos to @khorolets @chefsale and @mhalambek collaborating on implementation, documentation, and alerting configuration)
• New Explorer Stats backend solved the problem of empty charts on Explorer (it is yet to be finalized, but it is already partially live! Kudos to @telezhnaya for leading it!)

Blockers

• Lack of standardization of events for FT and NFT standards still block Explorer and Wallet on displaying the data to the end-users
  NFT NEP-171 require having minting/burning interface NEPs#254

Challenges

• Indexer for Explorer is considering various ideas on storage improvements since we won't be able to scale with PostgreSQL in the future

Result

• Explorer received some stability improvements:

  • Migrated to Prometheus format for exporting metrics, which we integrated into our alerting system 👀
    feat(prometheus): implement metrics prometheus endpoint instead of th… near-explorer#739
  • Removed all legacy code
    refactor(backend): Remove legacy code near-explorer#736

• On the path to release near-jsonrpc-client-rs a number of tweaks were landed:

  • Made EpochReference flatten-able when embedded in other structures that require flattening
    fix(near-primitives): EpochReference enum can be flattened into a struct #4848
  • Fixed backward compatibility on ExecutionOutcomeView after the introduction of the metadata field
    fix(primitives): backwards compatible ExecutionOutcomeView #4851
  • Added generic documentation on each RPC method to ease approachability, usability, and clarity (see an example here)
    doc(methods): add generic documentation on all rpc methods near-jsonrpc-client-rs#8
  • Fixed deserialization of InternalError variants from server response
    near/near-jsonrpc-client-rs@86b9248
  • Ensured JsonRpcClient instance is reusable, yet completely consumable
    near/near-jsonrpc-client-rs@edf1fcf
  • Added default connector, and a default JsonRpcClient::connect() method but also retain the ability to instantiate a custom connector
    near/near-jsonrpc-client-rs@6ecaa19
  • Fixed validators serialization logic
    near/near-jsonrpc-client-rs@693a639
  • Reintroduced EXPERIMENTAL_tx_status method
    reintroduce EXPERIMENTAL_tx_status near-jsonrpc-client-rs#10

• meta

  • @miraclx gave a speech at the NEAR Hacknode in Kano: https://gov.near.org/t/details-of-kano-hacknode-to-be-done-by-nearprotocol-nigeria/5714/4 💯

• operations

  • Resized archival nodes storage on testnet to 2TB

Action Items

• explorer

  • Support contributor with translating Explorer UI to Russian
    NEAR Explorer Russian translation near-explorer#738 (@shelegdmitriy @frol @telezhnaya)
  • Display Circulating Supply chart on Stats
    Display Circulating Supply chart on Stats near-explorer#740 (@shelegdmitriy)
  • Revamp UX and UI of dashboard and account details page (@reefoh)

• nearcore

  • Replace client re-export from near-jsonrpc with the new near-jsonrpc-client-rs and drop the old near-jsonrpc-client inside nearcore (@miraclx)
    • While doing this, a bug with error handling in near-jsonrpc-client-rs was uncovered
  • Look into Query ViewState with with non-existing account id and invalid prefix does not return error as expected #2793, since strict AccountId's introduction addresses this, cleanup the cases and close linked issues (@miraclx)
    • Blocked on replacing client with the new near-jsonrpc-client

• near-jsonrpc-client-rs

  • Fix edge cases where the handler result error doesn't cleanly deserialize to the respective types. Introduce a middleware for properly selecting the correct error (@miraclx)
    • Context: spotted an error while replacing client in nearcore

• operations

  • Escalate crashes as low priority alerts (@mhalambek)
  • Implement hybrid SSD/HDD storage to save on SSD costs (@mhalambek)

• indexer-for-explorer

  • Optimize stats collection for Explorer (@telezhnaya)
    https://github.com/telezhnaya/near-analytics
  • Consider alternatives to PostgreSQL storage (@frol @telezhnaya @khorolets)
  • Investigate and fix start-from-interruption on indexer-for-explorer as it is too slow for no reason (@khorolets)
  • Making indexer for explorer retries logic to panic after N attempts, clearing the code (@khorolets)
    refactor: Retry or panic macros to replace code duplication and add panic after N attempts near-indexer-for-explorer#168