Skip to content
This repository has been archived by the owner on Sep 26, 2024. It is now read-only.

Refactor: validators #955

Merged
merged 9 commits into from
Apr 28, 2022
Merged

Refactor: validators #955

merged 9 commits into from
Apr 28, 2022

Conversation

luixo
Copy link
Contributor

@luixo luixo commented Apr 22, 2022

This is a proper refactoring of the most entangled solution in the project - the validators.
Now, we have only two validators types - the one with the data from current epoch (ValidatorEpochData) and another one extending it with external data - contact info, fee, telemetry etc (ValidatorFullData).

Glossary:

  • current epoch stake (CES) - stake of the validator in current epoch (epoch X);
  • next epoch stake (NES) - stake of the validator in the upcoming epoch (epoch X+1);
  • after next epoch stake (ANES) - stake of the validator that it proposed to the network, to be picked (or not picked) in the epoch after the next one (epoch X+2);
  • contract stake (CS) - stake of the validator that is fetched from contract pool function get_total_staked_balance, used to fetch balance for potentially-validating-nodes not currently participating in validating process.

I'll describe current scheme briefly for historical purposes:

  • We get data about current epoch validators, next epoch validators and proposal (after next epoch) validators from RPC call validators and remap it to the list of validators with corresponding props.
  • Additionally, we add all the accounts that end up with poolv1.near to the list for a user to pick a newcoming pool with not enough tokens to take a validator seat. For these we get CS.

The order of validators rendering is:

  • first by amount of CES tokens (if any)
  • second by amount of NES tokens (if any)
  • third by amount of ANES tokens (if any)
  • last by amount of CS tokens (if any)

We render those validators with amount of tokens:

  • on the main line we render one of the stakes, the priority is: CES, NES, ANES, CS.
  • on the secondary line we render stake diff ONLY if we have CES and one of the following: NES or ANES, the diff is either NES-CES or ANES-CES.

We render validators token status based on following rules:

  • if contract has CES and NES - we render it as active
  • if contract has CES but no NES - we render it as leaving
  • if contract has no CES but has NES - we render it as joining
  • if contract has no CES and no NES but has ANES - we render it as proposal
  • else we decide by validator seat price:
    • if CS has more tokens than the seat price - we render it as onHold
    • if CS has more tokens than 20% of the seat price - we render it as newcomer
    • if CS has less tokens than 20% of the seat price - we render it as idle

@luixo luixo requested a review from shelegdmitriy as a code owner April 22, 2022 13:28
Base automatically changed from refactor/validator-metadata-row to master April 22, 2022 14:15
@luixo luixo force-pushed the refactor/validators branch from f590baf to 3c30985 Compare April 22, 2022 14:15
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 22, 2022 14:17 — with Render Active
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 22, 2022 14:17 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 22, 2022 14:17 — with Render Active
@luixo luixo force-pushed the refactor/validators branch from 3c30985 to 56d5a5a Compare April 22, 2022 14:34
@luixo luixo temporarily deployed to refactor/validators - frontend PR #955 April 22, 2022 14:35 — with Render Destroyed
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 22, 2022 14:35 — with Render Active
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 22, 2022 14:35 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 22, 2022 14:35 — with Render Active
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 22, 2022 14:44 — with Render Active
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 22, 2022 14:44 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 22, 2022 14:44 — with Render Active
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 22, 2022 16:30 — with Render Active
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 22, 2022 16:30 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 22, 2022 16:30 — with Render Active
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 22, 2022 18:53 — with Render Active
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 22, 2022 18:53 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 22, 2022 18:53 — with Render Active
@luixo luixo force-pushed the refactor/validators branch from 39f1099 to 56e8467 Compare April 25, 2022 11:35
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 25, 2022 11:35 — with Render Active
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 25, 2022 11:35 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 25, 2022 11:35 — with Render Active
Copy link
Collaborator

@frol frol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are some initial comments. I am still working on finishing the review of the front-end side of the PR

[
`SELECT
account_id
const queryNodeValidators = async (): Promise<string[]> => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we rename this function to queryStakingPoolAccountIds since the current name is misleading

valueMap: new Map(),
promisesMap: new Map(),
};
const poolInfos: CachedTimestampMap<ValidatorPoolInfo> = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use "staking pool" terminology since "pool" might be ambiguous (e.g. database connections pool)

let stakingPoolsInfo = new Map<string, StakingPoolInfo>();
let currentStakeInfo = new Map<string, string | undefined>();
let stakingPoolsMetadataInfo = new Map<string, PoolMetadataAccountInfo>();
const contractBalances: CachedTimestampMap<string> = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about "staking pool stake proposals"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we emphasize that these proposals are somehow different from the proposals we get from RPC (in this case - different by the source)?
Or stakingPoolStakeProposalsFromContract is too much? :)

Comment on lines +173 to +177
for (
let currentIndex = 0;
true;
currentIndex += VALIDATOR_DESCRIPTION_QUERY_AMOUNT
) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: while loop would be more readable, I feel

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually changed it to regular loop as anything with init and iterating by incrementation is automatically "for loop" for me :)

}
};

const getValidatorContractBalance = async (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use "staking pool" instead of "validator contract" since you can deploy any contract to a validator account, but we actually want to communicate with staking pool contract.

Suggested change
const getValidatorContractBalance = async (
const getStakingPoolStakedBalance = async (

Comment on lines 279 to 305
const updateContractStakeMap = async (
validators: ValidatorEpochData[],
cachedTimestampMap: CachedTimestampMap<string>
): Promise<void> => {
return updateRegularlyFetchedMap(
validators
.filter((validator) => !validator.currentEpoch)
.map((validator) => validator.accountId),
cachedTimestampMap,
getValidatorContractBalance,
regularFetchStakingPoolsInfoInterval,
fetchStakingPoolsInfoThrowawayTimeout
);
};

const updatePoolInfoMap = async (
validators: ValidatorEpochData[],
cachedTimestampMap: CachedTimestampMap<ValidatorPoolInfo>
): Promise<void> => {
return updateRegularlyFetchedMap(
validators.map((validator) => validator.accountId),
cachedTimestampMap,
getPoolInfo,
regularFetchStakingPoolsInfoInterval,
fetchStakingPoolsInfoThrowawayTimeout
);
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, it would be great to align the variable names. Currently, they are hard to follow.

cachedTimestampMap name is hard to reason about

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did my best though some variable names became huge

Comment on lines +85 to +88
const mapValidators = (
epochStatus: EpochValidatorInfo,
poolIds: string[]
): ValidatorEpochData[] => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The naming is hard to reason about the actual intent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which naming do you mean here? mapValidators? Should it be getValidatorsFromEpochStatus? Or just getValidators?

});
});
}
await updateValidatorDescriptions(stakingPoolsMetadataInfo);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the naming of the called function is not aligned with the naming of the outer functions

Comment on lines 27 to 30
<link
href="https://fonts.googleapis.com/css2?family=Roboto+Mono:wght@500&display=swap"
rel="stylesheet"
/>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we only need it here? Is it some sort of an artifact?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we need it one level deeper - in ValidatorTelemetryElements.tsx, as it's the only place that uses that font.

Copy link
Collaborator

@frol frol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the last piece of comments so far

Comment on lines +86 to +87
if (contractStake.gte(seatPriceBN)) {
return "onHold";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, ideally, on-hold should be only when it is explicitly set on pause in the contract:

https://github.com/near/core-contracts/blob/793e0594bab4e9cf730d37cd2a493a173b0e2ed3/staking-pool/src/lib.rs#L364-L366

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How should we name the one that is not in neither current epoch, nor next epoch or proposal?

): StakingStatus | null => {
if (validator.currentEpoch) {
if (validator.nextEpoch) {
return "active";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you feel about using SCREAMING_SNAKE_CASE for such const markers? (ACTIVE, ON_HOLD, ...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a few of types based on string discrimination, some of them are based on PascalCase we get from RPC, the rest is arbitrary.

The only thing I feel uncomfortable is to have a mapping with CAPS keys.

Why did you suggest that? Because of on-hold change to onHold, I presume?

Comment on lines +120 to +121
const nextVisibleStake =
validator.nextEpoch?.stake ?? validator.afterNextEpoch?.stake;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per our prior discussion we wanted to display the diff to the most recent change of the stake, so when validator submits an intent to unstake, we can clearly see that the stake will drop to 0 soon, so the order should be flipped:

Suggested change
const nextVisibleStake =
validator.nextEpoch?.stake ?? validator.afterNextEpoch?.stake;
const nextVisibleStake =
validator.afterNextEpoch?.stake ?? validator.nextEpoch?.stake;

P.S. It deserves a comment in the code with a link to this reply

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we have a few cases in here, with three stakes that may exist or not: currentEpoch.stake, nextEpoch.stake, afterNextEpoch.stake. We can make a table of what kind of deltas we want to show.

In case proposal exists:

current !current
next current -> next next -> proposal
!next current -> proposal 0 -> proposal

In case proposal does not exist:

current !current
next current -> next next -> proposal
!next current -> 0 -

Is that what you expect or not?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tables look too complex to me, and it seems they are not what I wanted to say.

  1. Well, my fix was incorrect since I did not rename nextVisibleStake variable name. Your logic was correct regarding this naming, but I actually wanted us to display not the nextVisibleStake diff, but rather mostRecentStake diff
  2. mostRecentStake is validator.afterNextEpoch?.stake ?? validator.nextEpoch?.stake

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostRecentStake is validator.afterNextEpoch?.stake ?? validator.nextEpoch?.stake

But in case of an active validator it will be the delta between the current epoch and the proposal, is that actually what we want to do?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luixo yes, this is what people expect to see, I believe. In the future we might want to educate people better about the fact that the next epoch validators list is already frozen, and the proposals are landing to the epoch that is the one that is next after the next one, but for now I feel it is fine to just hide that unnecessary technical detail from users.

// we take "active", "joining", "leaving" validators and sort them firstly
// after then we sort the rest
const validatingGroup = ["active", "joining", "leaving"];
const sortByBNComparison = (aValue?: string, bValue?: string) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that new BN(undefined) is BN(0), so it seems we can drop this function altogether and just write:

(a, b) => -new BN(a.currentEpoch?.stake).cmp(new BN(b.currentEpoch?.stake)),

@frol
Copy link
Collaborator

frol commented Apr 26, 2022

image

https://frontend-pr-955.onrender.com/nodes/validators

I am thinking about mixing those "joining" and "proposals" into the list of active nodes but without giving them an index number, and showing their stake in grey font color like it is commonly used to indicate "disabled" state. @shelegdmitriy I feel this used to be the case before, wasn't it? @reefoh Any objections? I don't want us to spend too much time on it, but since we are touching it anyway, we may as well do slight tuning.

@frol
Copy link
Collaborator

frol commented Apr 26, 2022

On the second thought, I feel we can merge this PR as is, and then consider to have a separate PR addressing the topic I raised in the previous comment.

@shelegdmitriy
Copy link
Contributor

On the second thought, I feel we can merge this PR as is, and then consider to have a separate PR addressing the topic I raised in the previous comment.

I agree with it. From my point of view It's better to focus on such things with new design (for example). Maybe it's better to create an issue and continue this discussion there

@luixo luixo self-assigned this Apr 27, 2022
@luixo luixo deployed to refactor/validators - backend/mainnet PR #955 April 28, 2022 17:23 — with Render Active
@luixo luixo deployed to refactor/validators - backend/guildnet PR #955 April 28, 2022 17:23 — with Render Active
@luixo luixo deployed to refactor/validators - backend/testnet PR #955 April 28, 2022 17:23 — with Render Active
@luixo luixo force-pushed the refactor/validators branch from adefa47 to 509998e Compare April 28, 2022 18:33
@luixo luixo temporarily deployed to refactor/validators - wamp PR #955 April 28, 2022 18:33 — with Render Destroyed
@luixo luixo temporarily deployed to refactor/validators - backend/mainnet PR #955 April 28, 2022 18:33 — with Render Destroyed
@luixo luixo temporarily deployed to refactor/validators - backend/guildnet PR #955 April 28, 2022 18:33 — with Render Destroyed
@luixo luixo temporarily deployed to refactor/validators - backend/testnet PR #955 April 28, 2022 18:33 — with Render Destroyed
@luixo luixo temporarily deployed to refactor/validators - frontend PR #955 April 28, 2022 18:33 — with Render Destroyed
@luixo luixo merged commit 04b20e9 into master Apr 28, 2022
@luixo luixo deleted the refactor/validators branch April 28, 2022 18:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants