Skip to content
This repository was archived by the owner on Aug 30, 2022. It is now read-only.

BUG: EOS locked by system contract - many voters' staked property <voters.staked> is less than total staked CPU/NET of the voter account, and leads to "stake for voting cannot be negative" assert when the voter account trying to unstake EOS. #34

Open
shawnbow opened this issue Jan 14, 2020 · 19 comments
Assignees

Comments

@shawnbow
Copy link

Take an EOS account huangpanlong (who has 18.3154 EOS sadly locked) as an example:

  • cleos -u https://mainnet.eos.dfuse.io get table eosio eosio voters -L huangpanlong -l 1, will see in the voter table, huangpanlong has 40032.8596 EOS staked.
{
  "rows": [{
      "owner": "huangpanlong",
      "proxy": "aboringgame1",
      "producers": [],
      "staked": 400328596,
      "last_vote_weight": "442764385327593.43750000000000000",
      "proxied_vote_weight": "0.00000000000000000",
      "is_proxy": 0,
      "flags1": 0,
      "reserved2": 0,
      "reserved3": "0.0000 EOS"
    }
  ],
  "more": true
}
  • cleos -u https://mainnet.eos.dfuse.io get table eosio huangpanlong delband, will see in the deband table, huangpanlong actually staked 2.5000+ 40048.6750 = 40048.675 EOS:
{
  "rows": [{
      "from": "huangpanlong",
      "to": "huangpanlong",
      "net_weight": "2.5000 EOS",
      "cpu_weight": "40048.6750 EOS"
    }
  ],
  "more": false
}
  • cleos -u https://mainnet.eos.dfuse.io get table eosio eosio rexbal -L huangpanlong -l 1, will see in the rexbal table, huangpanlong has no rex staked.
{
  "rows": [{
      "version": 0,
      "owner": "huangpanlong",
      "vote_stake": "0.0000 EOS",
      "rex_balance": "0.0000 REX",
      "matured_rex": 0,
      "rex_maturities": []
    }
  ],
  "more": true
}
  • As all we know the stake formula, voters.staked ≈ delband.net_weight + delband.cpu_weight + rexbal.vote_stake (rexbal.vote_stake constantly updated via the action updaterex), which means voters.staked >= delband.net_weight + delband.cpu_weight.
  • But for huangpanlong, the voters.staked: 40032.8596 EOS less than delband net&cpu weight: 40048.675 EOS, so there will be a 18.3154 EOS locked by voting system contract when huangpanlong trying to unstake CPU&NET more than 40032.8596 EOS.
Error
{
	"code": 500,
	"message": "Internal Service Error",
	"error": {
		"code": 3050003,
		"name": "eosio_assert_message_exception",
		"what": "eosio_assert_message assertion failure",
		"details": [{
			"message": "assertion failure with message: stake for voting cannot be negative",
			"file": "wasm_interface.cpp",
			"line_number": 964,
			"method": "eosio_assert"
		}, {
			"message": "pending console output: ",
			"file": "apply_context.cpp",
			"line_number": 113,
			"method": "exec_one"
		}]
	}
}

the assert code in system contract:

   void system_contract::update_voting_power( const name& voter, const asset& total_update )
   {
      auto voter_itr = _voters.find( voter.value );
      if( voter_itr == _voters.end() ) {
         voter_itr = _voters.emplace( voter, [&]( auto& v ) {
            v.owner  = voter;
            v.staked = total_update.amount;
         });
      } else {
         _voters.modify( voter_itr, same_payer, [&]( auto& v ) {
            v.staked += total_update.amount;
         });
      }
      check( 0 <= voter_itr->staked, "stake for voting cannot be negative" ); // HERE: the assertion
      if( voter == "b1"_n ) {
         validate_b1_vesting( voter_itr->staked );
      }
      if( voter_itr->producers.size() || voter_itr->proxy ) {
         update_votes( voter, voter_itr->proxy, voter_itr->producers, false );
      }
   }
  • As I inspected, the BUG happened a lot on those accounts who bought and sold rex, maybe they forgot to run updaterex action, but I also checked the code, rexbal.vote_stake also updated in buying and selling rex, so I failed to root cause the BUG by myself.
  • Please take the bug as high priority, I have found more than 600+ accounts has their EOS locked by system voting contract, and because I ignored the their rexbal staked, so there must be more EOS locked accounts than I searched, the attatchment is the list, Thanks!
    voter-stake-error-accounts.txt
@brianjohnson5972
Copy link
Contributor

brianjohnson5972 commented Mar 16, 2020

@shawnbow Did you also check the delband table for all other accounts that may have also staked on behalf of huangpanlong?
"cleos -u https://mainnet.eos.dfuse.io get table eosio delband"
finds all of the accounts with a "from" equal to "", like if you do
"cleos -u https://mainnet.eos.dfuse.io get table eosio eosio delband"
you get:
{
"rows": [{
"from": "eosio",
"to": "eosio.assert",
"net_weight": "1.0000 EOS",
"cpu_weight": "1.0000 EOS"
},{
"from": "eosio",
"to": "eosio.lost",
"net_weight": "1.0000 EOS",
"cpu_weight": "10.0000 EOS"
}
],
"more": false,
"next_key": ""
}
which are the 2 accounts that eosio has staked on behalf of. So if there are any accounts that have staked on behalf of huangpanlong, that would explain why your equation didn't add up.

@brianjohnson5972
Copy link
Contributor

Sorry, this wouldn't account for any difference here, it would account for a difference on the other ("from") account (if there was one) that was staking on the "to" accounts behalf.

@shawnbow
Copy link
Author

shawnbow commented Mar 17, 2020

@brianjohnson5972
I wrote a typescript script with eosjs to check more accounts that have the problem, and I did check the delband table for all accounts that have been staked on behalf of those accounts.
For example, this account: haztonrqhege:

root@ss:/opt# cleos -u https://mainnet.eos.dfuse.io get table eosio eosio voters -L haztonrqhege -l 1
{
  "rows": [{
      "owner": "haztonrqhege",
      "proxy": "proxy4nation",
      "producers": [],
      "staked": 1056150,  // 105.6150 EOS
      "last_vote_weight": "1282345165450.74853515625000000",
      "proxied_vote_weight": "0.00000000000000000",
      "is_proxy": 0,
      "flags1": 0,
      "reserved2": 0,
      "reserved3": "0.0000 EOS"
    }
  ],
  "more": true,
  "next_key": "7619978758482991264"
}
root@ss:/opt# cleos -u https://mainnet.eos.dfuse.io get table eosio haztonrqhege delband
{
  "rows": [{
      "from": "haztonrqhege",
      "to": "haztonrqhege",
      "net_weight": "0.2000 EOS",
      "cpu_weight": "114.2747 EOS" // cpu+net: 114.4747 EOS 
    }
  ],
  "more": false,
  "next_key": ""
}

haztonrqhege has 114.4747 EOS staked on himself, but only has 105.6150 EOS voter-staked, which means when he tries to unstake more than 105.6150 EOS, he will get the "stake for voting cannot be negative" assert and 8.8597 EOS permanently locked sadly.

My script is located at: https://github.com/shawnbow/eosio-helper/blob/master/scripts/check_voter.ts, you can git clone https://github.com/shawnbow/eosio-helper/, and run ts-node ./scripts/check_voter.ts to get part of locked accounts.
Attached is some of those poor accounts, you can see an account has 90+ EOS locked, it's so terrible for him.
eos-stake-locked-accounts.txt

@shawnbow
Copy link
Author

@brianjohnson5972
I could provide a fix for the issue as:

   void system_contract::update_voting_power( const name& voter, const asset& total_update )
   {
      auto voter_itr = _voters.find( voter.value );
      if( voter_itr == _voters.end() ) {
         voter_itr = _voters.emplace( voter, [&]( auto& v ) {
            v.owner  = voter;
            v.staked = total_update.amount;
         });
      } else {
         _voters.modify( voter_itr, same_payer, [&]( auto& v ) {
            v.staked += total_update.amount;
+            if (v.staked < 0) {
+              v.staked = 0;
+            }
         });
      }

      check( 0 <= voter_itr->staked, "stake for voting cannot be negative" );

      if( voter == "b1"_n ) {
         validate_b1_vesting( voter_itr->staked );
      }

      if( voter_itr->producers.size() || voter_itr->proxy ) {
         update_votes( voter, voter_itr->proxy, voter_itr->producers, false );
      }
   }

But I don't know whether it's the best solution, just for your reference.

@cic8ino
Copy link

cic8ino commented May 8, 2020

I'm one of them. Account gy2danbvgyge. I get stake for voting cannot be negative when I try to unstake 0.259 eos. I know this is a very low value, but I wanted to let you know

@shawnbow
Copy link
Author

shawnbow commented May 8, 2020

@cic8ino , many accounts have much more than your locked EOS, but seems developers of EOSIO don't have time to fix the serious bug.

@TimothyBanks TimothyBanks self-assigned this Jun 26, 2020
@TimothyBanks
Copy link

My understanding is that the rex sell order may not happen immediately. Could you check the rexqueue table to see if there are any orders that haven't been processed or that are closed?

@shawnbow
Copy link
Author

shawnbow commented Jul 10, 2020

@TimothyBanks, I've checked rexqueue table, there is no order for those locked EOS accounts.
https://bloks.io/account/safevault.x#votes, you can check this account, which has 28 EOS locked.

@mgbtcfv
Copy link

mgbtcfv commented Apr 8, 2021

What is the update on this?

@deckb
Copy link
Contributor

deckb commented Apr 9, 2021

I am currently investigating the issue.

@deckb
Copy link
Contributor

deckb commented Apr 21, 2021

Moving this to the new reference system contract repo.

I believe I have a test that replicates this issue. I believe I have a fix and will look to get it out soon.

@burstup
Copy link

burstup commented May 16, 2021

Hello. I have this problem as well. My EOS are all stuck in REX, and I get the "stake for voting cannot be negative" error. What can I do to resolve this?

@deckb deckb mentioned this issue Jun 21, 2021
2 tasks
@bks0115
Copy link

bks0115 commented Sep 25, 2021

I have 89.7 eos I accidentally sent to wrong address. On the explorer it shows it's an unused wallet. A Genesis account. Never used my deposit is the only transaction it shows. I've reached out to exchange I was trying to send to and they won't help. I don't know much about smart contracts etc. Trying to learn. I really need to get my tokens back. I can prove ownership and seen this thread hoping someone can help. stallingsbryant12@gmail.com

@burstup
Copy link

burstup commented Nov 4, 2021

This bug has still not been fixed as of November 11, 2021.

@ColinTalksCrypto
Copy link

ColinTalksCrypto commented Nov 14, 2021

Can someone please fix this bug? Some of my EOS is locked up in the REX and I can't remove it. Not a good user experience.

@mgbtcfv
Copy link

mgbtcfv commented Nov 14, 2021

What is the status on the fix? Can the core team post a comment here?

@mgbtcfv
Copy link

mgbtcfv commented Nov 14, 2021

@deckb Are you still working on this issue?

@tbfleming
Copy link

It's on my todo list to review deckb's workaround and backport it to https://github.com/eoscommunity/eosio.contracts/pulls . That repo is a temporary holding area for PRs to EOS until the community decides on a permanent location.

@tbfleming
Copy link

The PR is now in the backport holding area: eoscommunity/eosio.contracts#4

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants