Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

localdb 删除key存在问题 #1203

Closed
bysomeone opened this issue Dec 31, 2021 · 6 comments · Fixed by #1209
Closed

localdb 删除key存在问题 #1203

bysomeone opened this issue Dec 31, 2021 · 6 comments · Fixed by #1209
Labels
released

Comments

@bysomeone
Copy link
Collaborator

问题现象

开启execLocalSameTime, 设区块内3笔交易
交易1: 设置key value, value不为空
交易2: 删除key, 即将key对应value标记为nil
交易3, 获取key, 此时value非空, 为交易1设置的值

问题原因

由于execLocal和exec同步执行, 采用了交易缓存txCache和区块缓存cache
txCache忽略value为nil的key值设置, 交易执行完后 commit时, cache中没有同步设置对应key
此时cache中如果已经存在key value缓存, 后续获取key的数据是错误的
@bysomeone
Copy link
Collaborator Author

stateDB同样存在这样的问题, 且完全忽略value为nil的key设置, 要确认下是否正确 @vipwzw

@linj-disanbo
Copy link
Collaborator

问题现象补充

测试发现的其他问题:
在上诉情况下, 后续的localdb访问中发现 ListIndex 在后续还有数据时, 会返回ErrNotFound.

  1. ListIndex 参数 Primary == "" 调用正常
  2. ListIndex 参数 Primary == "上次的最后一个key", 可能出现问题, 失败时 返回 ErrNotFound . 但实际上并没有遍历完数据的情况下

问题原因:

可能原因: 怀疑 Del 可能有bug 导致哪里数据不对, 在特定的问题, 才会触发 List 报 NotFound

已经进行的尝试: 由于业务上可以发现这样的数据, 将对应的数据删除, ListIndex 在遍历数据时, 正常.

bysomeone added a commit to bysomeone/chain33 that referenced this issue Jan 4, 2022
@bysomeone
fix statedb key setting when value is nil(33cn#1203)
8ea9e0d
bysomeone added a commit to bysomeone/chain33 that referenced this issue Jan 7, 2022
@bysomeone
[[FIX]]add statedb nil value setting(33cn#1203)
ab8e0ea
33cn pushed a commit that referenced this issue Jan 10, 2022
@bysomeone @33cn
[[FIX]]add statedb nil value setting(#1203)
811e4a9
@linj-disanbo
Copy link
Collaborator

linj-disanbo commented Jan 10, 2022 •
edited
Loading

问题现象

用分支 chain33-fix-exec-del-key测试, 如果数据量超过一页的情况下, 调用ListIndex分页遍历数据 , 参数指定第一页的最后一条记录的key 为primaryKey, 返回依然为第一页的数据的. (预期为第二页数据)

添加日志发现, 在取第二页数据时, 在goLevelDBIt.Seek 中, 调用Seek时, 返回的不存在, 重新从开始取数据.

func (dbit *goLevelDBIt) Seek(key []byte) bool

日志

EROR[01-10|15:55:54] QueryMarketDepth.                        module=exchange.executor price(index)=coins.bty:USDT:1: primaryKey= count=10 Direction=0
EROR[01-10|15:55:54] QueryMarketDepth. 1                      module=exchange.executor price(index)=coins.bty:USDT:1: primaryKey= count=10 Direction=0
# 遍历时, 第一次 primaryKey 为空
EROR[01-10|15:55:54] table.ListIndex                          indexName=price prefix=coins.bty:USDT:1: primaryKey= count=10 dir=0
EROR[01-10|15:55:54] table.query.ListIndex                    indexName=price prefix=coins.bty:USDT:1: primaryKey= count=10 dir=0
EROR[01-10|15:55:54] table.query.ListIndex.listPrimary        prefix=coins.bty:USDT:1: primaryKey= count=10 dir=0 prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey=
EROR[01-10|15:55:54] table->executor.local.List               prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey= count=10 dir=0
EROR[01-10|15:55:54] table->executor->db.local.List           prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey= count=10 dir=0 mergedb has n db=3
EROR[01-10|15:55:54] table->executor->db.local->ListHelper.List prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey= count=10 dir=0 call=IteratorScanFromLast
EROR[01-10|15:55:54] QueryMarketDepth.                        module=exchange.executor left="execer:\"paracross\" symbol:\"coins.bty\"" right="execer:\"evmxgo\" symbol:\"USDT\"" len=10
EROR[01-10|15:55:54] QueryMarketDepth.                        module=exchange.executor left="execer:\"paracross\" symbol:\"coins.bty\"" right="execer:\"evmxgo\" symbol:\"USDT\"" len=10 key=coins.bty:USDT:1:0000000000040000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=12 count=0 priceKey=coins.bty:USDT:1:0000000000040000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=9000000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=8800000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=2000000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=1000000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=120000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=110000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=100000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=90000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=50000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=40000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=20 deepkey=coins.bty:USDT:1:0000000000040000

# 遍历时, 第二页, primaryKey=coins.bty:USDT:1:0000000000040000 (db/table 中)
# 在localdb 中, 自动加上前缀: primaryKey=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000
# goLevelDBIt Seek 日志发现, find 是false, 之后的是, seek 定位到第一条数据 (和第一页的第一条数据一致)
# Seek x=0 或 1的日志, 是两个内存缓存(txcache 和cache)的日志, 可以跳过
# Seek x=2 是 localdb 的日志 <----------
EROR[01-10|15:55:54] QueryMarketDepth.                        module=exchange.executor price(index)=coins.bty:USDT:1: primaryKey=coins.bty:USDT:1:0000000000040000 count=10 Direction=0
EROR[01-10|15:55:54] QueryMarketDepth 2.                      module=exchange.executor price(index)=coins.bty:USDT:1: primaryKey=coins.bty:USDT:1:0000000000040000 count=10 Direction=0
EROR[01-10|15:55:54] table.ListIndex                          indexName=price prefix=coins.bty:USDT:1: primaryKey=coins.bty:USDT:1:0000000000040000 count=10 dir=0
EROR[01-10|15:55:54] table.query.ListIndex                    indexName=price prefix=coins.bty:USDT:1: primaryKey=coins.bty:USDT:1:0000000000040000 count=10 dir=0
EROR[01-10|15:55:54] table.query.ListIndex.listPrimary        prefix=coins.bty:USDT:1: primaryKey=coins.bty:USDT:1:0000000000040000 count=10 dir=0 prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000
EROR[01-10|15:55:54] table->executor.local.List               prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 count=10 dir=0
EROR[01-10|15:55:54] table->executor->db.local.List           prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 count=10 dir=0 mergedb has n db=3
EROR[01-10|15:55:54] table->executor->db.local->ListHelper.List prefix=LODB-exchange-depth-d-coins.bty:USDT:1: primaryKey=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 count=10 dir=0 call=IteratorScan
EROR[01-10|15:55:54] mergedIterator Seek                      x=0
EROR[01-10|15:55:54] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false
EROR[01-10|15:55:54] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=
EROR[01-10|15:55:54] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=[] reverse=prev
EROR[01-10|15:55:54] mergedIterator Seek                      x=0 find=nil
EROR[01-10|15:55:54] mergedIterator Seek                      x=1
EROR[01-10|15:55:54] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false
EROR[01-10|15:55:54] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=
EROR[01-10|15:55:54] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=[] reverse=prev
EROR[01-10|15:55:54] mergedIterator Seek                      x=1 find=nil
EROR[01-10|15:55:54] mergedIterator Seek                      x=2
EROR[01-10|15:55:54] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false
EROR[01-10|15:55:54] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=
EROR[01-10|15:55:54] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=[] reverse=prev
EROR[01-10|15:55:54] mergedIterator Seek                      x=2 find=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000
EROR[01-10|15:55:54] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false
EROR[01-10|15:55:54] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=
EROR[01-10|15:55:54] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=[] reverse=prev
EROR[01-10|15:55:54] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false
EROR[01-10|15:55:54] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=
EROR[01-10|15:55:54] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=[] reverse=prev
EROR[01-10|15:55:54] QueryMarketDepth.                        module=exchange.executor left="execer:\"paracross\" symbol:\"coins.bty\"" right="execer:\"evmxgo\" symbol:\"USDT\"" len=10
EROR[01-10|15:55:54] QueryMarketDepth.                        module=exchange.executor left="execer:\"paracross\" symbol:\"coins.bty\"" right="execer:\"evmxgo\" symbol:\"USDT\"" len=10 key=coins.bty:USDT:1:0000000000040000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=12 count=0 priceKey=coins.bty:USDT:1:0000000000040000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=9000000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=8800000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=2000000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=1000000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=120000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=110000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=100000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=90000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=50000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13 count=0
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=13.200 count=0 price=40000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=20 deepkey=coins.bty:USDT:1:0000000000040000
EROR[01-10|15:55:54] matchLimitOrder_Fix1                     module=exchange.executor 1=21 deepkey=coins.bty:USDT:1:0000000000040000

加日志后的 Seek的代码

//Seek seek key
func (dbit *goLevelDBIt) Seek(key []byte) bool {
	exist := dbit.Iterator.Seek(key)
	log.Error("goLevelDBIt Seek1", "key", string(key), "find?", exist)
	dbKey := dbit.Key()
	log.Error("goLevelDBIt Seek2", "key", string(key), "find?", exist, "dbkey", string(dbKey))
	if dbit.reverse && !bytes.Equal(dbKey, key) {
		log.Error("goLevelDBIt Seek3", "key", string(key), "find?", exist, "dbkey", dbKey, "reverse", "prev")
		return dbit.Iterator.Prev() && dbit.Valid()
	}
	log.Error("goLevelDBIt Seek4", "key", string(key), "find?", exist, "dbkey", string(dbKey))
	return exist
}

@linj-disanbo
Copy link
Collaborator

补充日志: 在Seek x=2 情况下, goLevelDBIt Seek被调用了三次,

  1. 第一次是 mergedIterator.Seek 调用的
  2. 第2和第3次, 是 取数据时, Next内部调用的. (这里也有问题, 内部调整为key为指定第一条数据的时, seek 依然返回不存在)
    另外在Next 调用把取到数据的Key打印出来, 可以看出指定的key是存在的.
EROR[01-10|16:48:10] mergedIterator Seek                      x=2
EROR[01-10|16:48:10] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false
EROR[01-10|16:48:10] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=
EROR[01-10|16:48:10] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 find?=false dbkey=[] reverse=prev
EROR[01-10|16:48:10] mergedIterator Seek                      x=2 find=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000
EROR[01-10|16:48:10] Next->nextInternal dirSeek               call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000
EROR[01-10|16:48:10] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false
EROR[01-10|16:48:10] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=
EROR[01-10|16:48:10] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=[] reverse=prev
EROR[01-10|16:48:10] Next->nextInternal dirSeek               call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000
EROR[01-10|16:48:10] goLevelDBIt Seek1                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false
EROR[01-10|16:48:10] goLevelDBIt Seek2                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=
EROR[01-10|16:48:10] goLevelDBIt Seek3                        key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000009000000 find?=false dbkey=[] reverse=prev
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000008800000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000002000000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000001000000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000000120000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000000110000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000000100000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000000090000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000000050000
EROR[01-10|16:48:10] Next->nextInternal->Next                 call seek key=LODB-exchange-depth-d-coins.bty:USDT:1:0000000000040000

@linj-disanbo linj-disanbo mentioned this issue Jan 11, 2022
Open
@bysomeone
Copy link
Collaborator Author

image

确认下 这段代码是不是有问题 @vipwzw

在提供prefix和primaryKey情况下， prefix是primaryKey前缀， 最终list的k= p+prefix+primaryKey， 中间多了一个prefix

交易合约这个例子，就是
LODB-exchange-depth-d-coins.bty:USDT:1:coins.bty:USDT:1:0000000000040000 （用来list的参数key）
LODB-exchange-depth-d-coins.bty:USDT:1:0000000000040000 （数据库中实际存的key）

prefix参数 coins.bty:USDT:1:

bysomeone added a commit to bysomeone/chain33 that referenced this issue Jan 12, 2022
@bysomeone
[[FIX]] fix list table primary key(33cn#1203)
294b73c
@bysomeone bysomeone mentioned this issue Jan 13, 2022
Merged
vipwzw pushed a commit that referenced this issue Jan 14, 2022
@bysomeone @vipwzw
[[FIX]] fix list table primary key(#1203)
b82008a
33cn pushed a commit that referenced this issue Jan 14, 2022
@semantic-release-bot
chore(release): 1.66.3 [skip ci]
89ce5be 
<a name="1.66.3"></a>
## [1.66.3](v1.66.2...v1.66.3) (2022-01-14)

### Bug Fixes

* fix list table primary key(#1203) ([b82008a](b82008a)), closes [#1203](#1203)

## [1.66.3](v1.66.2...v1.66.3) (2022-01-14)
@33cn
Copy link
Owner

33cn commented Jan 14, 2022

🎉 This issue has been resolved in version 1.66.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@33cn 33cn added the released label Jan 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
released
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix list table primary key bysomeone/chain33
3 participants
@bysomeone @33cn @linj-disanbo