Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VMTests tests for istanbul #1676

Merged
merged 82 commits into from
Jun 4, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
9265900
New upstream auto-VMTests tests for istanbul
feliam Apr 23, 2020
7b256c9
Removed redundant tests
feliam Apr 23, 2020
6119235
bugfix
feliam Apr 23, 2020
39f182e
Reset CI
feliam Apr 24, 2020
589b7cd
Debug CI 1
feliam Apr 24, 2020
5b0e780
debug CI
feliam Apr 24, 2020
55cb7ad
wakey wakey GitHub Actions
feliam Apr 24, 2020
76c9cc1
Poke GH
Apr 24, 2020
d98bb48
Fix detector
feliam Apr 24, 2020
139d9aa
Merge branch 'dev-evm-vmtests' of github.com:trailofbits/manticore in…
feliam Apr 24, 2020
2658ec0
master merge weaky weaky
feliam Apr 24, 2020
0065173
New CI weaky weaky
feliam Apr 24, 2020
c39fcad
Change default smtlib max memory usage weaky weaky
feliam Apr 24, 2020
a980190
ci weaky waky
feliam Apr 24, 2020
f077ae6
ci weaky waky
feliam Apr 24, 2020
8ec27e2
blkn waky weaky
feliam Apr 24, 2020
9af79d6
CI waky weaky
feliam Apr 24, 2020
17db4c1
Fix test_general weaky weaky
feliam Apr 26, 2020
dc4e726
Fix ethtests. weaky waky
feliam Apr 28, 2020
fa9bf51
merge
feliam Apr 28, 2020
375d876
Unittest driven bugfixes. weaky waky
feliam Apr 29, 2020
91a1464
Avoid check-sat in the middle of a solving. weaky wakey
feliam Apr 30, 2020
f85a1a4
Fix config. weaky wakey
feliam Apr 30, 2020
9d65078
Reduce queries in IO detector
feliam Apr 30, 2020
f964723
CC
feliam Apr 30, 2020
edb0fbd
CC
feliam Apr 30, 2020
caba86c
fix nativve tests
feliam May 1, 2020
32f7791
CC and solver defaults
feliam May 1, 2020
f83bb0e
merge. weaky waky
feliam May 1, 2020
8d2dc29
merge. weaky waky
feliam May 1, 2020
ed310a6
Fixing tests
feliam May 4, 2020
98e92b1
Remove debug print
feliam May 4, 2020
7e80962
Fix in DetectUninitializedMemory
feliam May 4, 2020
384acf7
Change truffle test state count
feliam May 4, 2020
c8c1ab6
CC
feliam May 4, 2020
1a07471
Merge branch 'master' into dev-evm-vmtests
feliam May 4, 2020
e22e2aa
Better forking on failed transactions
feliam May 5, 2020
891802a
CC
feliam May 5, 2020
e5330a8
Fix truffle tst state count
feliam May 5, 2020
91c1915
Improve z3 speed
feliam May 5, 2020
69e509c
Fix truffle test state count
feliam May 5, 2020
dfc1fbb
Change default gas concretization in CALL
feliam May 5, 2020
82d094a
Better logging at fork
feliam May 5, 2020
9afb4fd
merge
feliam May 5, 2020
db74b17
Insane commit to fix several bugfixes and refator stuff
feliam May 12, 2020
86cd97a
black
feliam May 12, 2020
4ca610d
merge
feliam May 12, 2020
543504e
Fix selfdestruct test
feliam May 13, 2020
9ce3535
Fix some unittests
feliam May 13, 2020
2e3bd62
Default to bytes fo constant calldata
feliam May 13, 2020
c6f4937
Fix tests
feliam May 13, 2020
a569f6d
merge waky waky
feliam May 13, 2020
0a45ac8
Better doc
feliam May 13, 2020
51bb15a
https://github.com/trailofbits/manticore/pull/1676/files#r426927762
feliam May 20, 2020
11446aa
https://github.com/trailofbits/manticore/pull/1676/files#r426946198
feliam May 20, 2020
316ddf4
Leave native alone and fix pyevmasm dep
feliam May 20, 2020
26afd50
Merge branch 'master' into dev-evm-vmtests
feliam May 20, 2020
1108f72
Update manticore/platforms/evm.py
feliam May 20, 2020
a7ca2f5
Update manticore/platforms/evm.py
feliam May 20, 2020
561a6d9
isSeven
feliam May 20, 2020
492a22a
Merge branch 'dev-evm-vmtests' of github.com:trailofbits/manticore in…
feliam May 20, 2020
e83f6a9
OPTI/PESI
feliam May 20, 2020
327cf62
Truffle optimistic
feliam May 26, 2020
acfb7f2
truffle fix
feliam May 26, 2020
8e54edc
Truffle state count
feliam May 26, 2020
112b677
CC
feliam May 26, 2020
c075388
Fix tests PESI/OPTI
feliam May 26, 2020
362118f
Update manticore/core/state.py
feliam Jun 2, 2020
1903747
Better policy naming
feliam Jun 2, 2020
d26e25a
merge
feliam Jun 2, 2020
0bf1716
Update manticore/platforms/evm.py
feliam Jun 3, 2020
ab5a6b8
Update manticore/ethereum/manticore.py
feliam Jun 3, 2020
c3bf563
Update manticore/platforms/evm.py
feliam Jun 3, 2020
5fbeadc
Merge branch 'master' into dev-evm-vmtests
feliam Jun 3, 2020
aeefa3c
PESi PESSI
feliam Jun 3, 2020
09306fb
blkn
feliam Jun 3, 2020
36703b9
weaky
feliam Jun 3, 2020
1083bbd
wakey wakey
feliam Jun 3, 2020
96e0aa8
blkn
feliam Jun 3, 2020
4937c19
unreachable
feliam Jun 3, 2020
f49b491
Fix state count in checkpoint test (merge)
feliam Jun 3, 2020
f2edab4
remove unused exception var
feliam Jun 4, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
New upstream auto-VMTests tests for istanbul
  • Loading branch information
feliam committed Apr 23, 2020
commit 92659006d01c0a810d782bfd89e2af34f17b576f
22 changes: 11 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
type: ["ethereum_bench", "examples", "ethereum", "ethereum_vm", "native", "wasm", "wasm_sym", "other"]
type: ["ethereum_bench", "examples", "ethereum", "ethereum_vm", "ethereum_truffle", "native", "wasm", "wasm_sym", "other"]
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.6
Expand Down Expand Up @@ -114,13 +114,14 @@ jobs:
}

make_vmtests(){

DIR=`pwd`
if [ ! -f ethereum_vm/.done ]; then
echo "Automaking VMTests" `pwd`
cd ./tests/ && mkdir -p ethereum_vm/VMTests_concrete && mkdir -p ethereum_vm/VMTests_symbolic
rm -Rf vmtests; git clone https://github.com/ethereum/tests --depth=1 vmtests
for i in ./vmtests/VMTests/*; do python ./auto_generators/make_VMTests.py $i; done
for i in ./vmtests/VMTests/*; do python ./auto_generators/make_VMTests.py $i --symbolic; done
for i in ./vmtests/BlockchainTests/ValidBlocks/VMTests/*/*json; do python ./auto_generators/make_VMTests.py -f istanbul -i $i -vvv -o ethereum_vm/VMTests_concrete; done
rm ethereum_vm/VMTests_concrete/test_loop*.py #too slow
rm -rf ./vmtests
touch ethereum_vm/.done
fi
Expand Down Expand Up @@ -165,8 +166,8 @@ jobs:
# but Manticore fails to explore the paths due to the lack of the 0x1f opcode support
# see https://github.com/trailofbits/manticore/issues/1166
# if [ "$(ls output/*tx -l | wc -l)" != "41" ]; then
if [ "$(ls output/*tx -l | wc -l)" != "34" ]; then
echo "Truffle test failed" `ls output/*tx -l | wc -l` "!= 34"
if [ "$(ls output/*tx -l | wc -l)" != "32" ]; then
echo "Truffle test failed" `ls output/*tx -l | wc -l` "!= 32"
return 1
fi
echo "Truffle test succeded"
Expand All @@ -176,7 +177,8 @@ jobs:

run_tests_from_dir() {
DIR=$1
pytest --durations=100 --cov=manticore -n auto "tests/$DIR"
echo "Running only the tests from 'tests/$DIR' directory"
pytest -v --durations=100 --cov=manticore -n auto "tests/$DIR"
coverage xml
}

Expand All @@ -201,18 +203,17 @@ jobs:
case $TEST_TYPE in
ethereum_vm)
make_vmtests
echo "Running only the tests from 'tests/$TEST_TYPE' directory"
run_tests_from_dir $TEST_TYPE
RV=$?

;;
ethereum_truffle)
echo "Running truffle test"
install_truffle
run_truffle_tests
RV=$(($RV + $?))
RV=$?
;;
wasm)
make_wasm_tests
echo "Running only the tests from 'tests/$TEST_TYPE' directory"
run_tests_from_dir $TEST_TYPE
RV=$?
;;
Expand All @@ -222,7 +223,6 @@ jobs:
ethereum) ;& # Fallthrough
ethereum_bench) ;& # Fallthrough
other)
echo "Running only the tests from 'tests/$TEST_TYPE' directory"
run_tests_from_dir $TEST_TYPE
RV=$?
;;
Expand Down
1 change: 1 addition & 0 deletions manticore/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,6 @@
from .utils.log import set_verbosity
from .core.smtlib import issymbolic, istainted
from .ethereum.manticore import ManticoreEVM
from .core.plugin import Plugin

__all__ = [issymbolic.__name__, istainted.__name__, ManticoreEVM.__name__, set_verbosity.__name__]
33 changes: 24 additions & 9 deletions manticore/core/smtlib/expression.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,10 @@ def __rxor__(self, other):
return BoolXor(self.cast(other), self)

def __bool__(self):
from .visitors import simplify
x = simplify(self)
if isinstance(x, Constant):
return x.value
raise NotImplementedError("__bool__ for Bool")


Expand Down Expand Up @@ -479,7 +483,7 @@ class BitVecConstant(BitVec):
__slots__ = ["_value"]

def __init__(self, size: int, value: int, *args, **kwargs):
self._value = value
self._value = value & ((1 << size) - 1)
super().__init__(size, *args, **kwargs)

def __bool__(self):
Expand Down Expand Up @@ -798,6 +802,7 @@ def underlying_variable(self):
return array

def read_BE(self, address, size):
address = self.cast_index(address)
bytes = []
for offset in range(size):
bytes.append(self.get(address + offset, 0))
Expand Down Expand Up @@ -1072,6 +1077,7 @@ def store(self, index, value):
self._concrete_cache = {}

# potentially generate and update .written set
# if index is symbolic it may overwrite other previous indexes
self.written.add(index)
self._array = self._array.store(index, value)
return self
Expand Down Expand Up @@ -1150,11 +1156,17 @@ def is_known(self, index):

is_known_index = BoolConstant(False)
written = self.written
if isinstance(index, Constant):
if index.value in {x.value for x in written if isinstance(x, Constant)}:
return BoolConstant(True)

for known_index in {x for x in written if not isinstance(x, Constant)}:
is_known_index = BoolOr(is_known_index.cast(index == known_index), is_known_index)
return is_known_index

for known_index in written:
if isinstance(index, Constant) and isinstance(known_index, Constant):
if known_index.value == index.value:
return BoolConstant(True)
is_known_index = BoolOr(is_known_index.cast(index == known_index), is_known_index)

return is_known_index

def get(self, index, default=None):
Expand All @@ -1171,12 +1183,15 @@ def get(self, index, default=None):
if isinstance(index, Constant) and index.value in self._concrete_cache:
return self._concrete_cache[index.value]

value = self._array.select(index)
if default is None:
return value
if default is not None:
default = self.cast_value(default)
is_known = self.is_known(index)
if isinstance(is_known, Constant) and is_known.value == False:
return default
else:
return self._array.select(index)

is_known = self.is_known(index)
default = self.cast_value(default)
value = self._array.select(index)
return BitVecITE(self._array.value_bits, is_known, value, default)


Expand Down
10 changes: 6 additions & 4 deletions manticore/core/smtlib/visitors.py
Original file line number Diff line number Diff line change
Expand Up @@ -325,12 +325,12 @@ def visit_BitVecSignExtend(self, expression, *operands):
return operands[0]

def visit_BitVecExtract(self, expression, *operands):
if all(isinstance(o, Constant) for o in expression.operands):
value = expression.operands[0].value
if all(isinstance(o, Constant) for o in operands):
value = operands[0].value
begining = expression.begining
end = expression.end
value = value >> begining
mask = 2 ** (end - begining + 1) - 1
mask = (1 << (end - begining)) - 1
ehennenfent marked this conversation as resolved.
Show resolved Hide resolved
value = value & mask
return BitVecConstant(expression.size, value, taint=expression.taint)

Expand Down Expand Up @@ -618,6 +618,8 @@ def visit_BitVecExtract(self, expression, *operands):
new_operands.append(BitVecExtract(item, begining, item.size - begining))
size -= item.size - begining
begining = 0
elif isinstance(op, BitVecConstant):
return BitVecConstant(size, (op.value >> begining) & ~(1 << size))

if isinstance(op, (BitVecAnd, BitVecOr, BitVecXor)):
bitoperand_a, bitoperand_b = op.operands
Expand Down Expand Up @@ -767,7 +769,7 @@ def visit_Expression(self, expression, *operands):
return expression


arithmetic_simplifier_cache = CacheDict(max_size=150000, flush_perc=25)
arithmetic_simplifier_cache = CacheDict(max_size=250000, flush_perc=25)


@lru_cache(maxsize=128, typed=True)
Expand Down
2 changes: 1 addition & 1 deletion manticore/ethereum/account.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ def f(
signature: Optional[str] = None,
caller=None,
value=0,
gas=0xFFFFFFFFFFFF,
gas=210000,
**kwargs,
):
try:
Expand Down
Loading