System Call Audit

[ehennenfent]

As of 03/06/19, Manticore supports roughly 1/4th of the Linux system calls. For most binaries this is fine, but sometimes we run into binaries that require unimplemented system calls. The dev-syscall-audit branch will track efforts to evaluate which of the unimplemented system calls are most important to implement, and identify discrepancies between our implementations and the behavior expected by Linux programs.

I've started off by adding auto-generated stubs for most of the currently unimplemented calls. These stubs change the default behavior, so that instead of throwing a SyscallNotDefined exception when manticore reaches an unimplemented syscall, it will simply ignore it and continue execution.

This PR adds the wrapt library as a dependency. It's used for making sure that decorators properly copy wrapped function attributes. It's used to fix the problem of wrapped system calls having incorrect argument signatures

---

This change is 

[ehennenfent]

Reviewable status: 0 of 16 files reviewed, 2 unresolved discussions (waiting on @ekilmer, @feliam, and @Mossberg)

---

manticore/core/manticore.py, line 868 at r4 (raw file):

        self.remove_all()

        self._save_run_data()

@feliam Would like to get your thoughts on this. Not sure if you removed this call for a reason but . I noticed that the output directory and total time were no longer being printed.

---

manticore/platforms/platform.py, line 14 at r1 (raw file):

Previously, ekilmer (Eric Kilmer) wrote…

The change looks good! Thank you 👍

Resolved. Also added did/will_execute_syscall callbacks that should make this cleaner

[feliam]

Reviewable status: 0 of 16 files reviewed, 2 unresolved discussions (waiting on @ehennenfent, @ekilmer, @feliam, and @Mossberg)

---

manticore/core/manticore.py, line 868 at r4 (raw file):

Previously, ehennenfent (Eric Hennenfent) wrote…

@feliam Would like to get your thoughts on this. Not sure if you removed this call for a reason but . I noticed that the output directory and total time were no longer being printed.

It's friendlier to have it in a context[] item. So it will count the time of a run() iff you finalize immediatelly after that run(). Note that EVM will call run() several times, one for each tx. Not sure if "Total time" makes sense for EVM? I'd instantly agree to add the "save_run_data" to ManticoreLinux. The saving of the config file and (if not included in manticore.yml) the commandline needs work in general.

[ehennenfent]

Reviewable status: 0 of 18 files reviewed, 2 unresolved discussions (waiting on @ekilmer, @feliam, and @Mossberg)

---

manticore/core/manticore.py, line 868 at r4 (raw file):

Previously, feliam (feliam) wrote…

It's friendlier to have it in a context[] item. So it will count the time of a run() iff you finalize immediatelly after that run(). Note that EVM will call run() several times, one for each tx. Not sure if "Total time" makes sense for EVM? I'd instantly agree to add the "save_run_data" to ManticoreLinux. The saving of the config file and (if not included in manticore.yml) the commandline needs work in general.

Okay, moved this implementation to native/manticore.py for now. I notice that even with the old implementation, it only gets triggered when one explicitly calls finalize, which doesn't seem to happen from the command line tool. Was that deliberate?

[feliam]

Reviewable status: 0 of 18 files reviewed, 2 unresolved discussions (waiting on @ehennenfent, @ekilmer, and @Mossberg)

---

manticore/core/manticore.py, line 868 at r4 (raw file):

Previously, ehennenfent (Eric Hennenfent) wrote…

Okay, moved this implementation to native/manticore.py for now. I notice that even with the old implementation, it only gets triggered when one explicitly calls finalize, which doesn't seem to happen from the command line tool. Was that deliberate?

I think it was not deliberate. Previously linux generated the testcases online. We need to add an explicit finalize() somewhere in the Linux flavor to replicate old and expected behavior. Possible at main() in native/cli.py?

[feliam]

Reviewable status: 0 of 18 files reviewed, 7 unresolved discussions (waiting on @ehennenfent, @ekilmer, and @Mossberg)

---

examples/script/aarch64/basic.py, line 26 at r5 (raw file):

@m.init
def init(m, ready_states):
    for state in ready_states:

The alternative to consider is to do it from Manticore. Like m.ready_states instead of passing it as an explicit argument.

---

manticore/core/manticore.py, line 815 at r5 (raw file):

        self._publish('will_run', self.ready_states)
        with self.locked_context() as context:
            context['time_started'] = time.time()

Maybe move this to ManticoreNative.run() ?

---

manticore/core/manticore.py, line 877 at r5 (raw file):

    ############################################################################

    def _save_run_data(self):

Delete this?

---

manticore/core/worker.py, line 161 at r5 (raw file):

                        # Saved to a fresh id in case other worker have an old
                        # version this state cached over the old id
                        m._publish('will_terminate_state', current_state, exc)

This should not happen normally. The state here is in an exceptional condition like having a not feasible path constraint or other impossible thing. Not 100% sure the callbacks can deal with this kind of states? Maybe yes.

---

manticore/native/memory.py, line 383 at r5 (raw file):

    def __del__(self):
        if hasattr(self, '_data'):

maybe the getattr way? or not.

[ehennenfent]

Reviewable status: 0 of 18 files reviewed, 6 unresolved discussions (waiting on @ekilmer, @feliam, and @Mossberg)

---

examples/script/aarch64/basic.py, line 26 at r5 (raw file):

Previously, feliam (feliam) wrote…

The alternative to consider is to do it from Manticore. Like m.ready_states instead of passing it as an explicit argument.

Hmm, open to discussion on this. I figured adding a param would make it more clear that the API had changed. I suppose we really ought to make a formal spec for all the plugin callbacks and then inspect them before actually invoking

---

manticore/core/manticore.py, line 868 at r4 (raw file):

Previously, feliam (feliam) wrote…

I think it was not deliberate. Previously linux generated the testcases online. We need to add an explicit finalize() somewhere in the Linux flavor to replicate old and expected behavior. Possible at main() in native/cli.py?

Done. The old cli looped over all the states and called generate_testcase. Now we just call finalize.

---

manticore/core/manticore.py, line 815 at r5 (raw file):

Previously, feliam (feliam) wrote…

Maybe move this to ManticoreNative.run() ?

Going to leave as-is for now since native/manticore doesn't have a run method and this could theoretically be useful for evm scripts too

---

manticore/core/manticore.py, line 877 at r5 (raw file):

Previously, feliam (feliam) wrote…

Delete this?

Done.

---

manticore/core/worker.py, line 161 at r5 (raw file):

Previously, feliam (feliam) wrote…

This should not happen normally. The state here is in an exceptional condition like having a not feasible path constraint or other impossible thing. Not 100% sure the callbacks can deal with this kind of states? Maybe yes.

It looks like most other callbacks ignore the exc parameter, so we should be good on that front. The reason I re-added this is that we have some internal tooling that needs to be able to catch exceptions in Manticore. Basically everything gets caught right here and squashed. With this callback, we can still get access to those, even if it's not with a try...except

---

manticore/native/memory.py, line 383 at r5 (raw file):

Previously, feliam (feliam) wrote…

maybe the getattr way? or not.

Not sure what you mean. There's not really a good default case to past to munmap afaik. It's just that we don't want to try to unmap a nonexistent page.

[feliam]

---

manticore/core/worker.py, line 161 at r5 (raw file):

Previously, ehennenfent (Eric Hennenfent) wrote…

It looks like most other callbacks ignore the exc parameter, so we should be good on that front. The reason I re-added this is that we have some internal tooling that needs to be able to catch exceptions in Manticore. Basically everything gets caught right here and squashed. With this callback, we can still get access to those, even if it's not with a try...except

The state here is not terminated but it is killed.
I think that we now have a list for exceptional states. "killed_states"
Maybe we need to have different events for killing and terminating a correct state.
Maybe this tools need to subscribe to "will_kill_state" ?

[feliam]

Reviewable status: 0 of 20 files reviewed, 3 unresolved discussions (waiting on @ehennenfent, @ekilmer, and @Mossberg)

---

manticore/core/manticore.py, line 815 at r5 (raw file):

Previously, ehennenfent (Eric Hennenfent) wrote…

Going to leave as-is for now since native/manticore doesn't have a run method and this could theoretically be useful for evm scripts too

import timeit; timet.timeit(m.run(..)).
It may be useful to evm scripts that want to time just the last transaction, ignore all the others and execute a finalize().

[ehennenfent]

Reviewable status: 0 of 20 files reviewed, 3 unresolved discussions (waiting on @ekilmer, @feliam, and @Mossberg)

---

manticore/core/manticore.py, line 815 at r5 (raw file):

Previously, feliam (feliam) wrote…

import timeit; timet.timeit(m.run(..)).
It may be useful to evm scripts that want to time just the last transaction, ignore all the others and execute a finalize().

Done.

---

manticore/core/worker.py, line 161 at r5 (raw file):

Previously, feliam (feliam) wrote…

The state here is not terminated but it is killed.
I think that we now have a list for exceptional states. "killed_states"
Maybe we need to have different events for killing and terminating a correct state.
Maybe this tools need to subscribe to "will_kill_state" ?

Done.