Ability to stop processing certain files

[martonilles]

It would be important for the user to be able to stop processing at the “right” level, extracting too much details would just result in garbage, too little might not extract an important part of the firmware. Ideally we want to guess the right level otherwise it would require lot of automated work.

Unfortunately just specifying depth is not enough as different firmware could contain different level of packaging.

We want to be able to stop processing based on the file types, especially the following types should not be processed further as most likely that would result in some false-positive detection:

• media files (pictures, video content)
• office documents
• executable (ELF binaries & libraries)

We still want to extract other packaged contents:

• tar.gz/bz/etc.
• cab
• jar
• apk
• deb
• etc.

Preferably it should be configurable what file types to stop processing with a default specified above.

We can use file-magic to determine the file type. (We just want to detect if the whole file is a given type so file magic is probably a good enough option for us.

• polyglot files could be a problem (eg: gif/rar or firmware images starting with a logo.png, but we can take care of those once see more examples)

[martonilles]

• libmagic is for skipping from processing
• record magic/mime into metadata
  • make sure that "deepest" depth has metadata recorded even for files that are not processed
  • we can also run libmagic detection on carved out files as well, even though those are not processed, just extracted