non-compiled yara signature files?

[williballenthin]

Hi all,

Thanks for doing the (extensive) hard work to collect and format type and function signature information! I'm impressed by both the quantity and quality of the support files here.

Over at https://github.com/fireeye/flare-floss, we're planning to add better detection of statically-linked routines. This will help improve performance as we can skip the emulation of library code. The support files you've published here look like a great start.

We're not yet decided to add a dependency on yara. Therefore, would you be willing to publish the non-compiled yara signature for statically-linked functions? Then, we would translate the signatures into a representation that our analysis engine supports (vivisect uses "vamp" signatures).

Thanks!

[PeterMatula]

Hi,
ok, we will publish the text YARA files. We have not decided yet how will we do it (add them to the support package, create a separate package, something else ...) - I will get back to you when it is decided and done.

In the meantime, see the wiki page (if you have not read it already) - our non-msvc signatures are mostly for older compilers we use in regression tests and I don't think they would be very usefull in other scenarios. You can however generate your own signatures in YARA text format and do whatever you want with them.

[williballenthin]

Thanks @PeterMatula!

The wiki page was a good reference - thanks for pointing that out. I'll likely play with the utilities for generating our own signatures so that I have a good understanding of the process.

[PeterMatula]

I just uploaded package with YARA text rules for statically linked code:
https://github.com/avast-tl/retdec-support/releases/tag/2018-04-09

Few notes:

• static-code/pe/32/le/x86/delphi/kb2005.yara is missing in this release. For whatever reason, the generation toolchain thinks the source database is corrupted and does not generate any YARA rules. I don't know what changed from before. I will look into it, but not right away.
• Right now, the RetDec's master branch is not fully using all info in YARA signatures (see Use references in statically linked code detection retdec#113). As I was working on this in RetDec's branch issue-116, I realized some improvements could be made to signature format (e.g. adding types to references, i.e. are they functions, data objects, etc.?). If you end up using these signatures in any way, and you realize something could be improved, contact us, we may discuss it further.
• The code for checking references in matched rules I wrote in issue-116 as part of bin2llvmir may become a separate library that would allow full signature matching in any application.
• Take a look at retdec-bin2pat, if you plan to generate signatures in your own format, output of this may be more suitable for you then the final results of retdec-pat2yara. We ourselves were not (and still are not) sure how should the ideal signatures look like. Therefore we decided to split its generation into 2 parts:
  • retdec-bin2pat gives you a full info about function in YARA (PAT = pattern = YARA). This process should not change much and its results can be further used to generate final signatures with different properties.
  • retdec-pat2yara processes this info into the final YARA rules.