Potentially unreliable feature extraction on ELF sample

[mr-tz]

When running below rule on 294b8db1f2702b60fb2e42fdc50c2cee6a5046112da9a5703a548a4fa50477bc vivisect sometimes extract the localtime api feature and sometimes it doesn't. Is there a bug in the extractor, vivisect, or elsewhere?

rule:
  meta:
    name: get system time on Linux
    namespace: host-interaction/os/info
    authors:
      - "@ramen0x3f"
    scope: function
    examples:
      - 294b8db1f2702b60fb2e42fdc50c2cee6a5046112da9a5703a548a4fa50477bc:0x404970
  features:
    - and:
      - os: linux
      - or:
        - api: localtime

ref: mandiant/capa-rules#654

[williballenthin]

agree that the feature is not extracted:

viv doesn't seem to be finding the surrounding function (sub_404970):

[williballenthin]

viv doesnt recognize the reference to function sub_40BB10 (that calls sub_404970)

[williballenthin]

prolog of that function is quite standard:

[mr-tz]

Sometimes it does though?! I'm pretty sure I saw runs where vivisect worked...

[williballenthin]

i haven't seen it work on my machine, but if it's an issue of timing or something, that's maybe not surprising. if you can try a few more times and confirm that it sometimes works then i'll dive more into viv.

[williballenthin]

it seems like the emucode analyzer is supposed to work here. digging into why it doesn't work as expected.

[mr-tz]

Interesting! No match on new workspace every time:

> while true; do rm ../tests/data/294b8db1f2702b60fb2e42fdc50c2cee6a5046112da9a5703a548a4fa50477bc.elf_.viv; ~/venv/bin/python ../scripts/show-features.py ../tests/data/294b8db1f2702b60fb2e42fdc50c2cee6a5046112da9a5703a548a4fa50477bc.elf_ | grep "api(localtime)"; done

Match, when resusing existing workspace!?

> while true; do ~/venv/bin/python ../scripts/show-features.py ../tests/data/294b8db1f2702b60fb2e42fdc50c2cee6a5046112da9a5703a548a4fa50477bc.elf_ | grep -a "api(localtime)"; done
  insn: 0x404A45: api(localtime)
  insn: 0x404A45: api(localtime)
  insn: 0x404A45: api(localtime)

[mr-tz]

All exported features for a run that includes api(localtime):
feats.txt

[williballenthin]

root cause reported upstream. we'll be blocked on viv here, unless we want to introduce our own viv passes with capa to better handle things like this.