OOM on certain files when using Python module

[jwahsnakupaku]

Description

When using Capa as a library against certain files the process ends up just OOMing..

Steps to Reproduce

Find a file that triggers it, I can provide a few samples if that helps.
https://www.virustotal.com/gui/file/f88e1f280af5b7ea78a8f0f59fba910e54e2eaeb2f34611a7e36f33b505d2784
https://www.virustotal.com/gui/file/f0fa2602f5b65dd91ec0eb7626556f6e9d07ed39065a574e05974a0bc3651017

1. Run bulk-process.py script against the files, eg;

python3 bulk-process.py -r capa-rules-6.1.0/ -s sigs/ -n1 --no-mp /tmp/samples
INFO:capa:successfully loaded 1061 rules
INFO:capa:computing capa results for: /tmp/samples/f88e1f280af5b7ea78a8f0f59fba910e54e2eaeb2f34611a7e36f33b505d2784.exe
Killed

Expected behavior:
Runs successfully against the file.

Actual behavior:
Process is killed by the oom killer.

Versions

./capa --version
capa 6.1.0

pip list installed | grep -i flare
flare-capa 6.1.0

python3 --version
Python 3.8.10

Using Ubuntu 20.04

Additional Information

Interestingly, running the capa binary against the file works without any dramas.

[mr-tz]

Thanks for the detailed report. You mention that the capa binary works.
Do you mean the standalone binary or the Python script invocation?

[mr-tz]

I see what's going on. The bulk-processor script only analyzes samples as native PEs. These samples are .NET and when analyzed in vivisect causes the issues. So there's two issues:

• the bulk analyze script is not up-to-date (which we should change or at least document)
• vivisect has issues processing the files

[jwahsnakupaku]

Thanks for the detailed report. You mention that the capa binary works. Do you mean the standalone binary or the Python script invocation?

Standalone binary.

I see what's going on. The bulk-processor script only analyzes samples as native PEs. These samples are .NET and when analyzed in vivisect causes the issues. So there's two issues:

* the bulk analyze script is not up-to-date (which we should change or at least document)

* vivisect has issues processing the files

Oh cool, ran into this when using AssemblyLine, looks like they based the CAPA service on that bulk analyze script - https://github.com/CybercentreCanada/assemblyline-service-capa
They'll probably be keen to update the service to fix it.

[williballenthin]

the bulk-export bug should be fixed in #1948