Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up performance testing #100

Open
4 tasks
kissgyorgy opened this issue Dec 7, 2021 · 2 comments
Open
4 tasks

Set up performance testing #100

kissgyorgy opened this issue Dec 7, 2021 · 2 comments
Assignees
Labels
enhancement New feature or request performance performance improvements tasks

Comments

@kissgyorgy
Copy link
Contributor

kissgyorgy commented Dec 7, 2021

We need to measure how fast unblob as a whole can operate and what strategy can speed up extraction significantly.
Example question we want to answer: Which is faster? Matching on all YARA patterns at once or iterating on the file multiple times with less patterns?

Measure different scenarios:

  • One big file with few smaller files inside
  • Lots of small files concatenated and inside
  • Multiple big files concatenated and inside
  • Refact the priority handling by concatenating all YARA rules and handle the match results by priority instead of scanning a file multiple times. Measure the difference on various files.
@vlaci
Copy link
Contributor

vlaci commented Dec 7, 2021

We should measure how much wall time the different extraction steps take, e.g. Yara matching, chunk calculation, carving, extraction and so on.

@kissgyorgy
Copy link
Contributor Author

kissgyorgy commented Dec 8, 2021

There is pytest-benchmark which we can use to write benchmark tests with some special markers which would be ignored by default but can be easily selected to run.

@kissgyorgy kissgyorgy self-assigned this Dec 9, 2021
@vlaci vlaci assigned kissgyorgy and vlaci and unassigned kissgyorgy Jan 12, 2022
@qkaiser qkaiser added enhancement New feature or request performance performance improvements tasks labels Jan 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance performance improvements tasks
Projects
None yet
Development

No branches or pull requests

4 participants