Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gzip compressed output reduces speed by two-thirds #31

Open
jblachly opened this issue Mar 23, 2016 · 1 comment
Open

gzip compressed output reduces speed by two-thirds #31

jblachly opened this issue Mar 23, 2016 · 1 comment

Comments

@jblachly
Copy link

please disregard the following and see the followup comment - speed issues tracked down to output compression option

When using -m any I can max out 8-9 cores (Xeon X5670 @ 2.93 GHz).

However, using the exact same adapter file (2 adapters), in either -m head or -m tail mode my CPU usage pegs at < 300%. This means up to 8 times longer runtime.

In addition, I am going to file a probably related bug in that 5 prime adapters are not found and removed when run in -m any mode, while they are in the slower but apparently more effective -m head mode.

@jblachly jblachly changed the title any mode is more efficient than head or tail gzip compressed output reduces speed by two-thirds Mar 23, 2016
@jblachly
Copy link
Author

I need to issue a correction to the above: it turns out the key difference was the -z parameter.

When doing gzip compression on the output, skewer's CPU usage maxes at around a third (3 full cores vs 9) of when not using -z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant