Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization: Support batch size arguments for iterables #38

Open
Downchuck opened this issue Oct 2, 2015 · 0 comments
Open

Optimization: Support batch size arguments for iterables #38

Downchuck opened this issue Oct 2, 2015 · 0 comments

Comments

@Downchuck
Copy link

I'm going to take a peek at the performance on this -- if it's not worth it, I'll just close out the issue.

There may be overhead in the iterkeys/iteritems/itervalues calls when traversing a large set of data, which could be reduced by running a loop of iteration calls in the cython interface before returning results.

My goal is to show a significant increase in speed when traversing through more than a hundred thousand records. I'll just implement an iterbatch() method for the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant