-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
StringTable in extractor uses a lot of memory #14
Comments
Have you considered the unordered_map branch of stxxl (link: https://stxxl.svn.sourceforge.net/svnroot/stxxl/branches/unordered_map/)? Not sure how stable the branch is. |
Last time I checked with the student it was more or less running fine,
|
Flagging this as feature request since there is at least no leak. |
Fixed. It was an unnecessary memory allocation. |
Extractor uses a hash table to keep track of previously encountered strings to only store them once. Unfortunately, that table uses a lot of RAM on large extracts.
The text was updated successfully, but these errors were encountered: