Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 64bit Min Sketch #53

Merged
merged 7 commits into from
Nov 4, 2022
Merged

Add 64bit Min Sketch #53

merged 7 commits into from
Nov 4, 2022

Conversation

jponf
Copy link

@jponf jponf commented Nov 2, 2022

Added a new CMS with cell size uint64_t. This should address #51.

To control the cell size I've added a new parameter so the user has a choice to use 32 or 64 bit. By default is set to 32 bit so old code should work exactly the same.

I'm open to address any issue or concern that you may have.

@piskvorky
Copy link
Owner

piskvorky commented Nov 2, 2022

This looks good, thanks! Why did the CI fail?

@jponf
Copy link
Author

jponf commented Nov 2, 2022

@piskvorky sorry my bad I didn't check that the CI runs on python 3.4 and 3.5, I've added f-strings that's probably why these are failing. As for 3.7 I have no clue because 3.6 seems ok I'll check it now.

Josep Pon Farreny added 2 commits November 2, 2022 11:15
The project CI runs on 3.4 and 3.5 which do not support f-strings.
@jponf
Copy link
Author

jponf commented Nov 2, 2022

@piskvorky I've fixed the string's formatting issue and it works on all versions except 3.7, looking at the logs it seems the error is comming from setuptools rather than the bounter package as it fails at the setuptools import line in setup.py

Maybe trying a newer version of 3.7 such as 3.7.9 instead of 3.7.1 fixes the CI pipeline.

/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_importlib.py:23: UserWarning: `importlib-metadata` version is incompatible with `setuptools`.
This problem is likely to be solved by installing an updated version of `importlib-metadata`.
  warnings.warn(msg)  # Ensure a descriptive message is shown.
Traceback (most recent call last):
  File "setup.py", line 16, in <module>
    from setuptools import setup, find_packages, Extension
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/__init__.py", line 18, in <module>
    from setuptools.dist import Distribution
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/dist.py", line 34, in <module>
    from ._importlib import metadata
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_importlib.py", line 39, in <module>
    disable_importlib_metadata_finder(metadata)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_importlib.py", line 12, in disable_importlib_metadata_finder
    import importlib_metadata
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 471, in <module>
    __version__ = version(__name__)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 438, in version
    return distribution(package).version
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 411, in distribution
    return Distribution.from_name(package)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 179, in from_name
    dists = resolver(name)
  File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 886, in find_distributions
    found = self._search_paths(context.name, context.path)
AttributeError: 'str' object has no attribute 'name'

@jponf
Copy link
Author

jponf commented Nov 2, 2022

I've tried it on my computer using python 3.7.1 installed via pyenv and it worked without further changes.

I've also found that this error has already been reported to pypa pypa/setuptools#3293, maybe updating setuptools via the requirements files fixes the issue in the 3.7.1 image used by travis.

@piskvorky
Copy link
Owner

@mpenkov could you please check? This looks like a useful change, let's get it merged in and released. Many thanks.

@jponf
Copy link
Author

jponf commented Nov 3, 2022

Hi @mpenkov thanks for fixing the issue with Python 3.7.

Sadly now it failed in 3.6, but it is weird, as before all tests passed and all now all other versions are passing them too. My guess is that there is a rounding error that may or may not happen as the offending test is

self.assertAlmostEqual(self.cms['big number'], big_number, delta=self.delta * big_number)

Can you rerun https://app.travis-ci.com/github/RaRe-Technologies/bounter/jobs/587411973? If the error is due to floating point rounding maybe it was just bad luck that we had that rounding error.

@mpenkov
Copy link
Collaborator

mpenkov commented Nov 3, 2022

Yeah, seems like a flickering test. Everything passes now. I'll leave you some minor comments, then we should be good to merge.

bounter/count_min_sketch.py Outdated Show resolved Hide resolved
bounter/count_min_sketch.py Outdated Show resolved Hide resolved
@mpenkov mpenkov merged commit b34ab50 into piskvorky:master Nov 4, 2022
@mpenkov
Copy link
Collaborator

mpenkov commented Nov 4, 2022

Merged! Thank you for your work @jponf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants