Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding HNSW-QG AVX512f version #175

Merged
merged 1 commit into from
Aug 29, 2022
Merged

Conversation

Patrick-H-Chen
Copy link
Contributor

Issue #, if available:

Description of changes:

This change reflects the implementation of product quantization with AVX512f SIMD on top of HNSW to perform fast quantization-based graph search. Following are two major changes:

  1. Adding a ProductQuantizer with 4 bits local centroids.
  2. A HNSWProductQuantizer4Bits struct which leverages ProductQuantization as the distance calculation instead of original full L2 distance.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

pecos/core/ann/quantizer.hpp Fixed Show fixed Hide fixed
pecos/core/ann/quantizer.hpp Fixed Show fixed Hide fixed
pecos/core/ann/quantizer.hpp Fixed Show fixed Hide fixed
pecos/core/ann/quantizer.hpp Fixed Show fixed Hide fixed
pecos/core/ann/quantizer.hpp Fixed Show fixed Hide fixed
pecos/core/ann/hnsw.hpp Show resolved Hide resolved
pecos/core/ann/hnsw.hpp Show resolved Hide resolved
pecos/core/ann/quantizer_impl/common.hpp Outdated Show resolved Hide resolved
pecos/core/ann/hnsw.hpp Outdated Show resolved Hide resolved
pecos/core/ann/hnsw.hpp Outdated Show resolved Hide resolved
pecos/core/ann/hnsw.hpp Fixed Show fixed Hide fixed
pecos/core/ann/quantizer_impl/x86.hpp Show resolved Hide resolved
pecos/core/ann/quantizer_impl/x86.hpp Outdated Show resolved Hide resolved
pecos/core/ann/quantizer_impl/x86.hpp Outdated Show resolved Hide resolved
pecos/core/ann/quantizer_impl/x86.hpp Show resolved Hide resolved
pecos/core/ann/quantizer_impl/x86.hpp Outdated Show resolved Hide resolved
pecos/core/ann/quantizer_impl/x86.hpp Show resolved Hide resolved
pecos/core/ann/quantizer_impl/common.hpp Outdated Show resolved Hide resolved
pecos/core/ann/quantizer_impl/x86.hpp Show resolved Hide resolved
pecos/core/ann/hnsw.hpp Show resolved Hide resolved
pecos/core/ann/hnsw.hpp Outdated Show resolved Hide resolved
2nd fix of attribute ‘target(default)’ is not valid

3rd fix by moving pad_parameters into quantizer

Revise Quantizer by adding commoh.hpp

fix typo in default quantizer

Revise FeatVec_T and Quantizer #centroids

revise quantizer train, and some other minor fixes

add some comments on variables

Adding HNSW-QG AVX512f version

add type checker
@OctoberChang OctoberChang merged commit 4e7ff32 into amzn:mainline Aug 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants