Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set cache_precision = weights_precision in TBE if it is not explicitly set #3370

Closed
wants to merge 1 commit into from
Closed

Set cache_precision = weights_precision in TBE if it is not explicitly set #3370

wants to merge 1 commit into from

Commits on Nov 14, 2024

  1. Set cache_precision = weights_precision in TBE if it is not explicitl… 

    …y set (pytorch#3370)

Summary:

X-link: facebookresearch/FBGEMM#461

For historical reasons, we have decoupled cache_precision and weights_precision. This was to allow for lower precision embedding, while the hot ones in the cache are using higher precision. But it is not used.

This decoupling has been source of unintentional difference in cache_precision and weights_precision for typical EMO usecases where we do want cache_precision == weights_precision. We had enforced this in torchrec (see [this](https://www.internalfb.com/code/fbsource/[3868325cdafd]/fbcode/torchrec/distributed/batched_embedding_kernel.py?lines=962-963%2C1446-1447)), but as new stacks are enabled, this could be overwritten. 

Here we enforce cache_precision = weights_precision if cache precision is not explicitly set.

Differential Revision: D65865527
    @ehsanardestani @facebook-github-bot
    ehsanardestani authored and facebook-github-bot committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    dceb83d View commit details
    Browse the repository at this point in the history