Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TensorFlow pakage check #26842

Merged
merged 1 commit into from
Oct 17, 2023
Merged

Conversation

jayfurmanek
Copy link
Contributor

What does this PR do?

This fixes the _is_package_available check for tensorflow variants (like tensorflow-rocm) where their meta-data differs from the package name. There is an list of available candidate names there, but they are never used since _is_package_available will return false preventing the candidate list from being checked.
This also adds tf-nightly-rocm to the list of candidates.

Fixes # (issue)
N/A

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

This fixes a small functional error, no documentation updates are required.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@Rocketknight1
Copy link
Member

Taking this one!

@Rocketknight1
Copy link
Member

Rocketknight1 commented Oct 16, 2023

Hey @jayfurmanek, I checked this out! Adding tf-nightly-rocm to the candidate list is appreciated, but I don't think the other changes are required, and they seem to break a lot of the codebase! Can we revert that part of the PR and just keep the package name addition, or did that not work for you?

Update: Part of the issue might be that importlib.metadata is typoed as importlib_metadata - but I think we still should be okay if we just revert that change!

@jayfurmanek
Copy link
Contributor Author

jayfurmanek commented Oct 16, 2023

The problem with the check as-is is that it does _is_package_available() with "tensorflow" and then does the meta data check there as well. For tensorflow-rocm that part will fail and _is_available will be marked as False, so the candidates list below is skipped altogether.

@Rocketknight1
Copy link
Member

Hi @jayfurmanek I just tested locally and installing with pip install tensorflow-rocm works fine, and causes transformers.is_tf_available() to correctly return True. Can you tell me where I can get a wheel for tf-nightly-rocm so I can test that?

@jayfurmanek
Copy link
Contributor Author

@Rocketknight1
Copy link
Member

@jayfurmanek I confirmed after installing one of those wheels that just adding tf-nightly-rocm to the list is sufficient! Can we revert the rest of the PR and just keep that bit?

@Rocketknight1
Copy link
Member

Rocketknight1 commented Oct 17, 2023

If you want to test it yourself, you can run transformers.is_tf_available() to see if transformers can detect a TF installation. It works fine for me once the list is updated, with no other changes.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@jayfurmanek
Copy link
Contributor Author

I was using is_package_available() for tf not is_tf_available. With is_tf_available() it works.
I still think is_package_available() is a bit wrong though.
PR updated.

thx

@Rocketknight1
Copy link
Member

Yeah, that code is a little wonky, and we'll have to do a proper refactor eventually. I think this resolves the issue here for now, though, so thanks for the PR!

@Rocketknight1 Rocketknight1 merged commit ef42cb6 into huggingface:main Oct 17, 2023
21 checks passed
EduardoPach pushed a commit to EduardoPach/transformers that referenced this pull request Nov 19, 2023
Add tf-nightly-rocm to _is_tf_available check
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants