huggingface; fix huggingface_endpoint.py (initialize clients only with supported kwargs) #26378
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
By default, `HuggingFaceEndpoint` instantiates both the `InferenceClient` and the `AsyncInferenceClient` with the `"server_kwargs"` passed as input. This is an issue as both clients might not support exactly the same kwargs. This has been highlighted in huggingface/huggingface_hub#2522 by @morgandiverrez with the `trust_env` parameter. In order to make `langchain` integration future-proof, I do think it's wiser to forward only the supported parameters to each client. Parameters that are not supported are simply ignored with a warning to the user. From a `huggingface_hub` maintenance perspective, this allows us much more flexibility as we are not constrained to support the exact same kwargs in both clients.
dosubot
bot
added
the
size:S
Wauplin
commented
Sep 12, 2024
libs/partners/huggingface/langchain_huggingface/llms/huggingface_endpoint.py
Outdated
Show resolved
Hide resolved
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
dosubot
bot
added
langchain
morgandiverrez
approved these changes
Sep 12, 2024
Benvii
approved these changes
Sep 12, 2024
|
Hey @Wauplin ! Just tagging in @Jofthomas from your team who implemented the package - excited to have more folks from HF helping out with the integration package! Overall lgtm, and just want to make sure this makes sense to him too |
Jofthomas
approved these changes
Sep 18, 2024
sfc-gh-nmoiseyev
pushed a commit
to sfc-gh-nmoiseyev/langchain
that referenced
this pull request
Sep 21, 2024
…h supported kwargs) (langchain-ai#26378) ## Description By default, `HuggingFaceEndpoint` instantiates both the `InferenceClient` and the `AsyncInferenceClient` with the `"server_kwargs"` passed as input. This is an issue as both clients might not support exactly the same kwargs. This has been highlighted in huggingface/huggingface_hub#2522 by @morgandiverrez with the `trust_env` parameter. In order to make `langchain` integration future-proof, I do think it's wiser to forward only the supported parameters to each client. Parameters that are not supported are simply ignored with a warning to the user. From a `huggingface_hub` maintenance perspective, this allows us much more flexibility as we are not constrained to support the exact same kwargs in both clients. ## Issue huggingface/huggingface_hub#2522 ## Dependencies None ## Twitter https://x.com/Wauplin --------- Co-authored-by: Erick Friis <erick@langchain.dev>
Sheepsta300
pushed a commit
to Sheepsta300/langchain
that referenced
this pull request
Oct 1, 2024
…h supported kwargs) (langchain-ai#26378) ## Description By default, `HuggingFaceEndpoint` instantiates both the `InferenceClient` and the `AsyncInferenceClient` with the `"server_kwargs"` passed as input. This is an issue as both clients might not support exactly the same kwargs. This has been highlighted in huggingface/huggingface_hub#2522 by @morgandiverrez with the `trust_env` parameter. In order to make `langchain` integration future-proof, I do think it's wiser to forward only the supported parameters to each client. Parameters that are not supported are simply ignored with a warning to the user. From a `huggingface_hub` maintenance perspective, this allows us much more flexibility as we are not constrained to support the exact same kwargs in both clients. ## Issue huggingface/huggingface_hub#2522 ## Dependencies None ## Twitter https://x.com/Wauplin --------- Co-authored-by: Erick Friis <erick@langchain.dev>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
By default,
HuggingFaceEndpointinstantiates both theInferenceClientand theAsyncInferenceClientwith the"server_kwargs"passed as input. This is an issue as both clients might not support exactly the same kwargs. This has been highlighted in huggingface/huggingface_hub#2522 by @morgandiverrez with thetrust_envparameter. In order to makelangchainintegration future-proof, I do think it's wiser to forward only the supported parameters to each client. Parameters that are not supported are simply ignored with a warning to the user. From ahuggingface_hubmaintenance perspective, this allows us much more flexibility as we are not constrained to support the exact same kwargs in both clients.Issue
huggingface/huggingface_hub#2522
Dependencies
None
Twitter
https://x.com/Wauplin