Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[InferenceClient / langchain-HuggingFaceEndpoint] impossible to used Langchain HuggingFaceEndpoint whith trust_env=True #2522

Closed
morgandiverrez opened this issue Sep 6, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@morgandiverrez
Copy link
Contributor

Describe the bug

HuggingFaceEndpoint in the partners libs HuggingFace in the Langchain librairy cannot be used withtrust_env = True/False.

When we used the server_kwargs for set trust_env in client, this libs create AsyncInferenceClient and InferenceClient everytime, but InferenceClient everytime, but Client and AsyncInferenceClient doesn't have same parameter (trust_env in async and not in InferenceClient everytime, but InferenceClient everytime, but Client for exemple) .
if we used server_kwargs, we need to have same parameter in InferenceClient everytime, but Client doesn't have trust_env parameter.

I can fixed this in a PR by add trust_env parameter

Reproduction

from langchain_huggingface .llms import HuggingFaceEndpoint

HuggingFaceEndpoint(
endpoint_url="http://localhost:8010/",
max_new_tokens=512,
temperature=0.01,
repetition_penalty=1.03,
huggingfacehub_api_token="my-api-key",
server_kwargs={"trust_env"=True}
}

Logs

File "/home/Morgan.Diverrez/code/GitHub/tock-arkea/gen-ai/orchestrator-server/src/main/python/server/src/gen_ai_orchestrator/services/langchain/factories/llm/hugging_face_tgi_llm_factory.py", line 40, in get_language_model
    return HuggingFaceEndpoint(
  File "/home/Morgan.Diverrez/code/GitHub/tock-arkea/gen-ai/orchestrator-server/src/main/python/server/.venv/lib/python3.9/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for HuggingFaceEndpoint
__root__
  __init__() got an unexpected keyword argument 'trust_env' (type=type_error)

System info

- huggingface_hub version: 0.25.0.dev0
- Platform: Linux-5.15.0-117-generic-x86_64-with-glibc2.31
- Python version: 3.9.18
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/Morgan.Diverrez/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: morganDiverrez
- Configured git credential helpers: store
- FastAI: N/A
- Tensorflow: N/A
- Torch: N/A
- Jinja2: 3.1.4
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: N/A
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.8.2
- aiohttp: 3.10.5
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /home/Morgan.Diverrez/.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/Morgan.Diverrez/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/Morgan.Diverrez/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10
@morgandiverrez
Copy link
Contributor Author

hello, @Wauplin what do you think about this PR ?

Wauplin added a commit to Wauplin/langchain that referenced this issue Sep 12, 2024
By default, `HuggingFaceEndpoint` instantiates both the `InferenceClient` and the `AsyncInferenceClient` with the `"server_kwargs"` passed as input. This is an issue as both clients might not support exactly the same kwargs.  This has been highlighted in huggingface/huggingface_hub#2522 by @morgandiverrez with the `trust_env` parameter. In order to make `langchain` integration future-proof, I do think it's wiser to forward only the supported parameters to each client. Parameters that are not supported are simply ignored with a warning to the user. From a `huggingface_hub` maintenance perspective, this allows us much more flexibility as we are not constrained to support the exact same kwargs in both clients.
efriis added a commit to langchain-ai/langchain that referenced this issue Sep 20, 2024
…h supported kwargs) (#26378)

## Description

By default, `HuggingFaceEndpoint` instantiates both the
`InferenceClient` and the `AsyncInferenceClient` with the
`"server_kwargs"` passed as input. This is an issue as both clients
might not support exactly the same kwargs. This has been highlighted in
huggingface/huggingface_hub#2522 by
@morgandiverrez with the `trust_env` parameter. In order to make
`langchain` integration future-proof, I do think it's wiser to forward
only the supported parameters to each client. Parameters that are not
supported are simply ignored with a warning to the user. From a
`huggingface_hub` maintenance perspective, this allows us much more
flexibility as we are not constrained to support the exact same kwargs
in both clients.

## Issue

huggingface/huggingface_hub#2522

## Dependencies

None

## Twitter 

https://x.com/Wauplin

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
sfc-gh-nmoiseyev pushed a commit to sfc-gh-nmoiseyev/langchain that referenced this issue Sep 21, 2024
…h supported kwargs) (langchain-ai#26378)

## Description

By default, `HuggingFaceEndpoint` instantiates both the
`InferenceClient` and the `AsyncInferenceClient` with the
`"server_kwargs"` passed as input. This is an issue as both clients
might not support exactly the same kwargs. This has been highlighted in
huggingface/huggingface_hub#2522 by
@morgandiverrez with the `trust_env` parameter. In order to make
`langchain` integration future-proof, I do think it's wiser to forward
only the supported parameters to each client. Parameters that are not
supported are simply ignored with a warning to the user. From a
`huggingface_hub` maintenance perspective, this allows us much more
flexibility as we are not constrained to support the exact same kwargs
in both clients.

## Issue

huggingface/huggingface_hub#2522

## Dependencies

None

## Twitter 

https://x.com/Wauplin

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
@Wauplin
Copy link
Contributor

Wauplin commented Sep 23, 2024

As explained in #2525, I'll close this issue without merging in favor of langchain-ai/langchain#26378 that just got merged. Thanks again for reporting @morgandiverrez 🤗

@Wauplin Wauplin closed this as not planned Won't fix, can't repro, duplicate, stale Sep 23, 2024
Sheepsta300 pushed a commit to Sheepsta300/langchain that referenced this issue Oct 1, 2024
…h supported kwargs) (langchain-ai#26378)

## Description

By default, `HuggingFaceEndpoint` instantiates both the
`InferenceClient` and the `AsyncInferenceClient` with the
`"server_kwargs"` passed as input. This is an issue as both clients
might not support exactly the same kwargs. This has been highlighted in
huggingface/huggingface_hub#2522 by
@morgandiverrez with the `trust_env` parameter. In order to make
`langchain` integration future-proof, I do think it's wiser to forward
only the supported parameters to each client. Parameters that are not
supported are simply ignored with a warning to the user. From a
`huggingface_hub` maintenance perspective, this allows us much more
flexibility as we are not constrained to support the exact same kwargs
in both clients.

## Issue

huggingface/huggingface_hub#2522

## Dependencies

None

## Twitter 

https://x.com/Wauplin

---------

Co-authored-by: Erick Friis <erick@langchain.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants