Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Add Google Cloud TGI integration via dedicated DLCs #2612

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alvarobartt
Copy link
Member

What does this PR do?

This PR adds Google Cloud within the TGI Documentation https://huggingface.co/docs/text-generation-inference/main/en/reference/api_reference#cloud-providers side-by-side with the Amazon SageMaker integration.

Warning

Before merging I wanted to first clarify whether it would make sense to have a separate section within the TGI documentation for the Cloud Providers; as not sure that the Google Cloud integration fits within the HTTP API Reference page? Maybe you can help clarify a bit better @nbroad1881

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@nbroad1881 @pagezyhf @philschmid

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@pagezyhf
Copy link

pagezyhf commented Oct 7, 2024

My 2 cents:

In Reference > API Reference > HTTP API Reference > OpenAI Messages API > Cloud Provider, I would just point to https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library#supported-model-garden-models

What you wrote fits a generic introduction about using TGI on GCP. I think it's valuable to add it to the TGI doc but this is not the right place. It could be in Tutorials or Getting Started with simple pointers to the existing doc in https://huggingface.co/docs/sagemaker/en/index for AWS and https://huggingface.co/docs/google-cloud/en/index for Google Cloud.

@alvarobartt
Copy link
Member Author

In Reference > API Reference > HTTP API Reference > OpenAI Messages API > Cloud Provider, I would just point to https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library#supported-model-garden-models

Agree @pagezyhf, also the AWS SageMaker section there is also covering deployment + inference, which may be out of the scope of the section, right?

What you wrote fits a generic introduction about using TGI on GCP

Yes that's right, that's why I also think that we should maybe have a dedicated entry for those with brief introduction plus pointers to the actual documentation pages we host separately

Thanks for the replies @pagezyhf 🤗 WDYT about those @nbroad1881?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants