Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AIC-py] hf machine translation (text to text) model parser #753

Merged
merged 2 commits into from
Jan 5, 2024
Merged

[AIC-py] hf machine translation (text to text) model parser #753

merged 2 commits into from
Jan 5, 2024

Conversation

jonathanlastmileai
Copy link
Contributor

@jonathanlastmileai jonathanlastmileai commented Jan 4, 2024 •
edited
Loading

[AIC-py] hf mt model parser

text translation sans streaming. Streaming is not trivial here

Test:

import asyncio
from aiconfig_extension_hugging_face.local_inference.text_translation import HuggingFaceTextTranslationTransformer

from aiconfig import AIConfigRuntime, InferenceOptions, CallbackManager

# Load the aiconfig.

mp = HuggingFaceTextTranslationTransformer()

AIConfigRuntime.register_model_parser(mp, "translation_en_to_fr")

config = AIConfigRuntime.load("/Users/jonathan/Projects/aiconfig/test_hf_transl.aiconfig.json")
config.callback_manager = CallbackManager([])


def print_stream(data, _accumulated_data, _index):
    print(data, end="", flush=True)


async def run():
    # print("Stream")
    # options = InferenceOptions(stream=True, stream_callback=print_stream)
    # out = await config.run("test_hf_trans", options=options)
    # print("Output:\n", out)

    print("no stream")
    options = InferenceOptions(stream=False)
    out = await config.run("test_hf_trans", options=options)
    print("Output:\n", out)


asyncio.run(run())

Stack created with Sapling. Best reviewed with ReviewStack.

@jonathanlastmileai
[RFC][AIC-py] hf summarization model parser
13e590f 
Very similar to text gen

Test:

```
...
    {
      "name": "test_hf_sum",
      "input": "HMS Duncan was a D-class destroyer ...", # [contents of https://en.wikipedia.org/wiki/HMS_Duncan_(D99)]
      "metadata": {
        "model": {
          "name": "stevhliu/my_awesome_billsum_model",
          "settings": {
            "min_length": 100,
            "max_length": 200,
            "num_beams": 1
          }
        }
      }
    },
...
}
```

```
import asyncio
from aiconfig_extension_hugging_face.local_inference.text_summarization import HuggingFaceTextSummarizationTransformer

from aiconfig import AIConfigRuntime, InferenceOptions, CallbackManager

# Load the aiconfig.

mp = HuggingFaceTextSummarizationTransformer()

AIConfigRuntime.register_model_parser(mp, "stevhliu/my_awesome_billsum_model")

config = AIConfigRuntime.load("/Users/jonathan/Projects/aiconfig/cookbooks/Getting-Started/travel.aiconfig.json")
config.callback_manager = CallbackManager([])


def print_stream(data, _accumulated_data, _index):
    print(data, end="", flush=True)


async def run():
    print("Stream")
    options = InferenceOptions(stream=True, stream_callback=print_stream)
    out = await config.run("test_hf_sum", options=options)
    print("Output:\n", out)

    print("no stream")
    options = InferenceOptions(stream=False)
    out = await config.run("test_hf_sum", options=options)
    print("Output:\n", out)


asyncio.run(run())


# OUT:

Stream
Token indices sequence length is longer than the specified maximum sequence length for this model (2778 > 512). Running this sequence through the model will result in indexing errors
/opt/homebrew/Caskroom/miniconda/base/envs/aiconfig/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:430: UserWarning: `num_beams` is set to 1. However, `early_stopping` is set to `True` -- this flag is only used in beam-based generation modes. You should set `num_beams>1` or unset `early_stopping`.
  warnings.warn(
/opt/homebrew/Caskroom/miniconda/base/envs/aiconfig/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:449: UserWarning: `num_beams` is set to 1. However, `length_penalty` is set to `2.0` -- this flag is only used in beam-based generation modes. You should set `num_beams>1` or unset `length_penalty`.
  warnings.warn(
<pad> escorted the 13th Destroyer Flotilla in the Mediterranean and escorited the carrier Argus to Malta during the war  The ship was recalled home to be converted into an escoter destroyer in late 1942. The vessel was repaired and given a refit at Gibraltar on 16 November, and was sold for scrap later that year. The crew of the ship escores the ship to the Middle East, and the ship is a 'disaster' of the </s>Output:
 [ExecuteResult(output_type='execute_result', execution_count=0, data="<pad> escorted the 13th Destroyer Flotilla in the Mediterranean and escorited the carrier Argus to Malta during the war  The ship was recalled home to be converted into an escoter destroyer in late 1942. The vessel was repaired and given a refit at Gibraltar on 16 November, and was sold for scrap later that year. The crew of the ship escores the ship to the Middle East, and the ship is a 'disaster' of the </s>", mime_type=None, metadata={})]
no stream
Output:
 [ExecuteResult(output_type='execute_result', execution_count=0, data="escorted the 13th Destroyer Flotilla in the Mediterranean and escorited the carrier Argus to Malta during the war . The ship was recalled home to be converted into an escoter destroyer in late 1942. The vessel was repaired and given a refit at Gibraltar on 16 November, and was sold for scrap later that year. The crew of the ship escores the ship to the Middle East, and the ship is a 'disaster' of the .", mime_type=None, metadata={})]

```
@jonathanlastmileai jonathanlastmileai mentioned this pull request Jan 4, 2024
Merged
@jonathanlastmileai
[AIC-py] hf mt model parser
9fd71be 
text translation sans streaming. Streaming is not trivial here

Test:

```
import asyncio
from aiconfig_extension_hugging_face.local_inference.text_translation import HuggingFaceTextTranslationTransformer

from aiconfig import AIConfigRuntime, InferenceOptions, CallbackManager

# Load the aiconfig.

mp = HuggingFaceTextTranslationTransformer()

AIConfigRuntime.register_model_parser(mp, "translation_en_to_fr")

config = AIConfigRuntime.load("/Users/jonathan/Projects/aiconfig/test_hf_transl.aiconfig.json")
config.callback_manager = CallbackManager([])


def print_stream(data, _accumulated_data, _index):
    print(data, end="", flush=True)


async def run():
    # print("Stream")
    # options = InferenceOptions(stream=True, stream_callback=print_stream)
    # out = await config.run("test_hf_trans", options=options)
    # print("Output:\n", out)

    print("no stream")
    options = InferenceOptions(stream=False)
    out = await config.run("test_hf_trans", options=options)
    print("Output:\n", out)


asyncio.run(run())

```
@saqadri
Copy link
Contributor

saqadri commented Jan 4, 2024

What does "mt" stand for?

@jonathanlastmileai
Copy link
Contributor Author

What does "mt" stand for?

machine translation :)

@jonathanlastmileai jonathanlastmileai changed the title [AIC-py] hf mt model parser [AIC-py] hf machine translation (text to text) model parser Jan 4, 2024
rossdanlm
rossdanlm reviewed Jan 5, 2024
View reviewed changes
extensions/HuggingFace/python/src/aiconfig_extension_hugging_face/__init__.py
@@ -2,9 +2,15 @@
from .local_inference.text_generation import HuggingFaceTextGenerationTransformer
from .remote_inference_client.text_generation import HuggingFaceTextGenerationClient
from .local_inference.text_summarization import HuggingFaceTextSummarizationTransformer
from .local_inference.text_translation import HuggingFaceTextTranslationTransformer

# from .remote_inference_client.text_generation import HuggingFaceTextGenerationClient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this was from lsat diff, but can we delete? Thanks

rossdanlm
rossdanlm approved these changes Jan 5, 2024
View reviewed changes
Copy link
Contributor

@rossdanlm rossdanlm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me, thanks!

@jonathanlastmileai jonathanlastmileai merged commit 00b7acd into main Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@saqadri saqadri saqadri left review comments

@rossdanlm rossdanlm rossdanlm approved these changes

@rholinshead rholinshead Awaiting requested review from rholinshead rholinshead is a code owner

@suyoglastmileai suyoglastmileai Awaiting requested review from suyoglastmileai

@Ankush-lastmile Ankush-lastmile Awaiting requested review from Ankush-lastmile Ankush-lastmile is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jonathanlastmileai @saqadri @rossdanlm