Feat/llm responses #376

NotBioWaste905 · 2024-07-24T12:22:22Z

Description

Added functionality for calling LLMs via langchain API for utilizing them in responses and conditions.

Checklist

I have performed a self-review of the changes

List here tasks to complete in order to mark this PR as ready for review.

To Consider

Add tests
Update API reference / tutorials / guides
Update CONTRIBUTING.md

github-actions

It appears this PR is a release PR (change its base from master if that is not the case).

Here's a release checklist:

… type annotations

…alog_flow_framework into feat/llm_responses

chatsky/llm/methods.py

chatsky/llm/filters.py

chatsky/llm/wrapper.py

tests/llm/test_model_response.py

RLKRo · 2024-08-08T13:00:05Z

I got an idea for more complex prompts: we can allow passing responses as prompts instead of just strings.

And then it'd be possible to incorporate slots into a prompt:

model = LLM_API(prompt=rsp.slots.FilledTemplate("You are an experienced barista in a local coffeshop."
"Answer your customers questions about coffee and barista work.\n"
"Customer data:\nAge {person.age}\nGender: {person.gender}\nFavorite drink: {person.habits.drink}"
))

RLKRo

I've marked all resolved conversations as resolved (don't forget to put correct commit hash!).
There are still 25 unresolved conversations.
I've edited 9 of them with PROMPT REWORK or POSTPONED prefixes: the first ones are for me to resolve, the latter -- to be resolved in a later PR.

Please respond to the other 16 comments (plus the ones from this review): either with a commit hash of a commit that resolves it or with your comments regarding the suggestion.

RLKRo · 2024-11-28T23:14:23Z

chatsky/llm/filters.py

+        raise NotImplemented
+
+    def __call__(self, ctx, request, response, model_name):
+        return self.call(ctx, request, model_name) + self.call(ctx, response, model_name)


Needs to be bitwise or:

Suggested change

return self.call(ctx, request, model_name) + self.call(ctx, response, model_name)

return self.call(ctx, request, model_name) | self.call(ctx, response, model_name)

Add tests. They did not catch this.

RLKRo · 2024-11-28T23:15:09Z

chatsky/llm/filters.py

+        if request is not None and request.misc is not None and request.misc.get("important", None):
+            return self.Return.Request
+        if response is not None and response.misc is not None and response.misc.get("important", None):
+            return self.Return.Response


If both contain "important" this will return Request instead of Turn.
Implement this as MessageFilter.

Same for FromModel.

RLKRo · 2024-11-28T23:28:38Z

chatsky/slots/llm.py

+            }
+        return ExtractedGroupSlot(**res)
+
+    def __flatten_llm_group_slot(self, slot, parent_key=""):


You missed at least one in

if isinstance(value, LLMGroupSlot): items.update(self.__flatten_llm_group_slot(value, new_key))

Add tests that use nested LLMGroupSlots.

RLKRo · 2024-11-28T23:44:45Z

chatsky/llm/llm_api.py

+            raise ValueError
+
+    async def condition(
+        self, ctx: Context, prompt: str, method: BaseMethod, return_schema: Optional[BaseModel] = None


Why does condition not support context history?

RLKRo · 2024-11-28T23:45:38Z

chatsky/llm/llm_api.py

+            result.annotations = {"__generated_by_model__": self.name}
+        return result
+
+    async def condition(self, prompt: str, method: BaseMethod, return_schema=None):


Is it not possible to use message schema with log probs?

chatsky/llm/utils.py

RLKRo · 2024-11-28T23:50:26Z

tests/llm/test_llm.py

Some lines are clearly not covered by the tests:
Slots are not tested at all.

Run poe quick_test_coverage to generate html reports in htmlcov directory.
You can then view them (by opening htmlcov/index.html) to see which lines are not covered.

RLKRo · 2024-11-28T23:51:32Z

tutorials/llm/1_basics.py

+it will be reused across all the nodes and therefore it will store all dialogue history.
+This is not advised if you are short on tokens or if you do not need to store all dialogue history.
+Alternatively you can instantiate model object inside of RESPONSE field in the nodes you need.
+Via `history` parameter you can set number of dialogue _turns_ that the model will use as the history. Default value is `5`.


This is out of place.
This should be in filtering_history or as a comment near the line where LLMResponse is initialized with history=0.

RLKRo · 2024-11-28T23:53:10Z

tutorials/llm/5_llm_slots.py

+        },
+        "tell": {
+            RESPONSE: rsp.FilledTemplate(
+                "So you are {person.username} and your occupation is {person.job}, right?"


person group slot does not allow partial extraction.
Add the flag, mention it, link to partial extraction tutorial.

…ited from MessageFilter

…ter initialization

…gue pairs and reduce code redundancy

Ramimashkouk · 2024-12-19T09:37:28Z