Take on Agentic Workflow with instructor

[97k]

Hi @jxnl,

I recently discovered Instructor through your co-authored blog post, "What we learned in a year of building LLMs," and I've been thrilled to replace my LCEL with Instructor code. The experience has been seamless!
However, I'm now facing a dilemma: how to set up agentic-workflow with Instructor.

Previously, I used LangGraph+LangChain's runnables with SQLite persistence checkpointer. I'd love to hear your thoughts on setting up the desired logic and flow for reliable agentic workflows.
I like LangGraph's concept, particularly how ToolMessage could interact with LLMs, enabling them to reason. In one of my projects, I successfully used LLMs with tools and HITL, but I encountered JsonDecodeErrors and had to craft detailed prompts to make it work. Now, I'm using a smaller LLM with n-shot learning, and it's working as expected with Instructor.

I have two questions:

• Using tools with Instructor: I couldn't find documentation on using tools with Instructor. Is it not currently supported, or am I missing something?
• Adding persistence: What would you suggest for adding persistence to Instructor? Should I stick with SQLite, or are there other options?

Looking forward to your insights!

[jxnl]

Sqlmodel from the creator of fastapi makes it easy to persist.

If you look at parallel tools. We have tools. Response model actually uses tools behind the scenes.

[97k]

I didn't look into Parallel Tools as it is only supported by gpt-4. I am playing with llama3 on groq. 🤔

[Mr-Ruben]

I didn't look into Parallel Tools as it is only supported by gpt-4. I am playing with llama3 on groq. 🤔

If you want tools (gpt-4) with a model that only supports 1-call-to-tool-each-round (gpt-3.5), I guess (i.e. haven't tested) you could create a response_model that allows inside it to specify multiple calls.

Like an Object of optional objects, being each one effectively a function_call.

Update: Example on #666 (comment)

[97k]

Yup, That's exactly what I've been doing now
This came handy
https://python.useinstructor.com/concepts/types/?h=union#union

[Mr-Ruben]

Another one
https://github.com/jxnl/instructor/blob/main/examples/union/run.py

[elijahbenizzy]

Hi @jxnl,

I recently discovered Instructor through your co-authored blog post, "What we learned in a year of building LLMs," and I've been thrilled to replace my LCEL with Instructor code. The experience has been seamless! However, I'm now facing a dilemma: how to set up agentic-workflow with Instructor.

Previously, I used LangGraph+LangChain's runnables with SQLite persistence checkpointer. I'd love to hear your thoughts on setting up the desired logic and flow for reliable agentic workflows. I like LangGraph's concept, particularly how ToolMessage could interact with LLMs, enabling them to reason. In one of my projects, I successfully used LLMs with tools and HITL, but I encountered JsonDecodeErrors and had to craft detailed prompts to make it work. Now, I'm using a smaller LLM with n-shot learning, and it's working as expected with Instructor.

I have two questions:

• Using tools with Instructor: I couldn't find documentation on using tools with Instructor. Is it not currently supported, or am I missing something?
• Adding persistence: What would you suggest for adding persistence to Instructor? Should I stick with SQLite, or are there other options?

Looking forward to your insights!

Hey! Wanted to share our library Burr. https://github.com/dagworks-inc/burr.

Similar at a high level to LangGraph (represent your app as a state machine, graph of actions), but has an OS UI for telemetry + persistence built in/pluggable (has a SQLLite persister as well). Also generally less opinionated about LLM tooling than the langstack -- works with instructor quite nicely (the examples just use OpenAI/other LLM endpoints, but instructor obviously allows a very clean patch).

You can read about persistence here: https://burr.dagworks.io/concepts/state-persistence/.

[97k]

Thank you for sharing.
I watched the short demo and it is amazing!

[elijahbenizzy]

You got it! Feel free to reach out (open issues/join the discord and ping us) if you have any questions :)

[Mr-Ruben]

I recently discovered Instructor through your co-authored blog post, "What we learned in a year of building LLMs," and I've been thrilled to replace my LCEL with Instructor code. The experience has been seamless! However, I'm now facing a dilemma: how to set up agentic-workflow with Instructor.

Your reference and what I found there has got me reading for hours ... It is a huge compilation of experience/knowledge.

Applied LLMs - What We’ve Learned From A Year of Building with LLMs
https://applied-llms.org/

Prompting Fundamentals and How to Apply them Effectively
https://eugeneyan.com/writing/prompting/

Your AI Product Needs Evals – Hamel's Blog
https://hamel.dev/blog/posts/evals/#automated-evaluation-w-llms

LLM From the Trenches: 10 Lessons Learned Operationalizing Models at GoDaddy - GoDaddy Blog
https://www.godaddy.com/resources/news/llm-from-the-trenches-10-lessons-learned-operationalizing-models-at-godaddy

[97k]

Yup, all good finds!

[97k]

Hey @jxnl

I have been able to set up agentic flow with instructor thanks to your cookbooks. They cover so many cases and I love it!

with instructor, I am getting structured output, but I also wanna store every conversation as a message, be it Human, Assistant or Tool.

I've written a helper function, which converts pydantic base class (Messages) to groq/open-ai compatible list of messages.

The problem occurs in case of Tool Message. The Groq / OpenAI endpoint, for tools needs tool_id and other related stuff for the message to be used for providing context to LLM.

Here's a groq reference

What should be done in this case? Should I just save ToolMessage as Assistant Message?

---

Here's what I've been cooking, for complex tasks I want to store every conversation that has happened as it will help me in designing stable tasks.
I am working on building a concierge who can handle multiple things and also with help of tools can interact with APIs (I don't want LLM to think on calling which API, rather I want llm to generate structure output and based on it the execute function will take care of getting data for RAG.

🤔 @jxnl

[97k]

For any one who is crazy like me, I solved by adding tool_call_id by having a default "..." and it works well.

[jd-solanki]

It will be great if someone can paste minimal example of multi agent example using instructor or add directly in docs

I still can't understand how to build multi agent system using instructor.

[97k]

I’ve a multi agent setup with instructor and burr.
It’s on my TODO for a while, I will write one soon and share it with you.

FWIW, multi agent setup is more of a state machine, where each node is an action that a agent can do.

The agents communicate via a state dictionary which saves the information.