Comparing LangChain, Fixie, and Marvin

Task:

1. Load the "State of the Union" file locally (or remotely) and store them into a vector db (Chroma)
2. Ask questions for the document

Analysis

LangChain

Pros:

• Most powerful option among the three options
• You can create your own tool.
• Supports many third-party library integrations, different language models, tracing, and much more.

Cons:

• Not much complains, but their example notebooks are sometimes broken.

Fixie

Pros:

• You can quickly generate and deploy your custom agent
• You can add your own tool (e.g., @agent.register_func)

Cons:

• It seems like you cannot choose how the text should be splitted (e.g., text-splitter in LangChain)
• More examples will be great (i.e., Chat with PDF)
• Ability to easily add third-party libraries is missing

Misc:

• Here is a link to my agent.
• There are no tools for LLM output evaluation at the time of writing this (Apr 23, 2023). However you could add your own validation/evaluation function by integrating a third-party library like Guardrails or Evals.

Marvin

Pros:

• AI functions are super cool. (I didn't touch on it in this repo)
• You can use the LangChain document loader.
• Interactive option that runs on a terminal is useful.
• You can add your custom "plugin"s.

Cons:

• More examples will be great (e.g., Chat with PDF)
• Ability to easily add third-party libraries is missing