TransformChain wrong function types, enforced since 0.0.267

[mpartel]

System Info

LangChain 0.0.267, Python 3.10, Poetry virtualenv, Pop_OS 22.04

Who can help?

@agola11

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

from langchain.chains import TransformChain

def transform(inputs):
    return {
        'output': {
            'structured': {
                'report': 'Done'
            }
        }
    }
async def atransform(inputs): return transform(inputs)
chain = TransformChain(
    input_variables=['text'],
    output_variables=['output'],
    transform=transform,  # <-- type error below happens here
    atransform=atransform,  # <-- If I remove this, I get: Argument missing for parameter "atransform"
)

Argument of type "(inputs: Unknown) -> dict[str, dict[str, dict[str, str]]]" cannot be assigned to parameter "transform" of type "(Dict[str, str]) -> Dict[str, str]" in function "__init__"
  Type "(inputs: Unknown) -> dict[str, dict[str, dict[str, str]]]" cannot be assigned to type "(Dict[str, str]) -> Dict[str, str]"
    Function return type "dict[str, dict[str, dict[str, str]]]" is incompatible with type "Dict[str, str]"
      "dict[str, dict[str, dict[str, str]]]" is incompatible with "Dict[str, str]"
        Type parameter "_VT@dict" is invariant, but "dict[str, dict[str, str]]" is not the same as "str"

Expected behavior

Two type checking issues since 0.0.267:

• transform should probably take/return dict[str, Any] instead of dict[str, str] (like atransform does)
• atransfrom should probably not be mandatory

Most other code seems to use Dict[str, Any], and non-string values seem to work just fine.

Prior to 0.0.267 the type system didn't enforce the type of transform when constructing TransformChain. It also started requiring an atransform parameter even though it seems to be intended to be optional.

[baskaryan]

@mpartel do you know what version of pydantic this was happening with?

[mpartel]

2.2.1

Just tested with a fresh project:

$ poetry init
...
$ poetry add langchain
...
Using version ^0.0.268 for langchain

Updating dependencies
Resolving dependencies... (1.2s)

Package operations: 30 installs, 0 updates, 0 removals

  • Installing packaging (23.1)
  • Installing typing-extensions (4.7.1)
  • Installing annotated-types (0.5.0)
  • Installing certifi (2023.7.22)
  • Installing charset-normalizer (3.2.0)
  • Installing frozenlist (1.4.0)
  • Installing idna (3.4)
  • Installing marshmallow (3.20.1)
  • Installing multidict (6.0.4)
  • Installing mypy-extensions (1.0.0)
  • Installing pydantic-core (2.6.1)
  • Installing urllib3 (2.0.4)
  • Installing aiosignal (1.3.1)
  • Installing async-timeout (4.0.3)
  • Installing attrs (23.1.0)
  • Installing greenlet (2.0.2)
  • Installing marshmallow-enum (1.5.1)
  • Installing numpy (1.25.2)
  • Installing pydantic (2.2.1)   # <-----
  • Installing requests (2.31.0)
  • Installing typing-inspect (0.9.0)
  • Installing yarl (1.9.2)
  • Installing aiohttp (3.8.5)
  • Installing dataclasses-json (0.5.9)
  • Installing langsmith (0.0.24)
  • Installing numexpr (2.8.5)
  • Installing pyyaml (6.0.1)
  • Installing sqlalchemy (2.0.20)
  • Installing tenacity (8.2.3)
  • Installing langchain (0.0.268)   # <-----

[baskaryan]

for debugging purposes, could you try installing pydantic v1 (e.g. poetry run pip install "pydantic<2")? guessing it has something to do with that

[mpartel]

Pydantic < 2 had no effect:

$ poetry add 'pydantic<2'

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 0 installs, 1 update, 2 removals

  • Removing annotated-types (0.5.0)
  • Removing pydantic-core (2.6.1)
  • Updating pydantic (2.2.1 -> 1.10.12)

Both errors stayed the same. (Sanity-checked: even with Pydantic 1, poetry add 'langchain==0.0.266' made the errors go away, and poetry add 'langchain==0.0.268' made them come back.)

[dosubot]

Hi, @mpartel! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you reported is about two type checking problems in the TransformChain library. The transform function should be modified to take and return dict[str, Any] instead of dict[str, str], and the atransform function should not be mandatory. In the comments, baskaryan asked about the version of pydantic causing the issue, and you responded that it is version 2.2.1. You also mentioned that installing pydantic version 1 did not resolve the issue.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain repository!