TypeError: Type is not JSON serializable: numpy.float64

[mieslep]

I've narrowed down the problem to one that reproduces on the weblangchain repo https://github.com/langchain-ai/weblangchain

I'm trying to update to the latest versions of Langchain/Langsmith/LangchainJS/etc (including to the new "community" approach). I have reproduced on this (forked) branch https://github.com/mieslep/weblangchain/tree/compoent_and_update
Note that the prior commit runs correctly - so the most recent commit (the one that updates the LangChain versions) is the one that breaks.

The chain seems to run correctly https://smith.langchain.com/public/296c9325-2dce-4f2c-a533-64baac6c50fc/r

But it seems that somehow in the handling of the response back to the client something is getting mangled? (Not sure why the embedding vector might be returned back to the client...)

INFO:     127.0.0.1:62681 - "POST /chat/stream_log HTTP/1.1" 200 OK
ERROR:    Exception in ASGI application
RuntimeError: super(): __class__ cell not found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\fastapi\applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\middleware\cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\middleware\cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in __call__
    raise e
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\starlette\routing.py", line 69, in app
    await response(scope, receive, send)
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\sse_starlette\sse.py", line 255, in __call__
    async with anyio.create_task_group() as task_group:
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 597, in __aexit__
    raise exceptions[0]
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\sse_starlette\sse.py", line 258, in wrap
    await func()
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\sse_starlette\sse.py", line 245, in stream_response
    async for data in self.body_iterator:
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\langserve\api_handler.py", line 1106, in _stream_log
    "data": self._serializer.dumps(data).decode("utf-8"),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\phil\git\weblangchain\.venv\Lib\site-packages\langserve\serialization.py", line 168, in dumps
    return orjson.dumps(obj, default=default)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Type is not JSON serializable: numpy.float64

[mieslep]

fwiw, it fails in /chat/playground/ as well.

[eyurtsev]

numpy.float64 is not json serializable -- you need to convert it into a json format so that it can be transmitted over the wire to the front end.

[mieslep]

Yes sure - but this is Langchain/Langserve code, using a chain that worked on the previous version. So something there has changed... working on a debugger dump at the point of failure here. The tl;dr is that at

 "path": "/logs/FinalSourceRetriever/final_output"

the documents are carrying a state

state={'embedded_doc':[0.037251635410524446, 0.012951241830881097, 0.023566612850839656, ...

which I don't think they did previously.

[mieslep]

https://github.com/langchain-ai/langserve/blame/002902dfdd7df18bd88e288d1b1f372162730a25/langserve/serialization.py#L168

On the working version 0.0.19 the code is:

    def dumps(self, obj: Any) -> str:
        """Dump the given object as a JSON string."""
        return json.dumps(obj, cls=_LangChainEncoder)

And it is now 0.0.51 (and for 4+ months):

    def dumps(self, obj: Any) -> bytes:
        """Dump the given object as a JSON string."""
        return orjson.dumps(obj, default=default)

The old code dumped this slightly diffrently:

"state": {"embedded_doc": [0.004693315536778471, 0.009133123196791595, 

(note state versus "state").

The object is broadly similar, the issue looks to be that switching from the straight json.dumps to dumping via orjson has caused a problem? The attached file is output of

print(json.dumps(obj, cls=_LangChainEncoder)) (on v0.0.19 breakpoint), and
print(orjson.dumps(obj, default=default)) (on v0.0.51 breakpoint).

discussion-552.zip

[mieslep]

any thoughts/progress on this please? happy to help/test but need more direction :)

[eyurtsev]

Hi will take a look tomorrow

In the meantime it should be enough to append a runnable lambda to the chain and serialize the numpy output to json manually

support for numpy serialization in the past was likely accidental based on the presence of to_dict() method (you should be able to use that method to take over the serialization) -- I'll double check tomorrow when I look at the code more carefully

[mieslep]

The vector is being returned (for whatever reason), and that vector is seemingly represented/interpreted as a numpy array rather than a list of floats... I'll have a go at this workaround on my fork of weblangchain...

[mieslep]

So I'm baffled by this one - appreciating that you look to be quite busy on the 0.1.0 release :/

If I understood how to debug this stuff I might be able to reverse into how to do this, but this logic is getting handled at a deeper level than the chains, and it's not really a Numpy class anyhow, but something in the code is deciding that this big list of numbers is a Numpy array...

[designzoo-adrian]

Im new to using langserve and langchain, so excuse my lack of knowledge and best practice. I was also using EmbeddingFilter on my retrieved documents, led me to the same numpy float error. I am assuming the output on langserve gets serialised for the api result etc in JSON. As the 'context' object in my chain being passed though is a list of _DocumentWithState objects (used for the embeddingsfilter logic), it causes this issue.

To resolve I followed @eyurtsev advice and added a final RunnableLambda to the end of my 'served' chain, basically just formatted the _DocuentWithState objects to Document objects and overwrote the ["context"].

def prepare_output(output: dict):
try:
# Convert each document with state object to a standard document object
output['context'] = [doc.to_document() for doc in output['context']]
return output
except Exception as e:
LOGGER.error("An error occurred preparing output: %s", str(e))

Then added to my chain as:
| RunnableLambda(prepare_output)