Currentlty works with a hack involving manual changes to langchain source code: https://github.com/langchain-ai/langchain/issues/11168.