[Community] Milvus

[zrg-team]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

await vectorStore.addDocuments([
  new Document({
     pageContent: 'test',
     metadata: { test: '' }
   })
])

Error Message and Stack Trace (if applicable)

[ProjectSourceCreation] Error: Error upserting data: {"succ_index":[],"err_index":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54],"status":{"error_code":"IllegalArgument","reason":"upsert can not assign primary field data when auto id enabled langchain_primaryid: invalid parameter","code":1100,"retriable":false,"detail":"upsert can not assign primary field data when auto id enabled langchain_primaryid: invalid parameter"},"IDs":null,"acknowledged":false,"insert_cnt":"0","delete_cnt":"0","upsert_cnt":"0","timestamp":"0"} at Milvus.addVectors (/Users/zergling/Documents/projects/ai/jitera_ai/node_modules/@langchain/community/dist/vectorstores/milvus.cjs:247:19) at processTicksAndRejections (node:internal/process/task_queues:95:5) at Milvus.addDocuments (/Users/zergling/Documents/projects/ai/jitera_ai/node_modules/@langchain/community/dist/vectorstores/milvus.cjs:177:9) at MilvusEmbedding.indexDocuments (/Users/zergling/Documents/projects/ai/jitera_ai/packages/ai-core/src/embeddings/milvus/index.ts:338:7)

Description

• Index new document issues (even i do not assign langchain_primaryid)
  "upsert can not assign primary field data when auto id enabled langchain_primaryid: invalid parameter"

System Info

"dependencies": {
    "@langchain/community": "0.2.12",
    "@langchain/langgraph": "0.0.23",
    "@langchain/openai": "0.1.3",
    "@zilliz/milvus2-sdk-node": "2.4.3",
    "diff": "5.2.0",
    "langsmith": "0.1.32",
    "parse-git-patch": "^1.2.2",
    "strip-json-trailing-commas": "^1.1.0"
  },
  "resolutions": {
    "@langchain/core": "~0.2.6",
    "langchain": "0.2.3"
  },

Milvus 2.4.4

[dosubot]

The error you're encountering is due to the autoId property being enabled while trying to assign a primary field value. By default, autoId is set to true in the Milvus vector store integration within LangChain.js ^[1].

To resolve this issue, you have two options:

1. Disable autoId: If you want to manually assign primary field values, you need to set autoId to false. Ensure that the primary field value is provided in the metadata of each document.

   Example:

   const vectorStore = new Milvus({
     autoId: false,
     // other configurations...
   });
   
   await vectorStore.addDocuments([
     new Document({
       pageContent: 'test',
       metadata: { langchain_primaryid: 'unique_id_1' }
     })
   ]);

2. Remove Primary Field from Metadata: If you prefer to keep autoId enabled, ensure that you do not assign a primary field value in the metadata.

   Example:

   await vectorStore.addDocuments([
     new Document({
       pageContent: 'test',
       metadata: { test: '' }
     })
   ]);

Choose the approach that best fits your use case.

To continue talking to Dosu, mention @dosu.

[zrg-team]

It seen issue due to "upsert"
https://github.com/langchain-ai/langchainjs/pull/5760/files