PineconeVectorStore support loading data from Python and reading from TS

[david1542]

Hey everyone.

I'm using Pinecone as a vector DB for my RAG application. I've loaded some documents into it with python code, and now I'm trying to query the DB with TypeScript. However, the nodes in the query result always contains an empty "text" field, which prevents me from using the texts in my prompt.

Here is how I query the DB:

const embedModel = new OpenAIEmbedding({
  apiKey: process.env.OPENAI_API_KEY as string,
  model: "text-embedding-3-large",
  dimensions: 768,
});
const embedding = await embedModel.getTextEmbedding("my query");
const pcvs = new PineconeVectorStore({
  indexName: "my-index",
});
const result = await pcvs.query({
  queryEmbedding: embedding,
  similarityTopK: 5,
  mode: VectorStoreQueryMode.DEFAULT,
});

console.log(result.nodes); // Contains 5 items with scores but empty text field

Here is how I loaded them into the DB:

pc = Pinecone(api_key=pinecone_api_key)
pc_index = pc.Index(name=organization_id)

vector_store = PineconeVectorStore(pinecone_index=pc_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
embed_model = OpenAIEmbedding(
    api_key=openai_api_key,
    model="text-embedding-3-large",
    dimensions=768,
)

VectorStoreIndex.from_documents(
    total_documents,
    vector_store=pc_index,
    show_progress=True,
    embed_model=embed_model,
    storage_context=storage_context,
)

Here is an image during a debugging session:

I think this is related to #552 but I'm not sure.

I'd would love to get some help on this :)

[marcusschiesser]

@david1542 we just tested this with the new 0.1.19 release. Please have a try and reopen if necessary