Implement max_marginal_relevance_search in VectorStore of Pinecone

[neo]

This adds implementation of MMR search in pinecone; and I have two semi-related observations about this vector store class:

• Maybe we should also have a similarity_search_by_vector_returning_embeddings like in supabase, but it's not in the base VectorStore class so I didn't implement
• Talking about the base class, there's similarity_search_with_relevance_scores, but in pinecone it is called similarity_search_with_score; maybe we should consider renaming it to align with other VectorStore base and sub classes (or add that as an alias for backward compatibility)

Who can review?

Tag maintainers/contributors who might be interested:

• VectorStores / Retrievers / Memory - @dev2049

[rlancemartin]

Thanks for the contribution! (1) Can you add one of two examples of these to the Pinecone notebook docs/modules/indexes/vectorstores/examples/pinecone.ipynb? (2) Can the functions be combined given the small difference between them (passing query vs embedding)? May help reduce code duplication.

[neo]

Thank you for the review!

1. yes I will add an example
2. I agree, I only did both in order to stay consistent with supabase (which I used as the example for the implementation); I'm thinking to keep the query one?
   • edit: looking at the code, I remembered the base VectorStore abstract class also has both methods, so not sure whether we should have both in the pinecone implementation...
   • edit again: in fact, I did a search and most of the vectorstores have both of them?

What do you think about my comments about similarity_search_by_vector_returning_embeddings and _similarity_search_with_relevance_scores? Shall we consider implementing these two?

[rlancemartin]

I did a search and most of the vectorstores have both of them

Right, in that case, yes let's keep both. So, you can just add to the notebook :)

similarity_search_by_vector_returning_embeddings

You are welcome to add it here or in a follow-up PR. Please add a bit more documentation than Supabase has and a notebook example.

Talking about the base class, there's similarity_search_with_relevance_scores, but in pinecone it is called similarity_search_with_score

This is a good call out. I will check with Harrison, but let's take it up in a separate PR.

[neo]

• examples added in notebook
• might follow up on and maybe discuss/think more on similarity_search_by_vector_returning_embeddings as it seems only supabase has it
• agreed that similarity_search_with_relevance_scores should be a different PR, just wanted to bring up as it's also in pinecone – maybe could do something like https://github.com/hwchase17/langchain/pull/6015/files to deprecate it first before removal

[rlancemartin]

• agreed that similarity_search_with_relevance_scores should be a different PR, just wanted to bring up as it's also in pinecone – maybe could do something like https://github.com/hwchase17/langchain/pull/6015/files to deprecate it first before removal

Yes, we should add a DeprecationWarning. Feel free to put up a PR w/ that change.

[rlancemartin]

Nice work!

[neo]

Yes, we should add a DeprecationWarning. Feel free to put up a PR w/ that change.

Will do. How about similarity_search_by_vector_returning_embeddings? It might fit into the same PR as they're just method name changes. Do you want to maybe discuss with the wider team to see if it's something we want to standardize across vector stores? Otherwise, we prob don't need to add that.

[rlancemartin]

Yes, we should add a DeprecationWarning. Feel free to put up a PR w/ that change.

Will do. How about similarity_search_by_vector_returning_embeddings? It might fit into the same PR as they're just method name changes. Do you want to maybe discuss with the wider team to see if it's something we want to standardize across vector stores? Otherwise, we prob don't need to add that.

Yes, let's take this up in a new PR since name changes have broader scope. Feel free to put up a PR and I can circulate it with folks. In the meantime, I'm going to merge this one :)