Allow issue vector search in batch mode

[eddyxu]

Problem Statement

Instead of query one vector a time, allows user to issue multiple queries in the same request, exposing more opportunity for internal parallisms.

enable vector searching for a batch of queries. Currently Lance only allows 1D queries, but allowing 2D queries (parallelized) would be useful.
Use case #1 is: at query time to run multiple simultaneous queries 'around' the root query to diversify search results, and #2 is useful for ETL operations (like the above deduplication)
Agree it would be amazing that lance support this api directly to minimize read requests to object store. Recent work to cache the index already helps

[trueutkarsh]

Hi @eddyxu,
I have a few doubts regarding this problem which I wanted to discuss with you. With multiple queries instead of one

• should we blindly spawn threads, create physical plans and execute them ? OR
• does just adding more nodes in physical plan and executing all partitions instead of one seem like a possible solution ?
• how does the recordbatchstream now look for the result ? Up until now we only had one result so that wasn't an issue but we should ideally get multiple RecordBatchStreams, so the result would also be an array of record batch streams ? Is this the expected behaviour you'd want or do we have to combine multiple streams.
  Thanks

[tullie]

Curious if there's any ETA for solving this? cc @trueutkarsh, @eddyxu, @changhiskhan. At the moment, we have to make many batch queries (batch_size=500) to lance, and this is a huge bottleneck for us.