[Elasticsearch] Way forward for ES integration?

[dkarlovi]

Q	A
Bug report?	no
Feature request?	no
BC Break report?	no
RFC?	yes

So, looking at search-elasticsearch, it seems it some ways off from general availability. As my target use-case is using Elasticsearch with API Platform (as noted in api-platform/core#991) and your elaborate search solution seems like a good fit, what would be a good way for me to go forward?

I'd fork the repo to add some of the required functionality myself, but seeing all code is currently in a WIP branch, don't want to branch from a dead end.

Also, do you have any docs with planned features?

[sstok]

I pushed some bug fixes and other changes.

If you can help with the implementation that would be great 👍 especially the functional testing.
The Doctrine DBAL/ORM condition-processors use mocked-up e-commerce system to ensure the most common senario's work. If you can help with setting this up for ElasticSearch then writing the implementation will be easier as we can know for sure it's working as expected.

[dkarlovi]

I'm guessing you mean rollerworks/search-elasticsearch@738bb8c?

The Doctrine DBAL/ORM condition-processors use mocked-up e-commerce system to ensure the most common senario's work.

OK, seems fair, I'll look into it right away.

[dkarlovi]

@sstok could you submit to Packagist? That should allow me to install my fork for testing.

[dkarlovi]

Also, I see you've removed Elastica as a dependency, can you clarify why?

[sstok]

I removed it because it's not really needed to make this integration work.
The generator can produce a Query using array's only (which saves allot of memory in this case).

But don't worry an adapter for Elastica is definitely planned ;)

[dkarlovi]

OK, I guess it's OK to include it as a dev requirement?

[dkarlovi]

After merging the query generator, there are a couple of things needed here still, please correct me if I'm wrong here somewhere:

1. ElasticsearchFactory in search-elasticsearch, this one seems simple. Done in Add a Generator interface, CachedGenerator + Factory search-elasticsearch#3
2. search-elasticsearch support in RollerworksSearchBundle, both "regular" and api-platform sections, progress in WIP: Add Elasticsearch support RollerworksSearchBundle#37
3. search-elasticsearch support in search-api-platform, much like the Doctrine stuff, progress in WIP: Add Elasticsearch support search-api-platform#8

For 2):

1. we'd need to figure out how to build the mapping for the index/type. For now, I think it would be OK to rely on the user to do it on his/her own if the bundle suggests FOS Elastica Bundle (this would probably make sense because, as a advanced use case, you probably want control over your mappings). Delegate this to FOS Elastica bundle, we're just using it.
2. Also, if you have both Doctrine and Elasticsearch enabled, it should do a query on Elastic and then fetch the records (by ID) from Doctrine. Forgot FOS Elastica Bundle already does that, one more reason to depend on it. We're actually modifying the query from an Extension currently, it should be OK for now.

[sstok]

Agreed. RollerworksSearch is about the searching of records, index building and mapping is better suited by there integrations. This is also the main goal of this system, to generalize the searching in various storage systems, as this is the most complex part of any search system.

So for the API-Platform I guess we only need a CollectionDataProvider implementation that performs the Elasticsearch query and returns the result for the API-Platorm serializer part.
And a Paginator of course.

There is noting needed for writing on our part?

[sstok]

FYI, I looked into merging all repositories into a single one (while preserving the history as much as possible). This is possible but will require some work 💯 😄 there are total of 10 repositories (well 9 actually as pomm is empty atm).

So 🔜 we will have a mono-rep to speed-up development.