FUTURE OF ELASTIC-MODEL - PLEASE COMMENT

[clintongormley]

Hi all users of Elastic::Model

Elasticsearch 2.0.0 is out, and Elastic::Model doesn't support it. In fact, Elastic::Model doesn't support a number of things from Elasticsearch 1.x either. I apologise for neglecting this module.

My feeling is that Elastic::Model tries to do way too much. Like many frameworks, it ties you into doing things in a particular way, which may or may not make sense for your use case. Most people who use Elastic::Model seem to use a subset of the functionality, and then talk to Elasticsearch directly the rest of the time.

I don't think it makes sense to just update the code for 2.x, it needs a complete rethink

TELL ME HOW YOU USE IT

Please could you add comments to this issue explaining what bits you find useful, what bits you never use, and what bits you find annoying. Perhaps the code can be split out into smaller more useful chunks.

thanks

[diegok]

Hi @clintongormley

I'm one of those who use a subset of it and i'm also using raw ES queries some times. We have mainly two use cases:

1. As a search engine for our data-model
2. As a real time log analysis tool

On both cases we only have one type's so we don't use any relations. I've started using EM because of the moose serializer and I'm very happy with it, but what I've discovered to be the most important part is how it helps creating and maintaining the "mappings", it makes possible to write and a pleasure to read huge mappings. I've just found we have 26k lines of it on our log model!, maintaining it and using it is still fantastic thanks to the separation and reutilization that Elastic::Model::Types allows us.

Other than that I really like queryb but it makes harder to try queries on a ES console while copying back and forth and we're using more and more very complex aggs passing completely from EM.

Finishing... It would be very nice to be able to have the same async we have on ES :-)

[timbunce]

Hi @clintongormley!

We mostly use it for managing loading data into ES. We load data each day into a new index and repoint (hundreds of) aliases to the new index. The main type has ~30 fields, some with complex mappings.

Mappings include things like:

has_tokenizer 'zip_tokenizer' => ( type => 'pattern', pattern => '[^ \w]+|(?<!\b\w\d\w) (?!\d\w\d\b)' );
has_filter 'zip_filter' => (
    type => 'pattern_replace',
    pattern => '^(?!(\d{5}$|\w\d\w\s\d\w\d$)).*', # Match if NOT US "NNNNN" or CA "ANA NAN"
    replacement => '', # can be removed by a min_length_1 filter
);
has_analyzer  'zip_analyzer' => (
    tokenizer   => 'zip_tokenizer',
    filter      => ['zip_filter', 'min_length_1', 'lowercase'],
);

Alias management code has chunks like:

        $namespace->alias($board_alias)->to( $domain->name => {
            filterb => { mls_board_id => $mls_board_id },
            routing => $mls_board_id, # query only the appropriate shard
        });

We use Elastic::Model::View a little, including queryb.

[timbunce]

Ping! Any news on this @clintongormley?

[clintongormley]

Apologies for the silence. I've come to the reluctant conclusion that I don't have time to maintain or update this module. I would welcome somebody else stepping up to maintain it and would be happy to help or advise, but I have too much on my plate to do much else for the foreseeable future.

[diegok]

Hello @clintongormley,

I was working to to make EM to support Elasticsearch 2.x while keeping the support of 1.x and completely remove support for 0.90.x. It's now in a state where it pass all the tests and it let my company code to run on 2.4.0 and 1.7.4 with minimal changes, mostly replacing the usage of facets with aggs and wrapping the result to map facet() from agg().

https://github.com/diegok/Elastic-Model/tree/es2

I'm sure there is still a lot of room for improvements to make more 2.x (and even 1.x) features available, but I think this could be merged back and released. Do you care to check it out and let me know what do you think about it?.

@timbunce: Are you still interested on this?, in that case, could you please test this with your company code and let me know if you have any issue?

[timbunce]

Hey @diegok, that sounds great!
Yes, we're still interested, though sadly I'm unlikely to have the time to test in the short term.
If it passes the tests then I'd love to see a dev/trial release with this included.
@clintongormley, would you consider giving @diegok co-maint on EM?
A dev release would significantly lower the friction for myself (and others) to test out the code.

[clintongormley]

@diegok with pleasure - sorry for the late reply, been on holiday.

i've added you and @timbunce as collaborators on this repo, and given you both comaint perms on pause.

thanks for picking this up!

[diegok]

Search::Elasticsearch 5.0 was released when I was ready to merge and release my changes. v5.x has removed the support for old versions of ES and that makes a no-go to release this new version as it will never install clean from CPAN.

I was very sad about all this and moved away for some time but now I need to move on on this so I would like to know what do you think of it:

1. fork Search::Elasticsearch from v2.03 into Search::ElasticsearchV2 (or something like that), release it to CPAN and never touch it again.
2. replace the usage all around Elastic::Model to depend on that fork
3. document this and make clear Elastic::Model will not be updated to 5.x. I still like some features of EM but probably I would prefer to start fresh on a new namespace porting a small subset to work with >= 5.x

@clintongormley are you OK with this plan?, if not, do you have any idea of a better way to proceed?

[diegok]

Forget my last message. Today I've discovered that the new driver still has support for old versions but it should be installed separately. I will make it a dependency, document it properly and if everything goes OK, I'll push the new version to CPAN.

[timbunce]

For the record, I'm no longer working with Elastic::Model. Sadly.