_parent mapping and index ? #64

kwonder · 2013-03-28T08:48:33Z

how to parent mapping and index ?
PLZ

richardwilly98 · 2013-04-02T16:31:25Z

Index and mapping can be created before the river. Custom mapping can be implemented using this technique.

richardwilly98 · 2013-04-02T16:31:49Z

See here [1] for example.

[1] - https://github.com/richardwilly98/elasticsearch-river-mongodb/tree/master/resources/issues/54

kwonder · 2013-04-03T08:25:00Z

I have many time try to create implementation _river auto set _parent when index data but it is not working, the example
curl -XPUT 'http://localhost:9200/_river/post/_meta' -d '{ "type":"mongodb", "mongodb":{ "db":"myData", "collection":"post", "script":"if(ctx.document._parentId) ctx._parent = ctx.document._parentId" }, "index":{ "name":"myIndex", "type":"post" } }'
this is issues of CouchDB-River but it was fix, view link elastic/elasticsearch#1300
PLZ Support setting ctx._parent in MongoDB-River

Example of script: if(ctx.document._parentId) { ctx._parent = ctx.document._parentId; delete ctx.document._parentId;}

kwonder · 2013-04-04T09:13:39Z

Thank you for support
How can I use it (implements feature #64 ) for the installation on the server. I can just install elasticsearch-river-MongoDB-1.6.4, can not install your version have (implements feature #64 ) , my install:
bin/plugin -url https://github.com/downloads/richardwilly98/elasticsearch-river-mongodb/elasticsearch-river-mongodb-1.6.4.zip -install river-mongodb

richardwilly98 · 2013-04-04T09:39:19Z

Hi,

I need to release a new version. I will do that later today.

Thanks,
Richard.
Sent via BlackBerry by AT&T

-----Original Message-----
From: kwonder notifications@github.com
Date: Thu, 04 Apr 2013 02:13:40
To: richardwilly98/elasticsearch-river-mongodbelasticsearch-river-mongodb@noreply.github.com
Reply-To: richardwilly98/elasticsearch-river-mongodb reply@reply.github.com
Cc: Richard Louaprerichard.louapre@gmail.com
Subject: Re: [elasticsearch-river-mongodb] _parent mapping and index ? (#64)

Thank you for support
How can I use it (implements feature #64 ) for the installation on the server. I can just install elasticsearch-river-MongoDB-1.6.4, can not install your version have (implements feature #64 ) , my install:
bin/plugin -url https://github.com/downloads/richardwilly98/elasticsearch-river-mongodb/elasticsearch-river-mongodb-1.6.4.zip -install river-mongodb

Reply to this email directly or view it on GitHub:
#64 (comment)

kwonder · 2013-04-04T09:46:19Z

great, thank very much

richardwilly98 · 2013-04-04T23:15:29Z

You can use https://github.com/downloads/richardwilly98/elasticsearch-river-mongodb/elasticsearch-river-mongodb-1.6.5.zip

Please let me know if it works.

kwonder · 2013-04-05T08:42:00Z

so thank you for supporting this feature but it is not working on version 1.6.5, the following is my sample set
curl -XPUT 'http://localhost:9200/_river/user/_meta' -d '{ "type": "mongodb", "mongodb": { "db": "myData", "collection": "user", "script": "if(ctx.document.password) { ctx.deleted = true; }" }, "index": { "name": "myIndex", "type": "user" } }'

curl -XPUT 'http://localhost:9200/_river/post/_meta' -d '{ "type": "mongodb", "mongodb": { "db": "myData", "collection": "post", "script": "if(ctx.document._catagoryId) { ctx._parent = ctx.document._catagoryId; delete ctx.document._catagoryId;}" }, "index": { "name": "myIndex", "type": "post" } }'

richardwilly98 · 2013-04-05T10:01:06Z

Hi,

Can you please provide ES log and some sample data to store in Mongo?

Thanks,
Richard.

Sent via BlackBerry by AT&T

-----Original Message-----
From: kwonder notifications@github.com
Date: Fri, 05 Apr 2013 01:42:02
To: richardwilly98/elasticsearch-river-mongodbelasticsearch-river-mongodb@noreply.github.com
Reply-To: richardwilly98/elasticsearch-river-mongodb reply@reply.github.com
Cc: Richard Louaprerichard.louapre@gmail.com
Subject: Re: [elasticsearch-river-mongodb] _parent mapping and index ? (#64)

so thank you for supporting this feature but it is not working on version 1.6.5, the following is my sample set
curl -XPUT 'http://localhost:9200/_river/user/_meta' -d '{ "type": "mongodb", "mongodb": { "db": "myData", "collection": "user", "script": "if(ctx.document.password) { ctx.deleted = true; }" }, "index": { "name": "myIndex", "type": "user" } }'

curl -XPUT 'http://localhost:9200/_river/post/_meta' -d '{ "type": "mongodb", "mongodb": { "db": "myData", "collection": "post", "script": "if(ctx.document._catagoryId) { ctx._parent = ctx.document._catagoryId; delete ctx.document._catagoryId;}" }, "index": { "name": "myIndex", "type": "post" } }'

Reply to this email directly or view it on GitHub:
#64 (comment)

richardwilly98 · 2013-04-05T10:38:17Z

Hi,

Did you create the custom mapping in "myIndex" before to start the river (see here an example [1])?

You can also find here [2] the unit test used to validate this feature.

[1] - https://github.com/richardwilly98/elasticsearch-river-mongodb/blob/ae10fcff050cf08e128ccc91e23fe46c013cdf2d/resources/issues/64/_01_create-index-with-mapping.json
[2] - https://github.com/richardwilly98/elasticsearch-river-mongodb/tree/ae10fcff050cf08e128ccc91e23fe46c013cdf2d/resources/issues/64

Thanks,
Richard.

kwonder · 2013-04-05T16:22:19Z

Thank @richardwilly98 , I've done anything that you'd guided me to do, but only author type was indexed, and book type wasn't. My log file I had after testing this function is:

[2013-04-05 23:20:23,712][INFO ][cluster.metadata ] [Orka] [authors] creating index, cause [api], shards [1]/[1], mappings [author, book]
[2013-04-05 23:21:08,988][INFO ][cluster.metadata ] [Orka] [_river] creating index, cause [auto(index api)], shards [1]/[1], mappings []
[2013-04-05 23:21:09,289][INFO ][cluster.metadata ] [Orka] [_river] update_mapping authors54
[2013-04-05 23:21:09,314][INFO ][river.mongodb ] [Orka] [mongodb][authors54] Using mongodb server(s): host [localhost], port [27017]
[2013-04-05 23:21:09,315][INFO ][river.mongodb ] [Orka] [mongodb][authors54] starting mongodb stream. options: secondaryreadpreference [false], throttlesize [500], gridfs [false], filter [], db [mydb], script [null], indexing to [authors]/[author]
[2013-04-05 23:21:09,548][INFO ][river.mongodb ] [Orka] [mongodb][authors54] No known previous slurping time for this collection
[2013-04-05 23:21:09,549][INFO ][cluster.metadata ] [Orka] [_river] update_mapping authors54
[2013-04-05 23:21:10,288][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] Indexed 3 documents, 3 insertions 0, updates, 0 deletions, 3 documents per second
[2013-04-05 23:21:10,289][INFO ][cluster.metadata ] [Orka] [_river] update_mapping authors54
[2013-04-05 23:21:10,561][INFO ][cluster.metadata ] [Orka] [_river] update_mapping books54
[2013-04-05 23:21:10,571][WARN ][river ] [Orka] failed to create river [mongodb][books54]
org.elasticsearch.common.inject.CreationException: Guice creation errors:

Error injecting constructor, org.elasticsearch.ElasticSearchIllegalArgumentException: script_lang not supported [js]
at org.elasticsearch.river.mongodb.MongoDBRiver.(Unknown Source)
while locating org.elasticsearch.river.mongodb.MongoDBRiver
while locating org.elasticsearch.river.River

1 error
at org.elasticsearch.common.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:343)
at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:178)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:129)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:66)
at org.elasticsearch.river.RiversService.createRiver(RiversService.java:138)
at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:270)
at org.elasticsearch.river.RiversService$ApplyRivers$2.onResponse(RiversService.java:264)
at org.elasticsearch.action.support.TransportAction$ThreadedActionListener$1.run(TransportAction.java:87)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.elasticsearch.ElasticSearchIllegalArgumentException: script_lang not supported [js]
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:173)
at org.elasticsearch.script.ScriptService.executable(ScriptService.java:181)
at org.elasticsearch.river.mongodb.MongoDBRiver.(MongoDBRiver.java:317)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:54)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:98)
at org.elasticsearch.common.inject.FactoryProxy.get(FactoryProxy.java:52)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:819)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:56)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45)
at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:200)
at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:812)
at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
... 10 more
[2013-04-05 23:21:10,606][INFO ][cluster.metadata ] [Orka] [_river] update_mapping books54

kwonder · 2013-04-05T16:29:02Z

Oh, it works when install "plugin -install elasticsearch/elasticsearch-lang-javascript/1.3.0". Thanks again!

richardwilly98 added a commit that referenced this issue Apr 3, 2013

Implement feature #64

0411b84

Example of script: if(ctx.document._parentId) { ctx._parent = ctx.document._parentId; delete ctx.document._parentId;}

richardwilly98 added a commit that referenced this issue Apr 5, 2013

Working example for feature #64

ae10fcf

mzafer mentioned this issue May 8, 2013

mongodb to elasticsearch removal strategy #42

Closed

andriuwe4ka mentioned this issue May 13, 2013

mapping #75

Closed

richardwilly98 closed this as completed May 14, 2013

mzafer mentioned this issue Jun 12, 2013

Index fails if field used for _parent in the script is ObjectId #85

Closed

richardwilly98 mentioned this issue Sep 13, 2013

Add children attribute feature + Groovy test case (RiverMongoChildrenTes... #106

Closed

richardwilly98 mentioned this issue Mar 20, 2014

CreationException #237

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

_parent mapping and index ? #64

_parent mapping and index ? #64

kwonder commented Mar 28, 2013

richardwilly98 commented Apr 2, 2013

richardwilly98 commented Apr 2, 2013

kwonder commented Apr 3, 2013

kwonder commented Apr 4, 2013

richardwilly98 commented Apr 4, 2013

kwonder commented Apr 4, 2013

richardwilly98 commented Apr 4, 2013

kwonder commented Apr 5, 2013

richardwilly98 commented Apr 5, 2013

richardwilly98 commented Apr 5, 2013

kwonder commented Apr 5, 2013

kwonder commented Apr 5, 2013

_parent mapping and index ? #64

_parent mapping and index ? #64

Comments

kwonder commented Mar 28, 2013

richardwilly98 commented Apr 2, 2013

richardwilly98 commented Apr 2, 2013

kwonder commented Apr 3, 2013

kwonder commented Apr 4, 2013

richardwilly98 commented Apr 4, 2013

kwonder commented Apr 4, 2013

richardwilly98 commented Apr 4, 2013

kwonder commented Apr 5, 2013

richardwilly98 commented Apr 5, 2013

richardwilly98 commented Apr 5, 2013

kwonder commented Apr 5, 2013

kwonder commented Apr 5, 2013