Skip to content

dpnolte/indexer-elastic

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

#Nutch-2.3.1-ES5-Indexer

##Description

Apache Nutch 2.3.x is currently designed to work with ElasticSearch 2.3.3. While this works for most I wanted to update the indexer-elastic in Apache Nutch to work with ElasticSearch 5. This is my initial attempt...

##Notable Changes

  • Updated ivy.xml in Apache Nutch:
    • Added Netty-all version 3 dependency
    • Updated org.restlet to 2.3.9
  • Updated indexer-elastic to redo connection using newer Transport Client methods.

##How To Get This Running:


 1. Copy ixy.xml to $NUTCH_HOME/ivy/ivy.xml
 2. Copy indexer-elastic to $NUTCH_HOME/ivy/ivy.xml
 3. Copy default.properties to $NUTCH_HOME/
 4. Add Following Configuration to $NUTCH_HOME/conf/nutch-site.xml in the < configuration > element
 
 ```xml
<property>
    <name>elastic.host</name>
    <value>localhost</value>
</property>
<property>
    <name>elastic.port</name>
    <value>9300</value>
</property>
<property>
    <name>elastic.cluster</name>
    <value>elasticsearch</value>
</property>
<property>
    <name>elastic.index</name>
    <value>nutchindex</value>
</property>
<property>
  <name>elastic.max.bulk.docs</name>
  <value>250</value>
  <description>Maximum size of the bulk in number of documents.</description>
</property>
<property>
  <name>elastic.max.bulk.size</name>
  <value>2500500</value>
  <description>Maximum size of the bulk in bytes.</description>
</property>
```

 5. Run ant runtime on $NUTCH_HOME then run nutch as you normally would.

About

ElasticSearch 5 Indexer for Apache Nutch 2.3

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 100.0%