v1.0.0-ES-5.6.x-BETA-01

Elasticsearch (5.6.x) Data Import Handler, also supporting Algolia

A data import handler for Elasticsearch/Algolia

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch/Algolia

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

v0.20.3-ALPHA

Elasticsearch/Algolia Data Import Handler

A data import handler for Elasticsearch/Algolia

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch/Algolia

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.20.3-ALPHA

• combineDateTime UDF fix
• Non-string ids for Algolia records deletion fix

v0.20.2-ALPHA

Elasticsearch/Algolia Data Import Handler

A data import handler for Elasticsearch/Algolia

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch/Algolia

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.20.2-ALPHA

• Algolia Spark task serialization fix

v0.20.1-ALPHA

Elasticsearch/Algolia Data Import Handler

A data import handler for Elasticsearch/Algolia

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch/Algolia

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.20.1-ALPHA

• Algolia index records delete support using a configured query returning the objectIDs to be deleted

v0.20.0-ALPHA

Elasticsearch/Algolia Data Import Handler

A data import handler for Elasticsearch/Algolia

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch/Algolia

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.20.0-ALPHA

• Algolia index as an import target support

v0.10.0-ALPHA

Elasticsearch Data Import Handler

A data import handler for Elasticsearch

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.10.0-ALPHA

• Import queries are no longer cached between query run and count being called, this lowers memory use, but could lower performance. Preparatory statements were never cached by default.
• You can set cache settings (for example "cache" = true, "persist" = "MEMORY_ONLY") on any preparatory or import statement to add cache where needed and decide if all in memory, or spill to disk. See README for more information.

v0.9.0-ALPHA

Elasticsearch Data Import Handler

A data import handler for Elasticsearch

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.9.0-ALPHA

• Allow setting Spark configuration in the import configuration file
• Allow using sqlFile to reference external SQL files instead of inlining all SQL statements in the configuration file

v0.8.0-ALPHA

Elasticsearch Data Import Handler

A data import handler for Elasticsearch

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.8.0-ALPHA

• SSL support for ES
• BasicAuth support for ES
• Documentation including connecting to Elastic Cloud

v0.7.1-ALPHA

Elasticsearch Data Import Handler

A data import handler for Elasticsearch

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.7.1-ALPHA

• Added applying settings/mapping file to newly created indexes (fixed, had relative pathing bug)

v0.6.0-ALPHA

Elasticsearch Data Import Handler

A data import handler for Elasticsearch

• Simple
• Powerful
• Use SQL statements that can span multiple databases, text files, and Elasticsearch indexes
• Process full load and incremental updates
• Output columnar and structured JSON to Elasticsearch

Running is simple. With Java 8 installed, download a release and then run it:

kohesive-es-dih <configFile.conf>

Read the docs for configuration details.

New in v0.6.0-ALPHA

• New HTML stripping and entity decoding UDF functions for SQL
• Better state management is now per index per statement (instead of just statement ID)
• Fix to lastRun date being set incorrectly on failures
• Report number of documents processed (to screen, and to log index)
• Some error messages were not reported, now they are