Skip to content

transcovo/nifi-compose-bundle

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 

Repository files navigation

nifi-compose-bundle

An example NiFi Archive (.nar) with some custom Processors

Quick Build and Deploy

  1. git clone
  2. mvn package
  3. Copy nifi-compose-nar/target/nifi-compose-nar-1.0.0-BETA.nar to $NIFI_ROOT/lib assuming you took the defaults from the nifi.properties file and that you are running Apache Nifi version 1.0.0-Beta.

For details on these custom Processors please see the following blog (especially the end): What You Need to Know to Extend NiFi

For a general overview of Apache NiFi: Use NiFi to Lessen the Friction of Moving Data

Custom Processors in this bundle:

  • ComposeUniqueRocksDB provides the ability to locally check for uniqueness of an attribute. It is a persisted index of keys. Useful for local only unlike the de-duplication provided by NiFi via DetectDuplicate and its required Distributed Cache Service.

  • ComposeStreamingGetMongo provides immediate creation of FlowFiles for each Document prior to the query being finished. This is different than GetMongo which only calls session.commit() after the query is completed.

  • ComposeBatchPutMongo exposes Mongo's collection.insertMany(). It takes a FlowFile that has a JSON array of Documents to put into Mongo.

  • ComposeTailingGetMongo provides a snapshot of Mongo database plus a stream of oplog changes in the form of FlowFiles.

  • ComposeTailingPutMongo puts the dump from the TailingGet into another Mongo.

About

A NiFi Archive (nar) with some custom Processors

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages