Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

Commit

Permalink
Update Nifi to 1.14.0 version (#169)
Browse files Browse the repository at this point in the history 
* Nifi has been updated to version 1.14.0. This version comes already secured.
* Adding of differents ways to log in Nifi (Single-User, OIDC, LDAP)
* Adding of OpenLDAP dependencie
* Adapting files in conf/ to provide OIDC and LDAP
* Adding doc folder to show how to configure a Single-User, OIDC or LDAP authentication
* Various bugfixes and cleanup
  • Loading branch information
@zakaria2905
zakaria2905 authored Nov 10, 2021
1 parent 7f03bf5 commit 0867843
Show file tree
Hide file tree
Showing 34 changed files with 418 additions and 247 deletions.
3 changes: 3 additions & 0 deletions .helmignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,6 @@
*.tmproj

.circleci/

doc/
.github/
8 changes: 6 additions & 2 deletions Chart.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
apiVersion: v2
name: nifi
version: 0.7.9
appVersion: 1.12.1
version: 1.0.0
appVersion: 1.14.0
description: Apache NiFi is a software project from the Apache Software Foundation designed to automate the flow of data between software systems.
keywords:
- nifi
Expand Down Expand Up @@ -38,3 +38,7 @@ dependencies:
- name: ca
version: 1.0.1
condition: ca.enabled
- name: openldap
version: ~1.2.4
repository: https://charts.helm.sh/stable
condition: openldap.enabled
30 changes: 11 additions & 19 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

## Introduction

This [Helm](https://helm.sh/) chart installs [Apache NiFi](https://nifi.apache.org/) in a [Kubernetes](https://kubernetes.io/) cluster.
This [Helm](https://helm.sh/) chart installs [Apache NiFi](https://nifi.apache.org/) 1.14.0 in a [Kubernetes](https://kubernetes.io/) cluster.

## Prerequisites

Expand Down Expand Up @@ -40,10 +40,7 @@ The following items can be set via `--set` flag during installation or configure

#### Configure authentication

- You first need a secure cluster which can be accomplished by enabling the built-in CA nifi-toolkit container (`ca.enabled` to true). By default, a secure nifi cluster uses certificate based authentication but you can optionally enable `ldap` or `oidc`. See the configuration section for more details.

:warning: This feature is quite new. Please open an issue if you encounter a problem.
It seems that versions from 0.6.1 include some bugs for authentications. Please use version 0.6.0 of the chart until it is fixed.
- By default, the authentication is a `Single-User` authentication. You can optionally enable `ldap` or `oidc` to provide an external authentication. See the [configuration section](README.md#configuration) or [doc](doc/) folder for more details.

#### Use custom processors

Expand Down Expand Up @@ -78,15 +75,7 @@ helm install my-release cetic/nifi

### Install from local clone

```bash
git clone https://github.com/cetic/helm-nifi.git nifi
cd nifi
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add dysnix https://dysnix.github.io/charts/
helm repo update
helm dep up
helm install nifi .
```
You will find how to perform an installation from a local clone on this [page](doc/INSTALLATION.md).

## Uninstallation

Expand All @@ -106,7 +95,7 @@ The following table lists the configurable parameters of the nifi chart and the
| `replicaCount` | Number of nifi nodes | `1` |
| **Image** |
| `image.repository` | nifi Image name | `apache/nifi` |
| `image.tag` | nifi Image tag | `1.12.1` |
| `image.tag` | nifi Image tag | `1.14.0` |
| `image.pullPolicy` | nifi Image pull policy | `IfNotPresent` |
| `image.pullSecret` | nifi Image pull secret | `nil` |
| **SecurityContext** |
Expand All @@ -131,20 +120,23 @@ The following table lists the configurable parameters of the nifi chart and the
| `properties.httpPort` | web properties HTTP port | `8080` |
| `properties.httpsPort` | web properties HTTPS port | `null` |
| `properties.clusterPort` | cluster node port | `6007` |
| `properties.clusterSecure` | cluster nodes secure mode | `false` |
| `properties.needClientAuth` | nifi security client auth | `false` |
| `properties.provenanceStorage` | nifi provenance repository max storage size | `8 GB` |
| `properties.siteToSite.secure` | Site to Site properties Secure mode | `false` |
| `properties.siteToSite.port` | Site to Site properties Secure port | `10000` |
| `properties.siteToSite.authorizer` | | `managed-authorizer` |
| `properties.safetyValve` | Map of explicit 'property: value' pairs that overwrite other configuration | `nil` |
| `properties.customLibPath` | Path of the custom libraries folder | `nil` |
| **nifi user authentication** |
| `properties.webProxyHost` | Proxy to access to Nifi through the cluster ip address | `Port:30236`
| **[Authentication](/doc/USERMANAGEMENT.md)** |
| **Single-user authentication** | Automatically disabled if OIDC or LDAP enabled
| `auth.singleUser.username` | Single user identity | `username` |
| `auth.singleUser.password` | Single user password | `changemechangeme` |
| **Ldap authentication** |
| `auth.admin` | Default admin identity | ` CN=admin, OU=NIFI` |
| `auth.ldap.enabled` | Enable User auth via ldap | `false` |
| `auth.ldap.host` | ldap hostname | `ldap://<hostname>:<port>` |
| `auth.ldap.searchBase` | ldap searchBase | `CN=Users,DC=example,DC=com` |
| `auth.ldap.searchFilter` | ldap searchFilter | `CN=john` |
| **Oidc authentication**
| `auth.oidc.enabled` | Enable User auth via oidc | `false` |
| `auth.oidc.discoveryUrl` | oidc discover url | `https://<provider>/.well-known/openid-configuration` |
| `auth.oidc.clientId` | oidc clientId | `nil` |
Expand Down
89 changes: 57 additions & 32 deletions configs/authorizers-empty.xml
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
{{- $replicas := int .Values.replicaCount }}
{{- $chart := .Chart.Name }}
{{- $release := .Release.Name }}
{{- $fullname := include "apache-nifi.fullname" . }}
{{- $namespace := .Release.Namespace }}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
Expand Down Expand Up @@ -40,8 +45,14 @@
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">../data/users.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
{{- range $i := until $replicas }}
<property name="Initial User Identity {{ $i }}">john</property>
{{- end }}
{{- if .Values.auth.ldap.enabled}}
<property name="Initial User Identity admin">{{.Values.auth.ldap.admin}}</property>
{{- end}}
</userGroupProvider>
<!--
The LdapUserGroupProvider will retrieve users and groups from an LDAP server. The users and groups
Expand Down Expand Up @@ -104,33 +115,33 @@
NOTE: Any identity mapping rules specified in nifi.properties will also be applied to the user identities.
Group names are not mapped.
-->
<!-- To enable the ldap-user-group-provider remove 2 lines. This is 1 of 2.
{{- if .Values.auth.ldap.enabled}}
<userGroupProvider>
<identifier>ldap-user-group-provider</identifier>
<class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class>
<property name="Authentication Strategy">START_TLS</property>
<property name="Manager DN"></property>
<property name="Manager Password"></property>
<property name="TLS - Keystore"></property>
<property name="TLS - Keystore Password"></property>
<property name="TLS - Keystore Type"></property>
<property name="TLS - Truststore"></property>
<property name="TLS - Truststore Password"></property>
<property name="TLS - Truststore Type"></property>
<property name="TLS - Client Auth"></property>
<property name="TLS - Protocol"></property>
<property name="TLS - Shutdown Gracefully"></property>
<property name="Referral Strategy">FOLLOW</property>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">{{.Values.auth.ldap.admin}}</property>
<property name="Manager Password">{{.Values.auth.ldap.pass}}</property>
<property name="TLS - Keystore">/opt/nifi/nifi-current/conf/{{.Release.Name}}-nifi-0.{{.Release.Name}}-nifi-headless.{{.Values.properties.namespace}}.svc.cluster.local/keystore.jks</property>
<property name="TLS - Keystore Password">{{.Values.auth.SSL.keystorePasswd}}</property>
<property name="TLS - Keystore Type">jks</property>
<property name="TLS - Truststore">/opt/nifi/nifi-current/conf/{{.Release.Name}}-nifi-0.{{.Release.Name}}-nifi-headless.{{.Values.properties.namespace}}.svc.cluster.local/truststore.jks</property>
<property name="TLS - Truststore Password">{{.Values.auth.SSL.truststorePasswd}}</property>
<property name="TLS - Truststore Type">JKS</property>
<property name="TLS - Client Auth">NONE</property>
<property name="TLS - Protocol">TLS</property>
<property name="TLS - Shutdown Gracefully">false</property>
<property name="Referral Strategy">IGNORE</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url"></property>
<property name="Url">{{.Values.auth.ldap.host}}</property>
<property name="Page Size"></property>
<property name="Sync Interval">30 mins</property>
<property name="User Search Base"></property>
<property name="User Search Base">{{.Values.auth.ldap.searchBase}}</property>
<property name="User Object Class">person</property>
<property name="User Search Scope">ONE_LEVEL</property>
<property name="User Search Filter"></property>
<property name="User Identity Attribute"></property>
<property name="User Search Filter">{{.Values.auth.ldap.searchFilter}}</property>
<property name="User Identity Attribute">{{.Values.auth.ldap.UserIdentityAttribute}}</property>
<property name="User Group Name Attribute"></property>
<property name="User Group Name Attribute - Referenced Group Attribute"></property>
<property name="Group Search Base"></property>
Expand All @@ -141,7 +152,7 @@
<property name="Group Member Attribute"></property>
<property name="Group Member Attribute - Referenced User Attribute"></property>
</userGroupProvider>
To enable the ldap-user-group-provider remove 2 lines. This is 2 of 2. -->
{{- end}}

<!--
The CompositeUserGroupProvider will provide support for retrieving users and groups from multiple sources.
Expand All @@ -151,13 +162,14 @@
NOTE: Any identity mapping rules specified in nifi.properties are not applied in this implementation. This behavior
would need to be applied by the base implementation.
-->
<!-- To enable the composite-user-group-provider remove 2 lines. This is 1 of 2.
{{- if .Values.auth.ldap.enabled}}
<userGroupProvider>
<identifier>composite-user-group-provider</identifier>
<class>org.apache.nifi.authorization.CompositeUserGroupProvider</class>
<property name="User Group Provider 1"></property>
<identifier>composite-configurable-user-group-provider</identifier>
<class>org.apache.nifi.authorization.CompositeConfigurableUserGroupProvider</class>
<property name="Configurable User Group Provider">file-user-group-provider</property>
<property name="User Group Provider 1">ldap-user-group-provider</property>
</userGroupProvider>
To enable the composite-user-group-provider remove 2 lines. This is 2 of 2. -->
{{- end}}

<!--
The CompositeConfigurableUserGroupProvider will provide support for retrieving users and groups from multiple sources.
Expand Down Expand Up @@ -207,9 +219,21 @@
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">../data/authorizations.xml</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
{{- if .Values.auth.ldap.enabled}}
<property name="Initial Admin Identity">{{.Values.auth.ldap.admin}}</property>
{{- else }}
<property name="Initial Admin Identity">john</property>
{{- end}}
<property name="Legacy Authorized Users File"></property>
{{- if .Values.auth.ldap.enabled}}
{{- range $i := until $replicas }}
<property name="Node Identity {{ $i }}">CN={{ $fullname }}-{{ $i }}.{{ $fullname }}-headless.{{ $namespace }}.svc.cluster.local, OU=NIFI</property>
{{- end }}
{{- end }}
<property name="Node Identity"></property>
</accessPolicyProvider>
<!--
<!--
The StandardManagedAuthorizer. This authorizer implementation must be configured with the
Access Policy Provider which it will use to access and manage users, groups, and policies.
These users, groups, and policies will be used to make all access decisions during authorization
Expand Down Expand Up @@ -244,14 +268,15 @@
NOTE: Any identity mapping rules specified in nifi.properties will also be applied to the node identities,
so the values should be the unmapped identities (i.e. full DN from a certificate).
-->
<!-- <authorizer>
{{- if .Values.auth.ldap.enabled}}
<authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity"></property>
<property name="Initial Admin Identity">{{.Values.auth.ldap.admin}}</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>

</authorizer>
-->
</authorizers>
{{- end}}
</authorizers>
6 changes: 3 additions & 3 deletions configs/authorizers.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./auth-conf/users.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
{{- range $i := until $replicas }}
<property name="Initial User Identity {{ $i }}">CN={{ $fullname }}-{{ $i }}.{{ $fullname }}-headless.{{ $namespace }}.svc.cluster.local, OU=NIFI</property>
Expand Down Expand Up @@ -221,7 +221,7 @@
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./auth-conf/authorizations.xml</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
{{- if .Values.auth.ldap.enabled}}
<property name="Initial Admin Identity">{{.Values.auth.ldap.admin}}</property>
{{- else }}
Expand Down Expand Up @@ -279,4 +279,4 @@

</authorizer>
{{- end}}
</authorizers>
</authorizers>
Loading

0 comments on commit 0867843

Please sign in to comment.