Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

[cetic/nifi] Using subpath for persistence #289

Closed
timesy opened this issue Feb 25, 2023 · 2 comments · Fixed by #301
Closed

[cetic/nifi] Using subpath for persistence #289

timesy opened this issue Feb 25, 2023 · 2 comments · Fixed by #301
Labels
enhancement New feature or request

Comments

@timesy
Copy link

timesy commented Feb 25, 2023

Thank you for your work on the Helm chart, it was really helpful so far!

Is your feature request related to a problem? Please describe.
When using persistent storage with this chart, I want to use azurefile-premium (as it is costly with the many reads/writes on azurefile). However this would currently create a 100Gi volume for each directory (logs, config-data, data, ...).

Describe the solution you'd like
I would like to be able to set a sub path in order to put these files all on the same volume. I tested it locally by changing statefulset.yaml and values.yaml:

values.yaml

persistence:
    enabled: true
    subPath: data

statefulset.yaml (Just exemplary, would be replaced for every mountPath that has persistence):

- name: {{ .Values.persistence.subPath | default "auth-conf" }}
            mountPath: /opt/nifi/nifi-current/auth-conf/
            {{- if and .Values.persistence.enabled .Values.persistence.subPath }}
            subPath: auth-conf
	   {{- end }}

  ...

{{- if and .Values.persistence.enabled .Values.persistence.subPath }}
  volumeClaimTemplates:
    - metadata:
        name: {{ .Values.persistence.subPath }}
      spec:
        accessModes:
        {{- range .Values.persistence.accessModes }}
          - {{ . | quote }}
        {{- end }}
        storageClassName: {{ .Values.persistence.storageClass | quote }}
        resources:
          requests:
            storage: {{ .Values.persistence.size }}
{{- end }}
{{- if and .Values.persistence.enabled (not .Values.persistence.subPath) }}
 (same as before)
{{- end }}

Describe alternatives you've considered
Maybe I am missing something or there is a better way to reach that goal, however I wasn't able to find something else.

@banzo banzo added the enhancement New feature or request label Mar 2, 2023
@emrge-michaeld
Copy link
Contributor

I second this. 7 volumes is excessive. Digital Ocean only allows 7 PVs per node, so it is hard to fit nifi on there.

Here is the k8s docs on using subpath:
https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath
In response to the 'not use in production' see this link in response https://stackoverflow.com/questions/70358753/is-it-fundamentally-wrong-to-use-volumemounts-subpath-in-a-production-setup

@banzo
Copy link
Contributor

banzo commented Jun 2, 2023

@timesy @emrge-michaeld
Could you propose a pull request with this feature?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants