feat!: Add dynamic name prefix & suffix to reset metric name

[cavivie]

This feature adds dynamic name prefix and suffix to Telegraf. This PR was modified to improve that the name (prefix/suffix) value cannot restore the name prefix suffix.

Here is an example config below:

[[inputs.disk]]
   name_prefix = "my_app_namespace_"

[[outputs.influxdb]]
  namepass = ["my_app_namespace_*"]
  name_prefix = ""
  urls = ["http://influxdb.example.com"]
  database = "db_other"

Keep the final output metric name is the name of the input, more routing metric exmaples:

Metrics are prefixed in the input, which is then used to select the output.
The metric name will be removed/overrode/appended in the outputs before writing.

[[inputs.exec]]
# ...
name_prefix = "exec1_"

[[inputs.exec]]
# ...
name_prefix = "exec2_"

[[inputs.exec]]
# ...
name_prefix = "exec3_"

[[inputs.exec]]
# ...
name_prefix = "exec4_"

# this will only get exec1
[[outputs.influxdb]]
# ...
namepass = "exec1_*"

# this will only get exec2
[[outputs.influxdb]]
# ...
namepass = "exec2_*"
# this will remove the previously set prefix
# so final metric name will be first input name
name_prefix = ""

# this will only get exec3
[[outputs.influxdb]]
# ...
namepass = "exec3_*"
# this will override the previously set prefix
# so final metric name will start with exec3v1_
name_prefix = "exec3v1_"

# this will only get exec4
[[outputs.influxdb]]
# ...
namepass = "exec4_*"
# this will appends to the front of the previously set prefix
# so final metric name will start with telegraf_exec4_
name_prefix = "+telegraf_"

Required for all PRs

• Updated associated README.md.
• Wrote appropriate unit tests.
• Pull request title or commits are in conventional commit format

resolves #

[telegraf-tiger]

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

[cavivie]

!signed-cla

[powersj]

@cavivie can you please file an issue and detail what problem you are trying to solve in more detail? Thanks

[cavivie]

@powersj In order to meet the situation discussed below, we provide a similar solution as follows from @sparrc :

[[inputs.exec]]
# ...
name_prefix = "exec1_"

[inputs.exec]]
# ...
name_prefix = "exec2_"

# this influxdb will only get exec1
[[outputs.influxdb]]
# ...
namepass = "exec1_*"

# this will only get exec2
[[outputs.influxdb]]
# ...
namepass = "exec2_*"

Not only do we add and match prefixes, but we also want to remove this prefix from the output to maintain consistency with the input.

[[inputs.exec]]
# ...
name_prefix = "exec1_"

[inputs.exec]]
# ...
name_prefix = "exec2_"

# this influxdb will only get exec1
[[outputs.influxdb]]
# ...
namepass = "exec1_*"
name_prefix = "" # keep output metric name with input name

# this will only get exec2
[[outputs.influxdb]]
# ...
namepass = "exec2_*"
# name_prefix = "" # not set keep output metric name with prefix exec2_

See discussions below:

• Support namespacing plugins #8696
• Routing metrics by plugin #1778
• [Feature request] Add alias name to output plugin and use it in input plugin #2272
• [Feature request] specify database for each input without adding multiple outputs #3028
• Send metrics to specific outputs #809
• ...

This goal allow us to complete functions similar to namespacing plugin at the metric level.

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[powersj]

@cavivie,

I asked for you to file an issue and instead you provided me a list of 4 closed issues, all explaining how it is possible to do today. The one remaining open issue proposed creating a namespace for each plugin to control the flow. However, the closed issues have comments about how and why this is not as straightforward as it seems.

Without understanding why the current mechanism causes friction I am not inclined to add yet another way to do something.

[cavivie]

@powersj Thank you! I agree and know the truth behind this. I just want to complete routing through namepass/prefix/suffix and remove prefix/suffix when outputting. I know there are already many solutions, but this is the initial solution I came up with, but it cannot be implemented because prefix/suffix only has append but no restore function, which makes me have to give up this processing method. But if other solutions are used, one is ugly, and the other is unclear about the existence of namespaces. I think other solutions can meet the requirements, but they are not elegant enough. Anyway, we can close this PR.

[powersj]

I think other solutions can meet the requirements, but they are not elegant enough.

I don't disagree that they are not elegant ;) however, in order to maintain backwards compatibility a re-write isn't an option currently. Adding new options as I mentioned above, might be ok but we would really want to understand why we are doing that and ensure it makes sense before adding the additional maintenance burden.