rename of splunk_hec token to default_token not documented

[treethought]

The 0.19.0 highlights describe the changes to splunk acknowledgements and passthrough token but do not call out the ranming of token to default_token as a breaking change.

https://vector.dev/highlights/2021-12-28-0-19-0-upgrade-guide/

[jszwedko]

Hi @treethought ! Thanks for the report. token should still work too; it was just deprecated in-lieu of default_token. Are you find that not to be the case? Could you share your configuration if so? You are right that we didn't call out the rename and deprecation though. We'll remedy that.

[treethought]

Hey @jszwedko, yeah I only noticed the change because when deploying with the helm chart and with --set customConfig.sinks.splunk_sink.token="${SPLUNK_HEC_TOKEN}", the container failed to come up because default_token was not set

[jszwedko]

🤔 could you share the full output? I wasn't able to reproduce that running Vector locally using token.

[treethought]

Sure, here is the logs from the vector container

│ 2021-12-28T21:28:12.839914Z  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buff │
│ 2021-12-28T21:28:12.840026Z  INFO vector::app: Loading configs. paths=["/etc/vector"]                                                                            │
│ 2021-12-28T21:28:12.841108Z ERROR vector::cli: Configuration error. error=sinks.splunk_sink: missing field `default_token` at line 10 column 16   

With the following sink config. And as stated above, token is set via helm --set flag

    splunk_sink:
      compression: none
      encoding: text
      endpoint: https://splunk-endpoint
      healthcheck: false
      host_key: host
      index: staging
      indexed_fields:
      - cluster_name
      - k8s_source_type
      - k8s_container_image
      - k8s_container_name
      - k8s_pod_name
      - k8s_pod_namespace
      - k8s_pod_label_app
      - k8s_pod_label_pop
      - k8s_pod_label_function_id
      inputs:
      - k8s_cluster_logs
      token:
      type: splunk_hec

[treethought]

@jszwedko actually it does look like token still works. However, I think it requires the type to be renamed to splunk_hec_logs.

With type: splunk_hec_logs, token works. But it does not work with type: splunk_hec

[jszwedko]

Aaah, yeah, I see. We've run into this issue before with the configuration deserialization framework we use, serde. As you noted, splunk_hec was renamed to splunk_hec_logs. We have an alias for it as splunk_hec, but due to serde-rs/serde#1504 the field aliases don't also apply. This is a similar issue to #4903.

Apologies for the confusion here. I'll link this to #9115 so we can make sure whatever solution we come up with for the future of Vector's config deserialization handles these cases.

[treethought]

excellent, thanks for the context and quick response as always!