Add support for dynamic attributes in S3 prefix for AWS S3 Exporter

[shokada]

Component(s)

exporter/awss3

Is your feature request related to a problem? Please describe.

We are currently unable to dynamically include pod annotations in the s3_prefix when using the AWS S3 Exporter with the OpenTelemetry Collector. We want to organize our S3 object paths with pod-specific information to easily identify and categorize logs from different pods.

Describe the solution you'd like

We would like the AWS S3 Exporter to be enhanced to support dynamic attributes in the s3_prefix. This would allow us to construct S3 object paths that include values from specific attributes, such as pod annotations. For example, we could use the logging/prefix annotation from a pod's metadata to structure the path like so:

pod_log/cluster_name=${cluster_name}/namespace_name=${namespace_name}/prefix=${logging/prefix}/year=XXXX/month=XX/day=XX/hour=XX/

Describe alternatives you've considered

As an alternative, we are considering using Fluent Bit, which supports dynamic values in the S3 object key configuration. This would allow us to include pod-specific annotations in the S3 paths. However, we would prefer to use OpenTelemetry Collector if it supported dynamic attributes in the S3 prefix, as it would keep our observability stack consistent and leverage OpenTelemetry's advanced processing capabilities.

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/awss3: @atoulme @pdelewski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/awss3: @atoulme @pdelewski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been closed as inactive because it has been stale for 120 days with no activity.