X-Ray: how to set index_all_attributes (or provide a list of attributes to index)?

[benabineri]

Is your feature request related to a problem? Please describe.

I have added attributes to spans in my code and I would like them to be indexed so I can query them as annotations in X-Ray.

I can see that there's a config file (/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.yaml) which has this setting in it:

...
exporters:
    awsxray:
        ...
        index_all_attributes: false
        ...
...

I am configuring the agent in my user-data like this:

echo '{"agent":{"metrics_collection_interval":60,"run_as_user":"root"},"traces":{"traces_collected":{"xray":{},"otlp":{}}}}' > /tmp/amazon-cloudwatch-agent.json
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/tmp/amazon-cloudwatch-agent.json

I can't see that it's possible to set the index_all_attributes setting from this config file. I have tried various JSON blobs e.g.

echo '{"agent":{"metrics_collection_interval":60,"run_as_user":"root"},"traces":{"traces_collected":{"xray":{},"otlp":{}}},"exporters":{"awsxray":{"index_all_attributes":true}}}' > /tmp/amazon-cloudwatch-agent.json

Describe the solution you'd like

I'd like to be able to configure the agent to index these attributes.

Describe alternatives you've considered

I suppose I could use the ADOT collector but I've already put effort into configuring the Cloudwatch Agent.

[jefchien]

If you're using the X-Ray SDK, then the AWS X-Ray receiver should be able to retain the annotations set on the segments. If you're using the OTEL SDK, then there is a workaround where a slice attribute aws.xray.annotations can be added to spans (open-telemetry/opentelemetry-collector-contrib#17855) that includes the annotations that should be indexed. The AWS X-Ray exporter will remove this attribute before sending the segments to X-Ray.

The CloudWatch agent doesn't currently expose the index_all_attributes field in its configuration.

[benabineri]

Yes, setting an attribute like this:

attribute.StringSlice("aws.xray.annotations", annotationsToAdd),

where annotationsToAdd is a slice of attribute names was enough to get these indexed.

Thanks.