Using AWS resource detector with auto-instrumentation

[abstractOwl]

Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main.

Running a basic Flask application instrumented using opentelemetry-instrument in Docker python:3.12.2-slim. Set environment variable OTEL_EXPERIMENTAL_RESOURCE_DETECTORS=aws_ecs.

Dependency versions:

• Flask==3.0.2
• opentelemetry-distro[otlp]==0.41b0
• opentelemetry-sdk-extension-aws==2.0.1

Steps to reproduce
Describe exactly how to reproduce the error. Include a code sample if applicable.

Run docker-compose up -d with entrypoint opentelemetry-instrument flask run --host=0.0.0.0 --port=5000

What is the expected behavior?
What did you expect to see?
Start up using ECS resource detector

What is the actual behavior?
What did you see instead?

web_1             | Distribution distro configuration failed
web_1             | Traceback (most recent call last):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/_load.py",
line 40, in _load_distro
web_1             |     distro = entry_point.load()()
web_1             |              ^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2471, in load
web_1             |     return self.resolve()
web_1             | ^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2477, in resolve
web_1             |     module = __import__(self.module_name,
fromlist=['__name__'], level=0)
web_1             |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/distro/__init__.py",
line 22, in <module>
web_1             |     from opentelemetry.sdk._configuration import
_OTelSDKConfigurator
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_configuration/__init__.py",
line 36, in <module>
web_1             |     from opentelemetry.sdk._logs import
LoggerProvider, LoggingHandler
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/__init__.py",
line 16, in <module>
web_1             |     from opentelemetry.sdk._logs._internal import (
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 579, in <module>
web_1             |     class LoggerProvider(APILoggerProvider):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 582, in LoggerProvider
web_1             |     resource: Resource = Resource.create(),
web_1             |                          ^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/resources/__init__.py",
line 185, in create
web_1             |     next(
web_1             | StopIteration
web_1             | Failed to auto initialize opentelemetry
web_1             | Traceback (most recent call last):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py",
line 36, in initialize
web_1             |     distro = _load_distro()
web_1             |              ^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/_load.py",
line 55, in _load_distro
web_1             |     raise exc
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/auto_instrumentation/_load.py",
line 40, in _load_distro
web_1             |     distro = entry_point.load()()
web_1             |              ^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2471, in load
web_1             |     return self.resolve()
web_1             |            ^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 2477, in resolve
web_1             |     module = __import__(self.module_name,
fromlist=['__name__'], level=0)
web_1             |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/distro/__init__.py",
line 22, in <module>
web_1             |     from opentelemetry.sdk._configuration import
_OTelSDKConfigurator
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_configuration/__init__.py",
line 36, in <module>
web_1             |     from opentelemetry.sdk._logs import
LoggerProvider, LoggingHandler
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/__init__.py",
line 16, in <module>
web_1             |     from opentelemetry.sdk._logs._internal import (
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 579, in <module>
web_1             |     class LoggerProvider(APILoggerProvider):
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/__init__.py",
line 582, in LoggerProvider
web_1             |     resource: Resource = Resource.create(),
web_1             |                          ^^^^^^^^^^^^^^^^^
web_1             |   File
"/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/resources/__init__.py",
line 185, in create
web_1             |     next(
web_1             | StopIteration
web_1             |  * Debug mode: on
web_1             | WARNING: This is a development server. Do not use it
in a production deployment. Use a production WSGI server instead.
web_1             |  * Running on all addresses (0.0.0.0)
web_1             |  * Running on http://127.0.0.1:4000/
web_1             |  * Running on http://192.168.48.3:4000/
web_1             | Press CTRL+C to quit
web_1             |  * Restarting with stat
web_1             |  * Debugger is active!
web_1             |  * Debugger PIN: 102-228-721
web_1             | 192.168.48.1 - - [26/Mar/2024 07:50:59] "GET /
HTTP/1.1" 200 -

Additional context
Add any other context about the problem here.

[abstractOwl]

I did a little digging and I think I might see what's going on:

Looking at the code, the auto-instrumentation looks for resource detectors registered under the "opentelemetry_resource_detector" namespace:
https://github.com/open-telemetry/opentelemetry-python/blob/721beb8b530e7a830c1e27b70c2fb9af6465baf1/opentelemetry-sdk/src/opentelemetry/sdk/resources/__init__.py#L191-L194

I see that the Azure resource detectors have been registered using that entry point here:

opentelemetry-python-contrib/resource/opentelemetry-resource-detector-azure/pyproject.toml

Lines 30 to 32 in 24eadcf

	[project.entry-points.opentelemetry_resource_detector]
	azure_app_service = "opentelemetry.resource.detector.azure.app_service:AzureAppServiceResourceDetector"
	azure_vm = "opentelemetry.resource.detector.azure.vm:AzureVMResourceDetector"

So a corresponding change would need to be made to the sdk-aws pyproject.toml file here:

opentelemetry-python-contrib/sdk-extension/opentelemetry-sdk-extension-aws/pyproject.toml

Lines 29 to 32 in 24eadcf

	[project.entry-points.opentelemetry_id_generator]
	xray = "opentelemetry.sdk.extension.aws.trace.aws_xray_id_generator:AwsXRayIdGenerator"

I was able to confirm this by adding the following line to my Dockerfile, which writes the expected section to the package entrypoints:

RUN echo "\n\n[opentelemetry_resource_detector]\naws_elastic_beanstalk =
opentelemetry.sdk.extension.aws.resource.beanstalk:AwsBeanstalkResourceDetector"
 >>
/usr/local/lib/python3.12/site-packages/opentelemetry_sdk_extension_aws-2.0.1.dist-info/entry_points.txt

Using a fake /var/elasticbeanstalk/xray/environment.conf file, I was able to get this output in my Docker logs:

web_1             | {
web_1             |     "resource_metrics": [
web_1             |         {
web_1             |             "resource": {
web_1             |                 "attributes": {
web_1             |                     "telemetry.sdk.language": "python",
web_1             |                     "telemetry.sdk.name":
"opentelemetry",
web_1             |                     "telemetry.sdk.version": "1.20.0",
web_1             |                     "cloud.provider": "aws",
web_1             |                     "cloud.platform":
"aws_elastic_beanstalk",
web_1             |                     "service.name": "otel_test",
web_1             |                     "service.instance.id": "abc123",
web_1             |                     "service.namespace": "12345",
web_1             |                     "service.version": "0.0.1",
web_1             |                     "telemetry.auto.version": "0.41b0"
web_1             |                 },
web_1             |                 "schema_url": ""
web_1             |             },
web_1             |             "scope_metrics": [],
web_1             |             "schema_url": ""
web_1             |         }
web_1             |     ]
web_1             | }