Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuration of metric Views in distro #35932

Merged
merged 5 commits into from
Jun 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions sdk/monitor/azure-monitor-opentelemetry/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

### Features Added

- Enable views configuration
([#35932](https://github.com/Azure/azure-sdk-for-python/pull/35932))
- Rework autoinstrumentation: Configure exporters and samplers directly
([#35890](https://github.com/Azure/azure-sdk-for-python/pull/35890))

Expand Down
2 changes: 2 additions & 0 deletions sdk/monitor/azure-monitor-opentelemetry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ You can use `configure_azure_monitor` to set up instrumentation for your app to
| `instrumentation_options` | A nested dictionary that determines which instrumentations to enable or disable. Instrumentations are referred to by their [Library Names](#officially-supported-instrumentations). For example, `{"azure_sdk": {"enabled": False}, "flask": {"enabled": False}, "django": {"enabled": True}}` will disable Azure Core Tracing and the Flask instrumentation but leave Django and the other default instrumentations enabled. The `OTEL_PYTHON_DISABLED_INSTRUMENTATIONS` environment variable explained below can also be used to disable instrumentations. | `N/A` |
| `resource` | Specifies the OpenTelemetry [Resource][ot_spec_resource] associated with your application. Passed in [Resource Attributes][ot_spec_resource_attributes] take priority over default attributes and those from [Resource Detectors][ot_python_resource_detectors]. | [OTEL_SERVICE_NAME][ot_spec_service_name], [OTEL_RESOURCE_ATTRIBUTES][ot_spec_resource_attributes], [OTEL_EXPERIMENTAL_RESOURCE_DETECTORS][ot_python_resource_detectors] |
| `span_processors` | A list of [span processors][ot_span_processor] that will perform processing on each of your spans before they are exported. Useful for filtering/modifying telemetry. | `N/A` |
| `views` | A list of [views][ot_view] that will be used to customize metrics exported by the SDK. | `N/A` |

You can configure further with [OpenTelemetry environment variables][ot_env_vars].

Expand Down Expand Up @@ -224,6 +225,7 @@ contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additio
[ot_sdk_python]: https://github.com/open-telemetry/opentelemetry-python
[ot_sdk_python_metric_reader]: https://opentelemetry-python.readthedocs.io/en/stable/sdk/metrics.export.html#opentelemetry.sdk.metrics.export.MetricReader
[ot_span_processor]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#span-processor
[ot_view]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#view
[ot_sdk_python_view_examples]: https://github.com/open-telemetry/opentelemetry-python/tree/main/docs/examples/metrics/views
[ot_instrumentation_django]: https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-django
[ot_instrumentation_django_version]: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation/opentelemetry-instrumentation-django/src/opentelemetry/instrumentation/django/package.py#L16
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# license information.
# --------------------------------------------------------------------------
from logging import getLogger
from typing import Dict, cast
from typing import Dict, List, cast

from opentelemetry._logs import set_logger_provider
from opentelemetry.instrumentation.dependencies import (
Expand All @@ -18,6 +18,7 @@
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.metrics.view import View
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
Expand All @@ -37,6 +38,7 @@
RESOURCE_ARG,
SAMPLING_RATIO_ARG,
SPAN_PROCESSORS_ARG,
VIEWS_ARG,
)
from azure.monitor.opentelemetry._types import ConfigurationValue
from azure.monitor.opentelemetry.exporter._quickpulse import enable_live_metrics # pylint: disable=import-error,no-name-in-module
Expand Down Expand Up @@ -88,6 +90,8 @@ def configure_azure_monitor(**kwargs) -> None: # pylint: disable=C4758
Defaults to `False`.
:keyword str storage_directory: Storage directory in which to store retry files. Defaults to
`<tempfile.gettempdir()>/Microsoft/AzureMonitor/opentelemetry-python-<your-instrumentation-key>`.
:keyword list[~opentelemetry.sdk.metrics.view.View] views: List of `View` objects to configure and filter
metric output.
:rtype: None
"""

Expand Down Expand Up @@ -163,11 +167,13 @@ def _setup_logging(configurations: Dict[str, ConfigurationValue]):

def _setup_metrics(configurations: Dict[str, ConfigurationValue]):
resource: Resource = configurations[RESOURCE_ARG] # type: ignore
views: List[View] = configurations[VIEWS_ARG] # type: ignore
metric_exporter = AzureMonitorMetricExporter(**configurations)
reader = PeriodicExportingMetricReader(metric_exporter)
meter_provider = MeterProvider(
metric_readers=[reader],
resource=resource
resource=resource,
views=views,
)
set_meter_provider(meter_provider)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
RESOURCE_ARG = "resource"
SAMPLING_RATIO_ARG = "sampling_ratio"
SPAN_PROCESSORS_ARG = "span_processors"
VIEWS_ARG = "views"


# --------------------Autoinstrumentation Configuration------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
RESOURCE_ARG,
SAMPLING_RATIO_ARG,
SPAN_PROCESSORS_ARG,
VIEWS_ARG,
)
from azure.monitor.opentelemetry._types import ConfigurationValue
from azure.monitor.opentelemetry._version import VERSION
Expand Down Expand Up @@ -70,6 +71,7 @@ def _get_configurations(**kwargs) -> Dict[str, ConfigurationValue]:
_default_instrumentation_options(configurations)
_default_span_processors(configurations)
_default_enable_live_metrics(configurations)
_default_views(configurations)

return configurations

Expand Down Expand Up @@ -155,6 +157,10 @@ def _default_enable_live_metrics(configurations):
configurations.setdefault(ENABLE_LIVE_METRICS_ARG, False)


def _default_views(configurations):
configurations.setdefault(VIEWS_ARG, [])
jeremydvoss marked this conversation as resolved.
Show resolved Hide resolved


def _get_otel_disabled_instrumentations():
disabled_instrumentation = environ.get(
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS, ""
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry import metrics
from opentelemetry.sdk.metrics import Counter
from opentelemetry.sdk.metrics.view import View

# Create a view matching the counter instrument `my.counter`
# and configure the new name `my.counter.total` for the result metrics stream
change_metric_name_view = View(
instrument_type=Counter,
instrument_name="my.counter",
name="my.counter.total",
)

# Configure Azure monitor collection telemetry pipeline
configure_azure_monitor(
views=[
change_metric_name_view, # Pass in created View into configuration
]
)

meter = metrics.get_meter_provider().get_meter("view-name-change")
my_counter = meter.create_counter("my.counter")
my_counter.add(100)


input()
Original file line number Diff line number Diff line change
Expand Up @@ -409,11 +409,56 @@ def test_setup_metrics(
configurations = {
"connection_string": "test_cs",
"resource": TEST_RESOURCE,
"views": [],
jeremydvoss marked this conversation as resolved.
Show resolved Hide resolved
}
_setup_metrics(configurations)
mp_mock.assert_called_once_with(
metric_readers=[reader_init_mock],
resource=TEST_RESOURCE
resource=TEST_RESOURCE,
views=[],
)
set_meter_provider_mock.assert_called_once_with(mp_init_mock)
metric_exporter_mock.assert_called_once_with(**configurations)
reader_mock.assert_called_once_with(metric_exp_init_mock)

@patch(
"azure.monitor.opentelemetry._configure.PeriodicExportingMetricReader",
)
@patch(
"azure.monitor.opentelemetry._configure.AzureMonitorMetricExporter",
)
@patch(
"azure.monitor.opentelemetry._configure.set_meter_provider",
)
@patch(
"azure.monitor.opentelemetry._configure.MeterProvider",
autospec=True,
)
def test_setup_metrics_views(
self,
mp_mock,
set_meter_provider_mock,
metric_exporter_mock,
reader_mock,
):
mp_init_mock = Mock()
mp_mock.return_value = mp_init_mock
metric_exp_init_mock = Mock()
metric_exporter_mock.return_value = metric_exp_init_mock
reader_init_mock = Mock()
reader_mock.return_value = reader_init_mock
view_mock = Mock()

configurations = {
"connection_string": "test_cs",
"resource": TEST_RESOURCE,
"views": [view_mock],
}
_setup_metrics(configurations)
mp_mock.assert_called_once_with(
metric_readers=[reader_init_mock],
resource=TEST_RESOURCE,
views=[view_mock],
)
set_meter_provider_mock.assert_called_once_with(mp_init_mock)
metric_exporter_mock.assert_called_once_with(**configurations)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
from opentelemetry.sdk.environment_variables import OTEL_EXPERIMENTAL_RESOURCE_DETECTORS
from opentelemetry.sdk.resources import Resource, Attributes

from azure.monitor.opentelemetry._version import VERSION


TEST_DEFAULT_RESOURCE = Resource({
"test.attributes.1": "test_value_1",
Expand All @@ -55,31 +57,47 @@ def test_get_configurations(self, resource_create_mock):
configurations = _get_configurations(
connection_string="test_cs",
credential="test_credential",
resource=TEST_CUSTOM_RESOURCE
resource=TEST_CUSTOM_RESOURCE,
storage_directory="test_directory",
sampling_ratio=0.5,
instrumentation_options={
"flask": {
"enabled": False,
}
},
enable_live_metrics=True,
views=["test_view"],
logger_name="test_logger",
span_processors=["test_processor"],
)

self.assertEqual(configurations["connection_string"], "test_cs")
self.assertEqual(configurations["distro_version"], VERSION)
self.assertEqual(configurations["disable_logging"], False)
self.assertEqual(configurations["disable_metrics"], False)
self.assertEqual(configurations["disable_tracing"], False)
self.assertEqual(configurations["resource"].attributes, TEST_MERGED_RESOURCE.attributes)
self.assertEqual(environ[OTEL_EXPERIMENTAL_RESOURCE_DETECTORS], "azure_app_service,azure_vm")
resource_create_mock.assert_called_once_with(TEST_CUSTOM_RESOURCE.attributes)
self.assertEqual(configurations["sampling_ratio"], 1.0)
self.assertEqual(configurations["credential"], ("test_credential"))
self.assertEqual(configurations["credential"], "test_credential")
self.assertEqual(configurations["instrumentation_options"], {
"azure_sdk" : {"enabled": True},
"django": {"enabled": True},
"fastapi": {"enabled": True},
"flask": {"enabled": True},
"flask": {"enabled": False},
"psycopg2": {"enabled": True},
"requests": {"enabled": True},
"urllib": {"enabled": True},
"urllib3": {"enabled": True},
"previewlib1": {"enabled": False},
"previewlib2": {"enabled": False},
})
self.assertTrue("storage_directory" not in configurations)
self.assertEqual(configurations["storage_directory"], "test_directory")
self.assertEqual(configurations["enable_live_metrics"], True)
self.assertEqual(configurations["views"], ["test_view"])
self.assertEqual(configurations["logger_name"], "test_logger")
self.assertEqual(configurations["span_processors"], ["test_processor"])

@patch.dict("os.environ", {}, clear=True)
@patch("opentelemetry.sdk.resources.Resource.create", return_value=TEST_DEFAULT_RESOURCE)
Expand Down Expand Up @@ -107,11 +125,14 @@ def test_get_configurations_defaults(self, resource_create_mock):
self.assertTrue("credential" not in configurations)
self.assertTrue("storage_directory" not in configurations)
self.assertEqual(configurations["enable_live_metrics"], False)
self.assertEqual(configurations["logger_name"], "")
self.assertEqual(configurations["span_processors"], [])
self.assertEqual(configurations["views"], [])

@patch.dict(
"os.environ",
{
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS: "flask , requests,fastapi,azure_sdk",
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS: "flask,requests,fastapi,azure_sdk",
SAMPLING_RATIO_ENV_VAR: "0.5",
OTEL_TRACES_EXPORTER: "None",
OTEL_LOGS_EXPORTER: "none",
Expand Down