Skip to content

Commit f1eec45

Browse files
muthu-mpsmuthu-mps
authored
[Azure AI Foundry] Add native log category (#14410)
* Add audit and request response logs
1 parent a89bc5a commit f1eec45

File tree

9 files changed

+160
-9
lines changed

9 files changed

+160
-9
lines changed

‎packages/azure_ai_foundry/_dev/build/docs/README.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,18 @@ These are the supported Azure log categories:
1212

1313
| Data Stream | Log Category |
1414
|:-----------:|:------------------------:|
15+
| logs | Audit |
16+
| logs | RequestResponse |
1517
| logs | ApiManagementGatewayLogs |
1618

1719
#### Requirements and setup
1820

1921
Refer to the [Azure Logs](https://docs.elastic.co/integrations/azure) page for more information on how to set up and use this integration.
2022

23+
#### Native Logging
24+
25+
The Azure AI Foundry provides native logging and monitoring to track the telemetry of the service. The Audit and RequestResponse log categories come under the native logging. However, the default logging doesn't log the inputs and outputs of the service. This is useful to ensure that the services operates as expected.
26+
2127
#### API Gateway Logs
2228

2329
The API Management services provide the advanced logging capabilities. The `ApiManagementGatewayLogs` category comes under the advanced logging. This is not directly available in the Azure AI Foundry service itself. You have to set up the API Management services in Azure to access the Azure AI Foundry models. When the setup is complete, add the diagnostic setting for the API Management service.
@@ -44,7 +50,7 @@ The metrics data stream collects the cognitive service metrics that is specific
4450
**Model HTTP Request Metrics:**
4551
- `Requests`: Total number of calls made to the model API over a period of time.
4652

47-
**Model HTTP Request Metrics:**
53+
**Model HTTP Latency Metrics:**
4854
- `Latency`: Measures time taken to process the first byte of response, last byte of response and the request latency.
4955

5056
**Model Usage Metrics:**
@@ -71,7 +77,7 @@ If no resource filter is specified, then all Azure AI Foundry services inside th
7177

7278
The primary aggregation value will be retrieved for all the metrics contained in the namespaces. The aggregation options are `avg`, `sum`, `min`, `max`, `total`, `count`.
7379

74-
#### Logs Reference
80+
### Logs Reference
7581

7682
{{event "logs"}}
7783

@@ -81,7 +87,7 @@ For more details on ECS fields, check the [ECS Field Reference](https://www.elas
8187

8288
{{fields "logs"}}
8389

84-
#### Metrics Reference
90+
### Metrics Reference
8591

8692
The Azure AI Foundry metrics provide insights into the performance and usage of your AI resources. These metrics help in monitoring and optimizing your deployments.
8793

‎packages/azure_ai_foundry/changelog.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
- version: "0.4.0"
2+
changes:
3+
- description: Add native Audit and RequestResponse logs.
4+
type: enhancement
5+
link: https://github.com/elastic/integrations/pull/14410
16
- version: "0.3.0"
27
changes:
38
- description: Add billing dashboard to the Integration.

‎packages/azure_ai_foundry/data_stream/logs/_dev/test/pipeline/test-ai-foundry-audit.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"AssetIdentity":"07628fea-67bb-424d-b160-fdc46c82d0b4","Tenant":"eastus","category":"Audit","operationName":"ListKey","properties":{"objectId":"b610ab05-ce06-4cc1-a6dd-174b9f80468a"},"resourceId":"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/OBS-AIFOUNDRY-SERVICE-RS/PROVIDERS/MICROSOFT.COGNITIVESERVICES/ACCOUNTS/OBS-AIFOUNDRY-TEST-01","time":"2025-06-17T06:03:30.0000000Z"}

‎packages/azure_ai_foundry/data_stream/logs/_dev/test/pipeline/test-ai-foundry-audit.log-expected.json‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"expected": [
3+
{
4+
"@timestamp": "2025-06-17T06:03:30.000Z",
5+
"azure": {
6+
"ai_foundry": {
7+
"asset_identity": "07628fea-67bb-424d-b160-fdc46c82d0b4",
8+
"category": "Audit",
9+
"operation_name": "ListKey",
10+
"properties": {
11+
"object_id": "b610ab05-ce06-4cc1-a6dd-174b9f80468a"
12+
},
13+
"tenant": "eastus"
14+
},
15+
"resource": {
16+
"group": "obs-aifoundry-service-rs",
17+
"id": "/subscriptions/12cabcb4-86e8-404f-a3d2-1dc9982f45ca/resourcegroups/obs-aifoundry-service-rs/providers/microsoft.cognitiveservices/accounts/obs-aifoundry-test-01",
18+
"name": "obs-aifoundry-test-01",
19+
"provider": "microsoft.cognitiveservices/accounts"
20+
},
21+
"subscription_id": "12cabcb4-86e8-404f-a3d2-1dc9982f45ca"
22+
},
23+
"cloud": {
24+
"account": {
25+
"id": "12cabcb4-86e8-404f-a3d2-1dc9982f45ca"
26+
},
27+
"provider": "azure"
28+
},
29+
"event": {
30+
"original": "{\"AssetIdentity\":\"07628fea-67bb-424d-b160-fdc46c82d0b4\",\"Tenant\":\"eastus\",\"category\":\"Audit\",\"operationName\":\"ListKey\",\"properties\":{\"objectId\":\"b610ab05-ce06-4cc1-a6dd-174b9f80468a\"},\"resourceId\":\"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/OBS-AIFOUNDRY-SERVICE-RS/PROVIDERS/MICROSOFT.COGNITIVESERVICES/ACCOUNTS/OBS-AIFOUNDRY-TEST-01\",\"time\":\"2025-06-17T06:03:30.0000000Z\"}"
31+
},
32+
"tags": [
33+
"preserve_original_event"
34+
]
35+
}
36+
]
37+
}

‎packages/azure_ai_foundry/data_stream/logs/_dev/test/pipeline/test-ai-foundry-request-response.log‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
{"Tenant":"eastus","callerIpAddress":"81.2.69.***","category":"RequestResponse","correlationId":"9d3a6e98-fc11-48d0-82cf-4de065c1a1f8","durationMs":102,"event":"ShoeboxCallResult","location":"eastus","operationName":"Create_Thread","properties":"{\"apiName\":\"Azure AI FOUNDRY API version 2024-02-15-preview\",\"requestTime\":638481757794854611,\"requestLength\":2,\"responseTime\":638481757795877942,\"responseLength\":113,\"objectId\":\"\"}","resourceId":"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/OBS-AIFOUNDRY-SERVICE-RS/PROVIDERS/MICROSOFT.COGNITIVESERVICES/ACCOUNTS/OBS-AIFOUNDRY-TEST-01","resultSignature":"200","time":"2025-06-08T12:23:02.4350000Z"}
2+
{"Tenant":"eastus","callerIpAddress":"81.2.69.***","category":"RequestResponse","correlationId":"d427a391-2bc7-4f38-9cbc-6560600e0981","durationMs":1021,"event":"ShoeboxCallResult","location":"eastus","operationName":"ChatCompletions_Create","properties":{"apiName":"Azure AI FOUNDRY API version 2023-07-01-preview","requestTime":638515262815394150,"requestLength":128,"responseTime":638515262825606520,"responseLength":1411,"objectId":"","streamType":"Non-Streaming","modelDeploymentName":"gpt-chat-pilot","modelName":"gpt-35-turbo","modelVersion":"0301"},"resourceId":"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/OBS-AIFOUNDRY-SERVICE-RS/PROVIDERS/MICROSOFT.COGNITIVESERVICES/ACCOUNTS/OBS-AIFOUNDRY-TEST-01","resultSignature":"200","time":"2025-06-24T12:23:02.4350000Z"}

‎packages/azure_ai_foundry/data_stream/logs/_dev/test/pipeline/test-ai-foundry-request-response.log-expected.json‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
{
2+
"expected": [
3+
{
4+
"@timestamp": "2025-06-08T12:23:02.435Z",
5+
"azure": {
6+
"ai_foundry": {
7+
"caller_ip_address": "81.2.69.***",
8+
"category": "RequestResponse",
9+
"correlation_id": "9d3a6e98-fc11-48d0-82cf-4de065c1a1f8",
10+
"duration_ms": 102,
11+
"event": "ShoeboxCallResult",
12+
"location": "eastus",
13+
"operation_name": "Create_Thread",
14+
"properties": {
15+
"api_name": "Azure AI FOUNDRY API version 2024-02-15-preview",
16+
"request_length": 2,
17+
"request_time": 638481757794854611,
18+
"response_length": 113,
19+
"response_time": 638481757795877942
20+
},
21+
"result_signature": "200",
22+
"tenant": "eastus"
23+
},
24+
"resource": {
25+
"group": "obs-aifoundry-service-rs",
26+
"id": "/subscriptions/12cabcb4-86e8-404f-a3d2-1dc9982f45ca/resourcegroups/obs-aifoundry-service-rs/providers/microsoft.cognitiveservices/accounts/obs-aifoundry-test-01",
27+
"name": "obs-aifoundry-test-01",
28+
"provider": "microsoft.cognitiveservices/accounts"
29+
},
30+
"subscription_id": "12cabcb4-86e8-404f-a3d2-1dc9982f45ca"
31+
},
32+
"cloud": {
33+
"account": {
34+
"id": "12cabcb4-86e8-404f-a3d2-1dc9982f45ca"
35+
},
36+
"provider": "azure"
37+
},
38+
"event": {
39+
"duration": 102000000,
40+
"original": "{\"Tenant\":\"eastus\",\"callerIpAddress\":\"81.2.69.***\",\"category\":\"RequestResponse\",\"correlationId\":\"9d3a6e98-fc11-48d0-82cf-4de065c1a1f8\",\"durationMs\":102,\"event\":\"ShoeboxCallResult\",\"location\":\"eastus\",\"operationName\":\"Create_Thread\",\"properties\":\"{\\\"apiName\\\":\\\"Azure AI FOUNDRY API version 2024-02-15-preview\\\",\\\"requestTime\\\":638481757794854611,\\\"requestLength\\\":2,\\\"responseTime\\\":638481757795877942,\\\"responseLength\\\":113,\\\"objectId\\\":\\\"\\\"}\",\"resourceId\":\"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/OBS-AIFOUNDRY-SERVICE-RS/PROVIDERS/MICROSOFT.COGNITIVESERVICES/ACCOUNTS/OBS-AIFOUNDRY-TEST-01\",\"resultSignature\":\"200\",\"time\":\"2025-06-08T12:23:02.4350000Z\"}"
41+
},
42+
"tags": [
43+
"preserve_original_event"
44+
]
45+
},
46+
{
47+
"@timestamp": "2025-06-24T12:23:02.435Z",
48+
"azure": {
49+
"ai_foundry": {
50+
"caller_ip_address": "81.2.69.***",
51+
"category": "RequestResponse",
52+
"correlation_id": "d427a391-2bc7-4f38-9cbc-6560600e0981",
53+
"duration_ms": 1021,
54+
"event": "ShoeboxCallResult",
55+
"location": "eastus",
56+
"operation_name": "ChatCompletions_Create",
57+
"properties": {
58+
"api_name": "Azure AI FOUNDRY API version 2023-07-01-preview",
59+
"model_deployment_name": "gpt-chat-pilot",
60+
"model_name": "gpt-35-turbo",
61+
"model_version": "0301",
62+
"request_length": 128,
63+
"request_time": 638515262815394150,
64+
"response_length": 1411,
65+
"response_time": 638515262825606520,
66+
"stream_type": "Non-Streaming"
67+
},
68+
"result_signature": "200",
69+
"tenant": "eastus"
70+
},
71+
"resource": {
72+
"group": "obs-aifoundry-service-rs",
73+
"id": "/subscriptions/12cabcb4-86e8-404f-a3d2-1dc9982f45ca/resourcegroups/obs-aifoundry-service-rs/providers/microsoft.cognitiveservices/accounts/obs-aifoundry-test-01",
74+
"name": "obs-aifoundry-test-01",
75+
"provider": "microsoft.cognitiveservices/accounts"
76+
},
77+
"subscription_id": "12cabcb4-86e8-404f-a3d2-1dc9982f45ca"
78+
},
79+
"cloud": {
80+
"account": {
81+
"id": "12cabcb4-86e8-404f-a3d2-1dc9982f45ca"
82+
},
83+
"provider": "azure"
84+
},
85+
"event": {
86+
"duration": 1021000000,
87+
"original": "{\"Tenant\":\"eastus\",\"callerIpAddress\":\"81.2.69.***\",\"category\":\"RequestResponse\",\"correlationId\":\"d427a391-2bc7-4f38-9cbc-6560600e0981\",\"durationMs\":1021,\"event\":\"ShoeboxCallResult\",\"location\":\"eastus\",\"operationName\":\"ChatCompletions_Create\",\"properties\":{\"apiName\":\"Azure AI FOUNDRY API version 2023-07-01-preview\",\"requestTime\":638515262815394150,\"requestLength\":128,\"responseTime\":638515262825606520,\"responseLength\":1411,\"objectId\":\"\",\"streamType\":\"Non-Streaming\",\"modelDeploymentName\":\"gpt-chat-pilot\",\"modelName\":\"gpt-35-turbo\",\"modelVersion\":\"0301\"},\"resourceId\":\"/SUBSCRIPTIONS/12CABCB4-86E8-404F-A3D2-1DC9982F45CA/RESOURCEGROUPS/OBS-AIFOUNDRY-SERVICE-RS/PROVIDERS/MICROSOFT.COGNITIVESERVICES/ACCOUNTS/OBS-AIFOUNDRY-TEST-01\",\"resultSignature\":\"200\",\"time\":\"2025-06-24T12:23:02.4350000Z\"}"
88+
},
89+
"tags": [
90+
"preserve_original_event"
91+
]
92+
}
93+
]
94+
}

‎packages/azure_ai_foundry/data_stream/logs/elasticsearch/ingest_pipeline/default.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ processors:
2424
# GatewayLlmLogs - TODO - After Azure make it for production use.
2525
- drop:
2626
description: Drop logs other than GatewayLogs category.
27-
if: "ctx.azure.ai_foundry.category != 'GatewayLogs'"
27+
if: "ctx.azure.ai_foundry.category != 'Audit' && ctx.azure.ai_foundry.category != 'RequestResponse' && ctx.azure.ai_foundry.category != 'GatewayLogs'"
2828
ignore_failure: true
2929
- json:
3030
tag: json-extract-stringly-Properties

‎packages/azure_ai_foundry/docs/README.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,18 @@ These are the supported Azure log categories:
1212

1313
| Data Stream | Log Category |
1414
|:-----------:|:------------------------:|
15+
| logs | Audit |
16+
| logs | RequestResponse |
1517
| logs | ApiManagementGatewayLogs |
1618

1719
#### Requirements and setup
1820

1921
Refer to the [Azure Logs](https://docs.elastic.co/integrations/azure) page for more information on how to set up and use this integration.
2022

23+
#### Native Logging
24+
25+
The Azure AI Foundry provides native logging and monitoring to track the telemetry of the service. The Audit and RequestResponse log categories come under the native logging. However, the default logging doesn't log the inputs and outputs of the service. This is useful to ensure that the services operates as expected.
26+
2127
#### API Gateway Logs
2228

2329
The API Management services provide the advanced logging capabilities. The `ApiManagementGatewayLogs` category comes under the advanced logging. This is not directly available in the Azure AI Foundry service itself. You have to set up the API Management services in Azure to access the Azure AI Foundry models. When the setup is complete, add the diagnostic setting for the API Management service.
@@ -44,7 +50,7 @@ The metrics data stream collects the cognitive service metrics that is specific
4450
**Model HTTP Request Metrics:**
4551
- `Requests`: Total number of calls made to the model API over a period of time.
4652

47-
**Model HTTP Request Metrics:**
53+
**Model HTTP Latency Metrics:**
4854
- `Latency`: Measures time taken to process the first byte of response, last byte of response and the request latency.
4955

5056
**Model Usage Metrics:**
@@ -71,7 +77,7 @@ If no resource filter is specified, then all Azure AI Foundry services inside th
7177

7278
The primary aggregation value will be retrieved for all the metrics contained in the namespaces. The aggregation options are `avg`, `sum`, `min`, `max`, `total`, `count`.
7379

74-
#### Logs Reference
80+
### Logs Reference
7581

7682
An example event for `logs` looks as following:
7783

@@ -324,7 +330,7 @@ For more details on ECS fields, check the [ECS Field Reference](https://www.elas
324330
| data_stream.type | Data stream type. | constant_keyword |
325331

326332

327-
#### Metrics Reference
333+
### Metrics Reference
328334

329335
The Azure AI Foundry metrics provide insights into the performance and usage of your AI resources. These metrics help in monitoring and optimizing your deployments.
330336

‎packages/azure_ai_foundry/manifest.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
format_version: 3.3.2
22
name: azure_ai_foundry
33
title: "Azure AI Foundry"
4-
version: "0.3.0"
4+
version: "0.4.0"
55
source:
66
license: "Elastic-2.0"
77
description: "Collects Azure AI Foundry logs and metrics"
@@ -56,7 +56,7 @@ policy_templates:
5656
- cloud
5757
inputs:
5858
- type: "azure-eventhub"
59-
title: "Collect Azure OpenAI logs from Event Hub"
59+
title: "Collect Azure AI Foundry logs from Event Hub"
6060
description: "Collecting Azure AI Foundry logs from Azure (input: azure-eventhub)"
6161
input_group: logs
6262
- type: "azure/metrics"

0 commit comments

Comments
 (0)