Disable Host.Results logs at the function level for HttpTrigger functions

[CrazyTuna]

What problem would the feature you're requesting solve? Please describe.

I have some function apps exposed behind Front Door. Each application has an health-check endpoint (HttpTrigger).
Front door is calling these health endpoints quite a lot and I have a lots of requests logged to app insights

I managed to disable function logs like that:

{
"logging": {
    "logLevel": {
      "Function.health-check": "Warning"
    }
  }
}

But the requests (Host.Results category) are still logged to app insights which increase significantly app insights billing.

Describe the solution you'd like

I'd like to be able to specify Host.Results logs at the function level, so I can do something like that:

{
"logging": {
    "logLevel": {
      "Host.Results.health-check": "None"
    }
  }
}

Describe alternatives you've considered

• I've already reconfigured Azure Front Door to only call the health endpoint once per second.

• I've tried to hook up my own ITelemetryProcessor to filter some Host.Results logs as describe here: ITelemetryProcessor does not appear to be supported in Function Apps

  public void Process(ITelemetry item)
  {
      // Exclude Health check
      var request = item as RequestTelemetry;
      if (request != null)
      {
          if (request != null && request.Name == "health-check")
          {
              return;
          }
      }
      Next.Process(item);
  }
  

  While debugging, I could find any RequestTelemetry and inspecting the ITelemetry item there was no item with category Host.Results so I was a little bit stuck on that part.

[CrazyTuna]

In the host.json, if I set this property to false (enableHttpTriggerExtendedInfoCollection), I can see the RequestTelemetry coming to my ITelemetryProcessor :

"logging": {
  "applicationInsights": {
    "httpAutoCollectionOptions": {
      "enableResponseHeaderInjection": false,
      "enableW3CDistributedTracing": false,
      "enableHttpTriggerExtendedInfoCollection": true
    }
  }
}

This is an acceptable workaround for the moment but I am loosing some good information in app insights.

[brettsam]

Making the results more specific is a good idea.

But your ITelemetryProcessor should have worked. Can you tell it's getting hit? How were you checking for the Host.Results category?

[CrazyTuna]

The ITelemetryProcessor is triggered but there is no item of type RequestTelemetry. Also was checking the Category property but nothing relevant.

[DarinMacRae]

@CrazyTuna Possibly?

Just a hunch here but I ran into something similar.

HTH.

Check out...

#3741 (comment)

So whatever version of AI is used here:
https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Logging.ApplicationInsights

Is the one you have to use as well so the types match up.

[cataggar]

I've been trying to figure out a workaround to filter out the logs for the healthcheck function using OpenTelemetry so that they are not exported to Azure Monitor, but haven't got it working yet.

open-telemetry/opentelemetry-dotnet#3918

It would be much better if the Azure Function Host allowed certain functions to be disable from Host.Results.

[cataggar]

The heartbeat continues to fill up our logs with heartbeat entries. It would be awesome if this was configurable.

[cataggar]

I think the relevant code doing the logging is in FunctionExecutor.

It has a _resultsLogger defined with:

_resultsLogger = _loggerFactory.CreateLogger(LogCategories.Results);

Where LogCategories.Results is "Host.Results".

The call to _resultsLogger?.LogFunctionResult(instanceLogEntry); looks to be creating the log entry.

Would it be possible to add a filter there, such as Azure/azure-webjobs-sdk#2977 ? Is that a good way to do it. I do not yet understand how or where this project is using FunctionExecutor.

[cataggar]

@RohitRanjanMS, I see recent logging work from you in Azure/azure-webjobs-sdk#2946. Could you review this issue please?

[cataggar]

I was able to work-a-round the problem by disabling the Host.Results in the host.json:

{
    "version": "2.0",
    "functionTimeout": "00:10:00",
    "logging": {
      "logLevel": {
        "Host.Results": "None",
        "Function.Heartbeat": "None"
      }
    }
}

And then starting an Activity of ActivityKind.Server for each function that I wanted to trace. That was all functions except our Heartbeat function. I then use Azure.Monitor.OpenTelemetry.Exporter to send those traces.

[cataggar]

dotnet/runtime#82465 would potentially allow creating a filter.

[rwb196884]

Same problem here: ApplicationInsights is being spammed with thousands of useless messages from the health check.

It seems that you have to turn off Host.Results for everything.

Is this is a scam by MS to generate revenue for ingestion?

[JackD111]

@rwb196884 I've created a package to work around this: https://github.com/JackD111/azure-functions-custom-http-telemetry
I had to come up with this solution beause our .NET 8 azure function with front door was generating to much traffic

Note: This solution has been designed for Isolated Azure Functions, not in proc ones

[rwb196884]

@JackD111
I've got a csproj of Azure Functions that goes to an Azure Function App that is configured as NET 8 isolated.

However, when I try to add the object ignore parameter the build fails saying:

The attribute 'HttpTelemetryProcessorAttribute' is a WebJobs attribute and not supported in the .NET Worker (Isolated Process).

So is this actually for in process functions and not for isolated ones?

[JackD111]

@rwb196884 Can you share the csproj of your isolate azure function project?

[rwb196884]

Here it is (had to change to .txt to allow it to be attached).
Functions.txt

[JackD111]

@rwb196884 You need to include Worker.Extensions.HttpTelemetryProcessor instead of WebJobs.Extensions.HttpTelemetryProcessor as per the instructions in the usage description :P

[rwb196884]

That's it, ta.

(Might help to update the link to NuGet from https://github.com/JackD111/azure-functions-custom-http-telemetry and/or give the name of the NuGet package(s).)

[JackD111]

@rwb196884 WebJobs.Extensions.HttpTelemetry is only mentioned in the How Does It Work?
Which part of the README.md pointed you to the WebJobs.Extensions.HttpTelemetryProcessor package?

[rwb196884]

I was trying to find what package it's in and clicked the link with anchor text NuGet near the bottom of the page.