[Routing] Significant performance regression when using a simple MapControllerRoute instead of MapGet on a bare-bones/starter .net 6 app (running on Azure App Service)

[samcic]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Consider this bare-bones .net 6 app (Github repo linked below):

// Program.cs

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
var app = builder.Build();

// Option 1: "Fast"
// app.MapGet("/home/content", () => "Hello World!");

// Option 2: "Slow"
// app.MapControllerRoute("default", "{controller}/{action}");

app.Run();

// HomeController.cs

using Microsoft.AspNetCore.Mvc;
namespace WebApplication27.Controllers;

public class HomeController : Controller
{
    public ContentResult Content()
    {
        return Content("hello world");
    }
}

If I deploy this app to Azure App Service with "Option 1 (fast)" uncommented (i.e. using MapGet) and use the App Service Metrics blade to measure the average response time for sequential requests to /home/content (with a 100ms wait between each request) over a time span of many minutes, the average response time is of the order of just a few milliseconds 👍 .

If I then redeploy with nothing changed except for uncommenting "Option 2 (slow)" instead (i.e. using MapControllerRoute), the average response time becomes of the order of 20 milliseconds ☹️ .

Is this "by design"? That is, is this overhead of using this standard MapControllerRoute expected? Considering how simple this app is, I was quite surprised to see how much the average response time (for every request) increased when using Option 2, i.e. when invoking the controller-routing middleware.

Is there anything that could be done for this bare-bones app to improve the performance of this MapControllerRoute option?

Some related notes for context:

• I have a much more complex .net core app deployed to App Service that interacts with various dependencies (e.g. database) and has an average production response time of between 40-50ms. The same test based on a copy of our production app (not included here) showed the same routing performance hit. That is, this routing performance hit of ~20ms for every requested controller action seems to be responsible for a significant portion of my average production response time. If there were a way to eliminate this ~20ms routing cost (while still using controllers) we could significantly optimize our response time.

• On my local dev machine (a high-end ThinkPad X1), the average response time was 0.750ms for the MapGet case and 1.250ms for the MapControllerRoute case (i.e. not quite as pronounced). The metrics shown above were from an App Service Plan at level S1, which is a "standard" Azure App Service plan level for production workloads.

• Using MapDynamicControllerRoute has similar performance to MapControllerRoute.

• I tried both 32-bit and 64-bit App Service options (same results).

• I also tried a .net 5 app with the same configuration (same results).

Expected Behavior

Although we of course understand that controller-based routing is more sophisticated, we would expect a simple app like this (i.e. a single controller) to not have such a performance regression for every request when using MapControllerRoute instead of MapGet.

Steps To Reproduce

Minimal repo project:

https://github.com/samcic/WebApplication27

Exceptions (if any)

N/A

.NET Version

6.0.101

Anything else?

Azure App Service Test Environment:
App Service Plan Level: S1

IDE: VS2022 17.0.4

> dotnet --info

.NET SDK (reflecting any global.json):
 Version:   6.0.101
 Commit:    ef49f6213a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\6.0.101\

Host (useful for support):
  Version: 6.0.1
  Commit:  3a25a7f1cc

.NET SDKs installed:
  2.1.701 [C:\Program Files\dotnet\sdk]
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.2.207 [C:\Program Files\dotnet\sdk]
  2.2.301 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]
  3.1.403 [C:\Program Files\dotnet\sdk]
  5.0.300 [C:\Program Files\dotnet\sdk]
  6.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

[davidfowl]

The mvc pipeline is quite a bit heavier and more feature rich than the routing pipeline. There are things that can be improved in completely trivial examples but not bigger ones that support the extensibility that max controllers offer

[samcic]

@davidfowl Many thanks for your comments!

Just to make sure I'm understanding this fully: As soon as a developer opts for using a controller, even a trivial one, he/she simply has to accept this base performance hit and has no possibility to reduce it even partially, correct? For example, is there not some option to reduce the "heaviness" of the pipeline by overriding some configuration options, or even replacing the MapControllerRoute call with a custom middleware component that mimics the default one but essentially omits the "heavy" stuff that one knows one doesn't need?

Thanks for contacting us.

We're moving this issue to the .NET 7 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[sebastienros]

Just to be sure I am doing the exact same repro can you confirm you are using https/h2 or plain http.

[samcic]

@sebastienros yes it was https with h2 (note my app service config screenshot above). Let me know if you have any difficulty reproducing this!

[davidfowl]

@samcic Can you reproduce this gap locally or only in app service?

[samcic]

@davidfowl not to the same extent, no. See my comment on this in the original description above. It might however just be to do with the fact that I have a high end local machine.

If this were an App Service only issue, do you have an idea for what could explain why it only happens there? From what I understand, the file system on App Service is backed by Azure Storage or something, so this is a random idea but could it be trying to load some assembly or something from the file system dynamically, on every request?

App Service is of course somewhat sandboxed so I'm not quite sure how one could proceed to isolate what's happening there. I have an Azure standard support account if you think it'd be useful to create a ticket (I'm not aware of any other way to report such issues on App Service).

[davidfowl]

If this were an App Service only issue, do you have an idea for what could explain why it only happens there? From what I understand, the file system on App Service is backed by Azure Storage or something, so this is a random idea but could it be trying to load some assembly or something from the file system dynamically, on every request?

Every request? No.

If this is an app service only issue then at least we can spend more time looking there for the differences.

[sebastienros]

I can't reproduce the problem. I used your application, but renamed the two endpoints /home/content1 and /home/content2 to have them enabled at the same time.

Tried locally with http 1.1 and 2.0, and also on Azure with the S1 plan in WestUS2, on Windows. Used bombardier for 5s and 1 connection to measure latency only.

I could measure the same latencies using Edge.

.\bombardier.exe -c 1 -l -k https://localhost:7170/home/content1 -d 5s

  Reqs/sec      8853.32     814.13   10499.84
  Latency      111.51us    63.72us    10.96ms

.\bombardier.exe -c 1 -l -k https://localhost:7170/home/content2 -d 5s	

  Reqs/sec      8611.50     671.55    9500.62
  Latency      114.80us    66.42us    12.02ms

.\bombardier.exe -c 1 -l -k https://localhost:7170/home/content1 -d 5s --http2

  Reqs/sec      5369.61     404.11    6000.15
  Latency      184.83us    79.08us    11.00ms

.\bombardier.exe -c 1 -l -k https://localhost:7170/home/content2 -d 5s --http2

  Reqs/sec      4889.42     410.94    5449.95
  Latency      203.12us   133.53us    16.00ms

.\bombardier.exe -c 1 -l -k http://gh39250.azurewebsites.net/home/content1 -d 5s --http1

  Reqs/sec        43.64      21.72     336.07
  Latency       23.48ms    10.54ms   157.95ms

.\bombardier.exe -c 1 -l -k http://gh39250.azurewebsites.net/home/content2 -d 5s --http1

  Reqs/sec        42.29      10.41     100.24
  Latency       23.62ms     9.72ms   119.73ms

.\bombardier.exe -c 1 -l -k https://gh39250.azurewebsites.net/home/content1 -d 5s --http2

  Reqs/sec        43.73      12.20     105.17
  Latency       22.84ms    11.41ms   175.00ms

.\bombardier.exe -c 1 -l -k https://gh39250.azurewebsites.net/home/content2 -d 5s --http2

  Reqs/sec        42.51      13.67     103.26
  Latency       23.49ms    12.65ms   165.01ms

[sebastienros]

Can close this issue? It's been almost a year since the last feedback and we couldn't reproduce the problem by replicating the description.