Deprecate HttpListener #88
Comments
|
But Kestrel on Unix does not directly support the Windows authentication. aspnet/KestrelHttpServer#2168 |
|
@valeri2 Even Kestrel doesn't want to implement Windows authentication on Linux due to security concerns: aspnet/KestrelHttpServer#2168 (comment). The less reason to implement it in |
|
I would not consider |
|
@springy76 not all From .NET point of view, |
|
@karelz Wait a sec. Someone points out how being based on http.sys provides very useful functionality that they're relying on, that Kestrel can't duplicate, and your response is "meh, we're probably going to remove that anyway"? Seriously?!? Also, how is it that you regard something that drags in a dependency on ASP.Net Core as a good alternative to a simple, lightweight server class with no dependencies outside of the standard library? Sorry, but no. From the perspective of a developer actually using it, that's simply insane! |
|
@masonwheeler we need to weight in usefulness vs. maintenance cost. Is that particular feature so much more important to EVERY developer vs. other investments. Maintaining something, keeping it up to latest standards, etc. does cost something - see the cost in Kestrel. If you want to help change things, I would recommend to focus on facts, and usage scenarios / commonality (e.g. how many customers truly depend on specific feature?) Also note, that the fact we do not evolve HttpListener further, that it is compat-only API is a fact since its introduction into .NET Core 2.0. The deprecation is just clear public statement of where we are and where we have been for last year. It sets expectations. If you want something modern, it is not HttpListener. |
I chose
Yes, I understand that that's your position. I'm saying that this is a bad position and needs to be reconsidered. If there are new and better core HTTP server features available, what's stopping you from making them available to everyone even if they don't want to bloat up their project with unwanted frameworks and unnecessary IIS crud? |
|
So, late to the party here. Please consider the magic that is happening over at Ooui, which allows you to pipe the rendering of .NET elements straight into a webpage. Additionally, you can perform this same magic in a native-hosted/store scenario (UWP as an example). This is all supported at present through the use of |
|
First a little bit of history: Long time ago, If current
As I suggested above, if you want us to reconsider, we will need some hard data on usage, limitations of Kestrel, etc. So far I am aware of only 1 feature which is not better in Kestrel. Its usefulness seems to be fairly limited.
If your projects are satisfied with current limited
That sounds like a very good idea. I am sure quite a few developers would appreciate that. I hope some Kestrel experts will be able to provide such guidance. FYI: For context, check some of the discussion here: https://github.com/dotnet/designs/issues/9 (just please do not reopen the discussion there - it does not belong on the uber-issue). |
|
Cool... thanks @karelz ... I am starting to dig and do some analysis around this space, myself. In my view it is incredibly important as at a glance it totally (and finally) satisfies the request for a Ubiquitous.NET. That is, using this magic bridge/connection, you can enable a world where it's one .NET codebase and one .NET viewbase total in your solution, rendered through a WebView (as presented/demonstrated above). That is a big deal and I do not want to jeopardize it. 😄 So it does sound like at the very least we will still be able to use
So I thought that UWP supported .NET Standard 2.0? That is the |
Yes, I am not aware of any instance when we deprecate things and they immediately stop working :). The warnings are soft, easy to disable and opt-in currently - read more in this repo about the platform compat tooL; https://github.com/dotnet/platform-compat
I'd ask Kestrel folks where and how they are supported. IMO Kestrel needs to listen on ports which is not supported in UWP. Not sure if it can work without that (you have reached the limits of my knowledge). BTW: Also note, that there are currently limitations on various platforms:
|
So to add a little more context here, the |
I wonder how this is being checked? |
|
@springy76 It's probably better stated that it doesn't accept any connections from external requests, so as a result it only accepts requests/connections running from within the process from which the application is running. That is my understanding, at least. Also, while I have gotten this to work locally on UWP, it hasn't been officially verified as a deployed product. As such, I do have an outstanding question on Xamarin's forums, but if I do not get an answer there, I will have to brogrammer-up and get some HelloWorld's deployed to get a definitive answer. Unfortunately, Droid development hasn't exactly been friendly to me, and I haven't even touched iOS. If someone has some valuable knowledge around these scenarios it would be greatly appreciated if you could share. 👍 |
|
See discussion at: https://github.com/dotnet/designs/issues/9 |
|
@valeri2 , it seems to me that Kerberos (e.g. Active Directory) is one of the few reasons a developer would want to run an application on Windows vs. Linux. The Linux community by and large doesn't want to deal with Kerberos, whereas Windows has been pretty good at handling it for a long time. If Kerberos is a requirement (and tbh I would use if I could for an internal or enterprise app), you should stick with Windows. |
|
I would also like it if you could keep the HttpListener as it is and maybe evolve it as a no-frills HTTP Api/Server alongside the other ones. Seeing how many times it has been ported by other people already there seems to be plenty of demand for it. As for my own use: I like that in 3 lines of code I can listen for a connection and do almost anything with it without needing a lot of dependencies or learning a new paradigm. There are some things that can easily be made better, like the certificate format it expects. And some other things I am sure you are already aware of. In other words, I think you'd make a lot of people happy, including me, if you keep HttpListener around and Microsoft would still support it. |
|
Hello @karelz , your recommendation is not to use HttpClient and use Kestrel that has been moved into ASP.NET Core. However, I don't fully understand how to go from one to the other. I have a desktop application that listens on port 443 and replies web requests, and not full blown ASP.NET Core project with all the configuration and setup that a web server requires, frameworks, paradigms and all its baggage. Is there a way to make a console application to respond to HTTP requests? or are we adding the constraint that only ASP.NET Core projects can do that? |
|
@pablocar80 AFAIK you can write minimal Kestrel server with little overhead, baggage, etc. Check out their docs, I am not expert on the topic. |
|
The docs are at https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel?view=aspnetcore-2.2 . The minimal sample linked from the docs is not a full blown ASP.NET project, but it still carries some of the baggage with it: https://github.com/aspnet/Docs/blob/master/aspnetcore/fundamentals/servers/kestrel/samples/2.x/KestrelSample/Startup.cs#L4 cc @davidfowl |
|
Would it make sense for the DotNetCore team to offer the HttpClient class as a thin wrapper that under the hood uses Kestrel? This would help with migration A LOT. |
|
@jkotas thank you for the links. I checked out the documentation and the references needed are specific to NET Core. Ideally we should have HttpClient or its replacement compiling under NET Standard. |
|
Nothing in the docs show a minimal example. There are different levels of minimal:
The latter isn’t pretty but we could spend some cycles making a nicer API to get to the pre request callback if that was a concern. |
|
Hello @davidfowl ! If I understood correctly then option 2 is what HttpListener is for. Right now, if you have HttpListener, the Portability Analysis tool tells you that you're good to go. Then you realize that not really -- the .NET Core implementation doesn't support HTTPS which is a reasonable requirement to put a system in production. Then you look up the issue on why HTTPS isn't supported, and it's because there's this thread that says that HttpListener should be abandoned altogether. So there's really a disconnect between the happy land of the portability analyzer that tells you you're ready to jump, and the reality that HttpListener in NET Core compiles yet isn't really implemented. |
Suggestion: Spend those cycles on an API that mimics the .NET HttpListner API. How does that sound? |
|
Exposing the HttpListener API from Kestrel IMO doesn’t make any sense. It would make more sense to just “fix” HttpListener but to make it production ready would be a wasted effort (wasting cycles fixing hard networking and security issues I’m not places etc). Now you could argue that Kestrel and HttpListener could share the same underlying core but there’s tons of code in the upper layers that actually handle the protocol control flow that wouldn’t be shared. HttpListener is also a pull model and ASP.NET Core exposes a push model for handling requests. There are other layering concerns with replatting one on the other (like namespace and assembly dependencies). Even the core of kestrel is pluggable and things like TLS are plugged in via this extensibility (also multiple transports) and I’m not sure there’s an appetite to push this down into the lower levels. Layering the other way (building Kestrel on top of HttpListener) would be wasteful layering wise and it’s already possible plug server implementations into ASP.NET Core wholesale so this would be pointless IMO. All that said I’m not sure what the best call is here because we don’t have true replacement that’s built into the “core” shared framework that is a “production ready” (whatever that means here) server. That’s a bigger strategic decision that needs to be discussed. |
|
This thread, IMHO, does not mention the alternative to
If the class is deprecated, may we have either the clear alternative description or the clear statement that the class is deprecated without the alternative? My two cents: my team does not develop "web applications", the "API surface" term has no meaning for our system, for us any connection is valid, so we need (as some people who commented above) a simple "TcpListener-style" abstraction with HTTP support that can be used locally 1) without raising the functionality to the "application level", 2) without imposing any external "coding doctrines" and 3) without any other features we don't need like model validation, routing, authorization, serialization helpers, and so on. |
|
Something like this?: https://github.com/davidfowl/BasicKestrel/tree/master/BasicKestrel |
|
@davidfowl thank you for the code. This is one approach someone can take. For those who's interested but didn't do it, the other one is the creation of a single middleware class with Unfortunately, your example code depicts the issues (well, not issues per se, but the pain points for the people who need a listener-only kind of abstraction) I've been talking about:
|
Can you clarify what "still there in the dependency graph" means? I might have an idea but I'd like to understand what added "cost" you are describing.
I don't know what that means, perhaps a code sample would clear things up for me.
Same as above, show me some sample code and I'll try to provide you what what I think is the alternative. I don't think it raises the level of abstraction to the level of the "whole application", that's just the impression people have because of the history of ASP.NET (tied to IIS etc). |
|
Hello! for anyone new coming to this thread -- I can confirm that I was able to migrate everything with HttpListener to Net Standard using Kestrel some months ago. It required lots of work refactoring but it was possible. |
Yeah since we're updating everyone, I ended up dumping UWP/native altogether and am now on Blazor, which has that WebSocket magic built-in. This doesn't account for native hosted scenarios just yet but we have that identified and discussed in the Blazor repo so that's good enough for me ATM. 😁 That aside, I have to give everyone on the .NET team so much credit and respect for how .NET has shaped up as of late. The generic hosting model, EF, Blazor, Asp.NET Core, Razor Components, EVERYTHING!!! Really, truly, it's like developing back in the days of Silverlight again. You should unexpectedly and suddenly announce its unexplained deprecation and sunset for old times' sake. (No don't actually do that please). Bad jokes aside, kudos to everyone there. I keep meaning to make a blog post about it but random GitHub issue praises is all I can muster BECAUSE I AM DEVELOPING SO HARD THESE DAYS. 😅 |
/* somewhere deep inside the calculations */
if (notEnoughData)
{
try
{
var listener = new HttpListener();
SetupPrefixes(listener);
listener.Start();
await OrderNDataPackagesAsync(dataPackagesToOrder);
foreach (var i in Enumerable.Range(dataPackagesToOrder))
{
var context = await listener.GetContextAsync();
/* some processing and returning 200 OK at the end of the iteration */
}
}
finally
{
listener.Close();
}
}The code sometimes needs the additional data and it orders it via http call, then receives the required number of packages through the listener, closes the listener and proceeds. It can't get the data as responses to the requests because it's being requested from the system that is out of our control (and it can only push data to the endpoint). |
I think the default template absolutely does that but not the framework itself. There are minimal ways to use the framework that aren't documented as mainstream but exist as public API.
Right, like I said above, the docs maybe pointed you in that direction but that isn't the case.
I believe you, you could have written something in about ~20 LOC without services and dependency injection. The biggest pattern change is the push vs pull API which I agree can be hard if you were doing something like handing 4 requests to then shut the server down. Here are 2 different implementations: Push: using System.Threading;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
class Program
{
static async Task Main(string[] args)
{
var dataPackagesToOrder = 5;
var processingTask = new TaskCompletionSource<object>(TaskCreationOptions.RunContinuationsAsynchronously);
var host = new HostBuilder()
.ConfigureWebHost(webBuilder =>
{
webBuilder.UseKestrel(options => options.ListenLocalhost(8081))
.Configure(app =>
{
app.Run(async context =>
{
var remaining = Interlocked.Decrement(ref dataPackagesToOrder);
if (remaining < 0)
{
// Nothing to do here
return;
}
// Handle the request here
if (remaining == 0)
{
// Done handling the last package to order, we can shut the server down
processingTask.TrySetResult(null);
}
});
});
})
.Build();
await host.StartAsync();
await processingTask.Task;
await host.StopAsync();
}
}Pull: ```C#
using System;
using System.Threading.Channels;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
class Program
{
static async Task Main(string[] args)
{
var dataPackagesToOrder = 5;
var requests = Channel.CreateBounded<HttpContext>(dataPackagesToOrder);
using var host = new HostBuilder()
.ConfigureWebHost(webBuilder =>
{
webBuilder.UseKestrel()
.UseUrls("http://localhost:8081")
.Configure(app =>
{
app.Run(async context =>
{
// Handle the request here
var tcs = new TaskCompletionSource<object>();
context.SetCompletion(tcs);
await requests.Writer.WriteAsync(context);
await tcs.Task;
});
});
})
.Build();
await host.StartAsync();
for (int i = 0; i < dataPackagesToOrder; i++)
{
var context = await requests.Reader.ReadAsync();
// process request
// ...
// complete the request
context.CompleteRequest();
}
// Don't want to handle any more requests, don't do graceful shutdown
try
{
await host.StopAsync(TimeSpan.Zero);
}
catch (OperationCanceledException) { }
// Shut down the requests channel
requests.Writer.TryComplete();
}
}
public static class HttpContextExtensions
{
private static readonly object _completionKey = new object();
public static void SetCompletion(this HttpContext httpContext, TaskCompletionSource<object> tcs)
{
httpContext.Items[_completionKey] = tcs;
}
public static void CompleteRequest(this HttpContext httpContext)
{
if (httpContext.Items[_completionKey] is TaskCompletionSource<object> tcs)
{
tcs.TrySetResult(null);
}
}
} |
|
@davidfowl wow, looks great. I was trying to implement something similar using Do you think it would be possible in the future to see more examples like this in the official documentation? I guess I'm not the only one who had the one-sided impression of the concept. |
|
I read through this entire thread yesterday, and it has deviated quite a lot from it's original topic. I'd like to return to the original post for a moment:
Your point is well taken - I don't think performance is an issue for most of those who protested this issue, who just want to embed a lightweight HTTP server in their applications - that's likely not motivation for them to switch. That's just extra work for something that makes no practical difference to them, and it's obviously not the use-case that Kestrel was designed to satisfy. Here's what I would propose: refactor and extract a simple, low-level HTTP kernel that the ASP framework can depend on. It just needs to do what you said: support modern protocols, scale to high performance and modern requirements, and of course it needs to be maintained. This is the approach taken by most other platforms and languages, including at least Dart, Node, PHP (via PSR-7/15/17 standards) and Ruby - for example, the Rails framework is essentially middleware that can run on different servers. From an HTTP kernel, all I'd expect is request/response abstractions, a simple abstraction of header names/values, a URI abstraction, and a simple middleware delegate. Anything beyond that can be layered into middleware. That gives users of And it would probably make the .NET platform more attractive to framework authors - which seems to be an area where the .NET community isn't especially prolific, perhaps because there's a "guilty feeling" associated with abstracting away the entire ASP stack to achieve something perceptibly simpler, or because the barrier of entry is too high. There's currently no practical way around ASP if you want to build for the web with .NET, and I think this can hamper creativity. I realize there are a few micro-frameworks out there, but for the most part, these depend on the ASP stack, and rather than providing a simpler place to start for people who aren't ready to invest their time in the entire stack, these effectively become just another thing you have to learn. There's a sense of ASP being almost synonymous with .NET for web development, as there was for Rails and Ruby in it's early days - and this is not to knock on ASP or anything, but lots of people out here have a more minimalist mindset, or just different opinions about what a web framework should be like, and there is currently very little incentive for these people to get onboard and experiment with the .NET platform. Providing a lightweight HTTP kernel where multiple frameworks can coexist would elmost definitely elevate adoption among those who stick with Node, PHP or Ruby for the simple reason that it gives them more choices; even if it's just the sense of having more choices, that's a very real decision making factor for people who aren't totally sold on ASP. Just my 2¢ 😊 |
|
I agree in theory but not in practice. ASP is such a bad name it’s been an uphill battle to change the feel of that legacy but the fact is frameworks should be embedding kestrel, and building on the asp.net core primitives. It was built with that purpose in mind. I’m not opposed to having something built in that’s extremely low level, but it would need to be low level for kestrel to be built on top of it. It comes down to effort de-duplication of efforts. Kestrel supports HTTP/1/2/3 currently and doesn’t pay the cost of translating between abstractions (header copying anyone?). That low level ness goes against having nice high level abstractions that library authors can use. Another option would be to ship 2 APIs, the HTTP kernel and the abstractions or just another higher level set of APIs that ASP.NET Core doesn’t use. |
|
Another option would be to expose a .NET interface to MsQuic along with a .NET HTTP3 parsing lib and finally end this issue. https://techcommunity.microsoft.com/t5/networking-blog/msquic-is-open-source/ba-p/1345441 |
I don't think that would solve the issue for the masses, you need parsers for all the HTTPs. Also HTTP/2 is sufficiently complex that hard to do well, HTTP/3 is less hard but still non-trivial compared to HTTP/1 (which also has lots of quirks). All of that to say I think that wouldn't solve what's being asked. Likely in .NET 6, it's internal in 5.0: |
|
@davidfowl I think maybe you misunderstand me. Of course, yes, we would want the transport independence between HTTP 1/2/3 - other platforms afaik have successfully upgraded their kernels to HTTP 3 without breaking anything. What all HTTP versions have in common is headers, methods, status codes, URIs, request and response, etc. - that's as "low level" as I'm talking about. Extracting that to a separate layer/package ought to be feasible, as it is in other languages, why not? As soon as you get to the layer where, for example, all the known header types are abstracted as objects, that's where things cross over into opinionated framework territory. It's not essential to abstracting the transport layer, or to interoperability between frameworks, for example - it's just ASP's opinions about how headers should be done. Building that on top of kernel layer where headers are just name and values represented as strings ought to be completely feasible, since ultimately they have to be strings to get trabsported, right? HTTP kernels in other languages to my knowledge all represent headers as strings. Just one example, but I don't think there's any technical reason why there can't be a lower level of abstraction that has only an essential degree of abstraction, e.g. request/response and a simple request->response delegate. (or context object, since likely you'd have to stick with the idea of mutable request/response instances for the sake of performance and compatibility with the high-level middleware abstraction in ASP.) I don't think a separate, incompatible package is the way to go. That'd almost be worse than the status quo. Might as well just leave HttpListener in place then, since it already seems to meet the humble requirements of those who commented on this thread. |
|
I don't misunderstand you.
Sure, that's not low level enough to control the types of performance optimizations that happen today in ASP.NET Core. Trust me we've done this before with OWIN which built further abstractions on top of HttpListener. In fact we still do this today with a ASP.Core. We gutting the HttpListener implementation on windows to directly integrate with the ASP.NET Core abstractions (https://github.com/dotnet/aspnetcore/tree/master/src/Servers/HttpSys/src). All that said, it's completely reasonable to build this high level API. There would just need be something lower that ASP.NET Core could build on top of.
So HttpListener is already too high level? https://docs.microsoft.com/en-us/dotnet/api/system.net.httplistenerresponse?view=netcore-3.1#properties. Even the cookies are represented as a CookieCollection. I don't buy your definition of what makes an opinioned framework.
Should be bytes not strings, that the kernel I need.
I'd be fine not deprecating it, but it's not going to get any new features. |
No, that's totally fine - of course we should have a collection of cookies that makes them manageable. Where it gets opinionated is where ASP also abstracts individual types of cookies - something that can easily be built on top of a simple cookie collection.
This makes me think we're not really in disagreement here. 🙂 I didn't mean for that short list to be exhaustive - I only omitted cookies because I forgot. Things like sessions, authentication, and so on are just cookies being exchanged though, and all that doesn't need to exist in an HTTP kernel layer - you could comfortably and efficiently build those things on top of a lightweight kernel with a simple cookie collection, right? |
It does? Clue me in please? Which ones? Is it on the abstraction?
The kernel I want wouldn’t have headers as strings or cookies exposed. Nor would it represent the body as a stream. That’s the core of why making another abstraction is a no go. If you can reasonably use the guts of the kernel somehow then I’m all ears. Having spent the last 5 years of my life working on an HTTP server I have some strong opinions on how to build them 😬 |
|
Just having the same functionality as the current HTTP listener does would be great. Scratch the stuff that is outdated ( like the cert handling etc. etc. ) and I would be more than happy. :) |
There's a whole bunch of classes to deal with many common header types: https://github.com/dotnet/aspnetcore/tree/master/src/Http/Headers/src |
Those aren't even part of the abstractions. They're optional and were made intentionally so as a convenience to other framework authors (and higher level applications). |
|
@davidfowl okay, sorry about the bad examples - tbh I'm returning to .NET after 7 years away, and I don't know the current state of the platform very well yet. I'm only noticing, as somebody who's worked with several other languages/platforms, that the API surface area of ASP is much bigger and much more complex than basically anything else I've worked with. I wish there was a much smaller and simpler kernel layer that was easier to learn and more suitable for smaller/simpler tasks, embedding, etc. - but I guess I can't say what that would entail in concrete terms. You would definitely have a much better idea than me as to what would be possible and meaningful. The problem is real enough though, so I do hope this is something that improves in the future. I want to be able to tell a friend, "try this", rather than "read this book and spend the next couple of weeks learning this", you know? I think the main reason Node is so popular, for example, is because it's easy to get on board. If you know HTTP, you'll figure out middleware in a couple of hours. It should be possible to design something that gives new devs a more immediate sense of security - "this works and I understand why". I don't think most people get that with ASP right now. |
|
Point your friends here https://github.com/featherhttp/tutorial |
|
@davidfowl I think I'll point myself there! 😀 ... this looks nice and clean. I'm also aware of Nancy in this space. They are of course both abstracting away a large, complex framework - but I guess, in the same sense, PSR standards abstract away PHP's outmoded CGI API, and I was quite happy with that, so this might be the direction I'm looking for. Thanks! 🙂 |
System.Net.HttpListenerdoes not support many modern protocols. It is compat-only (i.e. only critical fixes, no new improvements, enhancements). It's useful for low-volume basic server requests, but does not scale to modern requirements of HTTP servers (incl. performance).Use KestrelHttpServer instead.
The text was updated successfully, but these errors were encountered: