net/http: configurable error message for Client sent an HTTP request to an HTTPS server.

[mzky]

What version of Go are you using (go version)?

$ go version
go version go1.17.2 linux/amd64

Does this issue reproduce with the latest release?

yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/var/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/var/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/root/go/src/net/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1909543103=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Start the HTTPS service written in golang, and the user accesses the HTTP address.

What did you expect to see?

Display localized language information or customize html page or automatically jump to https address.

What did you see instead?

1. Chrome and Firefox:
   display the following information
   

2. Other browsers:
   The IE browser prompts a 400 error, and the instructional content is not displayed.
   Browsers using the Chromium kernel do not display any visible content.
   
   Users will not be able to understand the current situation.

[thanm]

Thanks for raising the issue.

From my read this sounds more like a feature request as opposed to a bug, correct?

@neild

[neild]

The feature request is to respond to an HTTP request on an HTTPS port with a configurable error message.

Testing several major websites (www.google.com, www.amazon.com, www.microsoft.com), none of them respond with an error message to an HTTP request on port 443. Two close the connection without response, one leaves the connection open but does not respond.

I have not checked any other HTTPS server implementations (Apache, nginx, etc.) to see how they handle this condition. It would be interesting to know if any attempt to report an error to the peer in this case.

In the absence of evidence that this is a common feature, I don't think we should add this.

[mzky]

Hi ! This is a feature request, not a bug.

I found some examples of Nginx configuring HTTP redirection to HTTPS：https://linuxize.com/post/redirect-http-to-https-in-nginx/

1. Old version Parameters：
   ···
   rewrite ^(.*)$ https://$host$1 permanent;
   ···
2. New version Parameters：
   ···
   return 301 https://$server_name$request_uri;
   ···

I want to replace nginx's auto-jump feature with Golang.
Of course, this is just a suggestion, the changes will not affect the original functions.

[seankhliao]

HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers

go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        // handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)

[mzky]

HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers

go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
        // handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)

Yes, different port forwarding is also possible, but I need a forwarding on the same port

[mzky]

Go language simulates google：

[neild]

They all jump to https with the same port:

An https:// URL with no port uses a default port of 443, not 80. All these are redirecting from HTTP on port 80 to HTTPS on port 443.

[hktalent]

I would also like to make some comments on this issue

• First of all, since the server can output "Client send an HTTP request to an HTTPS server", it should give the control of the output to the developer instead of burying him deeply

• Secondly, to solve this problem, we should not use nginx or listen to port 80 more, which are very cumbersome

[hktalent]

Yes, any core modifications don't seem to be standard-compliant, but, that's the truth, we need a way to intercept when "Client send an HTTP request to an HTTPS server" happens, or have a kind of friendly wrapper that allows him to Controllable output, because "Client send an HTTP request to an HTTPS server" directly output to the browser is very unfriendly, and it is uncontrollable and unexpected

[mitar]

I would also like that this is configurable. Maybe http.Server could get the following field:

TLSHandshakeBadRequest func(conn net.Conn)

And if not defined, current:

io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

is called. Otherwise TLSHandshakeBadRequest(re.Conn) is called.

Open to other field names.

[martin0104]

i have same problem that response content not friendly, i wish can configurable the content

[bddjr]

The nginx example you provided is incorrect, it should be like this:

# port 443
error_page 497 https://$host$request_uri;

# other ports
error_page 497 https://$host:$server_port$request_uri;

Then, attempting to access the HTTPS port using HTTP will return a 302 redirect.

[gopherbot]

Change https://go.dev/cl/564997 mentions this issue: net/http: configurable error message for Client sent an HTTP request …

[neild]

It looks like nginx responds to an HTTP request on an HTTPS port with an error 497 (an nginx-specific code), and permits code-specific error handler overrides. The example nginx configuration from above is interesting, because it seems to indicate that nginx is parsing the full HTTP request in this case:

error_page 497 https://$host$request_uri;

This points at a question: Is the desired feature request to customize the static message we send when responding to a misdirected HTTP request, or to respond with a redirect to the correct location? The latter is substantially more complicated to implement, because it requires effectively restarting the connection.

If were were to add a feature, I could imagine something like:

type Server struct {
  // HTTPOnHTTPSPortErrorResponseString is sent on an HTTPS connection
  // which receives what looks like an HTTP request.
  HTTPOnHTTPSPortErrorResponse string
}

But that's static, and it sounds like a common use case is a redirect. So maybe what people want is:

type Server struct {
  // HTTPOnHTTPSPortErrorHandler handles HTTP requests sent to an HTTPS port.
  HTTPOnHTTPSPortErrorHandler Handler
}

Although in that case, what happens when someone inevitably sets Server.Handler and Server.HTTPOnHTTPSPortErrorHandler to the same thing? Does that work? Should that work? Are there edge cases where it fails? I think implementation for this would also be complicated; unless I'm wrong, I don't think this is something we'd want to do.

A middle ground, suggested by @mitar above, is essentially a roll-your-own handler where we hand the net.Conn to the user:

type Server struct {
  TLSHandshakeBadRequest func(conn net.Conn)
}

You're going to want the already-consumed bytes from the net.Conn as well though, no?

I don't have any clear answers here. There's been a fair bit of discussion on this issue over time, but half of it is about cases other than an HTTP request sent to an HTTPS port. (For example, redirecting from HTTP on port 80 to HTTPS on port 443, which of course is something you can do today.) Some of the example use cases (such as that nginx config) involve more than just customizing the error string.

Would a statically customizable response string be enough? That's at least implementable without adding a ton of complexity.

[mitar]

@neild Yea, I realized this in https://go-review.googlesource.com/c/go/+/564997 as well. I think we could have:

type Server struct {
  TLSHandshakeBadRequest func(conn net.Conn, recordedBytes []byte)
}

But I think those recorded bytes are really optional and it might be hard for later on for implementation to change its behavior. If we look at current code, it does not use those bytes, it just writes to the connection:

io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

So I was envisioning that the TLSHandshakeBadRequest would do the same, only maybe write a Location header redirect.

I am not sure how much utility is in inspecting recordedBytes?

[mitar]

I see. To allow virtual hosts redirects. And if requestBytes does not contain enough data (not yet all headers), you can still read more from the connection?

[mitar]

@bddjr Please read contributing guide. I think this proposal has not yet been accepted. Making a pull request is fine to be able to discuss code, but until the proposal is accepted it cannot be merged.

And about your last proposal: personally, I think the Config is overkill, I think we should have only one function callback and not so many options.

Also, what is badRequestResponse in HttpOnHttpsPortErrorHandler?

[bddjr]

✅ I made a go mod to implement this feature ✨

👉 https://github.com/bddjr/hlfhr 👈

It uses the http.ReadRequest to read the request headers, and its custom Handler uses the http.Handler format, which handles HTTP requests in a standard way that is close to the standard, rather than giving a bunch of characters for the user to deal with.

Support HTTP/2 on HTTPS

// Use hlfhr.New
srv := hlfhr.New(&http.Server{
	// Write something...
})
// Then just use it like [http.Server]
err := srv.ListenAndServeTLS("localhost.crt", "localhost.key")

srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	url := "https://" + r.Host + r.URL.Path
	if r.URL.ForceQuery || r.URL.RawQuery != "" {
		url += "?" + r.URL.RawQuery
	}
	w.Header().Set("Location", url)
	w.WriteHeader(302)
})

[mzky]

A simple example of the redirect method.

package main

import (
	"fmt"
	"github.com/gin-gonic/gin"
	"github.com/mzky/utils/hook"
	"net/http"
)

func main() {
	engine := gin.New()
	engine.GET("/", func(c *gin.Context) {
		c.String(http.StatusOK, "Hello World!")
	})
	engine.GET("/login.html", func(c *gin.Context) {
		c.String(http.StatusOK, "login.html ok!")
	})

	// The default value of `hook` is "HTTP/1.1 302 Found\r\n\r\n<script>location.protocol='https:'</script>\r\n"
	// Restore the default values of the standard library ↓
	// srv.SetDefaultBadRequest("HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")

	srv := hook.NewServer(":1234", engine)

	// custom response
	//srv.SetKeepAlivesEnabled(false) 
	//body := "---- message ----"
	//srv.SetResponse(body, func(r *http.Response) {
	//	r.StatusCode = 400
	//	r.Status = http.StatusText(400)
	//	r.Header.Set("Content-Type", "text/html")
	//})

	// custom response 2
	//srv.SetRedirectPath("/login.html") // Optional redirection address
	//srv.SetResponse("", func(r *http.Response) {
	//	r.StatusCode = 307
	//	r.Status = http.StatusText(307)
	//	r.Header.Set("Timing-Allow-Origin", "*")
	//	r.Header.Set("Non-Authoritative-Reason", "HSTS") // HSTS mode to redirect faster than `script`
	//	r.Header.Set("x-xss-protection", "0")
	//	r.Header.Set("Cross-Origin-Resource-Policy", "Cross-Origin")
	//	r.Header.Set("Content-Type", "text/html; charset=utf-8")
	//	r.Header.Set("Location", "https://192.168.0.188:7569") // Optional redirection address
	//})

	fmt.Println(srv.ListenAndServeTLS("server.pem", "server.key"))
}

[bddjr]

In theory, implementing this feature only requires an additional buffer and handler parameter. When the first byte of the request is detected to be like an HTTP method, the TLS handshake is skipped and the HTTP/1.1 handler is directly entered.

[mzky]

@mzky 你代码写得一团糟。

你正在使用的方案与hlfhr弃用的旧版本相似，这会导致网址与请求头超过576字节时没被完整读取，最终导致客户端显示connection was reset。（你应该知道我写那么多代码不是没有原因的）

你是通过劫持Write实现，如果未来标准库支持了这个功能，可能会导致你的方案不能正常工作。

我不清楚使用unsafe获取tls的私有函数会不会导致go版本1.23无法正常使用。

http响应内容还有更轻量的，在确保请求头全部读取完成的情况下，可以直接返回这个

"HTTP/1.1 302 Found\r\nConnection: close\r\n\r\n<script>location.protocol='https:'</script>"

After verification in multiple browsers, they can meet my needs.

Test the 7439 kB request header, and the verification result is normal：

[bddjr]

Your implementation approach has a bug, it will cause the HTTPS don't use HTTP2.

---

✅ The mod I wrote would obviously be more in line with neild's wishes.

👉 https://github.com/bddjr/hlfhr 👈

It uses the http.ReadRequest to read the request headers, and its custom Handler uses the http.Handler format, which handles HTTP requests in a standard way that is close to the standard, rather than giving a bunch of characters for the user to deal with.

Support HTTP/2 on HTTPS

// Use hlfhr.New
srv := hlfhr.New(&http.Server{
	// Write something...
})
// Then just use it like [http.Server]
err := srv.ListenAndServeTLS("localhost.crt", "localhost.key")

[mzky]

The existing solution can solve this problem without using the standard library, so it has been closed.

[mitar]

Please reopen the issue, it is not yet solved in standard library.

[bronze1man]

Your implementation approach has a bug, it will cause the HTTPS don't use HTTP2.

I have write a library that can auto redirect http to https at same port.
It works on tls+http/2

https://github.com/bronze1man/httpRedirectToHttps

[bddjr]

Your code isn't as good as mine.

[bronze1man]

Your code isn't as good as mine.

Your code do not have "auto check correct" test. But my code have . (https://github.com/bronze1man/httpRedirectToHttps/blob/master/httpRedirectToHttpsTest/example_test.go)

[bddjr]

☝ You plagiarized my project: https://github.com/bddjr/hahosp

srv := &http.Server{
    Addr:    ":5688"
    // Use hahosp.HandlerSelector
    Handler: &hahosp.HandlerSelector{
        HTTPS: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            io.WriteString(w, "ok\n")
        }),
        HTTP: nil, // If nil, redirect to HTTPS.
    },
}

// Use hahosp.ListenAndServe
err := hahosp.ListenAndServe(srv, "localhost.crt", "localhost.key")

---

✅ The mod I wrote would obviously be more in line with neild's wishes.

👉 https://github.com/bddjr/hlfhr 👈

It uses the http.ReadRequest to read the request headers, and its custom Handler uses the http.Handler format, which handles HTTP requests in a standard way that is close to the standard, rather than giving a bunch of characters for the user to deal with.

Support HTTP/2 on HTTPS

// Use hlfhr.New
srv := hlfhr.New(&http.Server{
	// Write something...
})
// Then just use it like [http.Server]
err := srv.ListenAndServeTLS("localhost.crt", "localhost.key")

[seankhliao]

As the various implementations demonstrate, this is quite possible to do outside of the standard library. See https://go.dev/doc/faq#x_in_std
Given that it is not a widely needed feature, and there is no right solution, it is probably best to choose an implementation that makes the appropriate tradeoffs for your given situation and needs.