fix(meshpassthrough): disable tls and http inspector for mysql protoc…

…ol (#12839)

## Motivation

When trying to communicate with MySQL, the user wants to enable
communication with the database. Unfortunately, the MySQL protocol is
slightly different, and the usual method of configuring filter chains
does not work when using MeshPassthrough.

## Implementation information

* Added `original_dst` listener filter
* Added a protocol `mysql` which works only with CIDR/IP and requires
port since we need to disable `tls_inspector` and `http_inspector`
listener filters for the port
* Mysql protocol creates a `tcp_proxy` but with with disabled listener
filters
* Added test

> Why added mysql protocol and not just disable on ports with TCP
protocol?

The user may have rules with HTTP traffic and TCP traffic on the same
port matching different IP. example: tls matching on port 8080(IP:
192.168.1.1) and TCP matching on 8080 (IP: 172.1.1.1), that would
disable TLS inspector on the port 8080 and wouldn't match

## Supporting documentation


https://dev.mysql.com/doc/dev/mysql-server/8.4.3/page_protocol_connection_phase_packets.html
envoyproxy/envoy#21044

---------

Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>

app/kumactl/cmd/install/testdata/install-control-plane.defaults.golden.yaml

@@ -3995,13 +3995,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

app/kumactl/cmd/install/testdata/install-control-plane.gateway-api-present.yaml

@@ -3995,13 +3995,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

app/kumactl/cmd/install/testdata/install-control-plane.with-helm-set.yaml

@@ -4015,13 +4015,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

app/kumactl/cmd/install/testdata/install-crds.all.golden.yaml

@@ -6025,13 +6025,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

deployments/charts/kuma/crds/kuma.io_meshpassthroughs.yaml

@@ -65,13 +65,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

docs/generated/openapi.yaml

@@ -9413,13 +9413,15 @@ components:
                         default: tcp
                         description: >-
                           Protocol defines the communication protocol. Possible
-                          values: `tcp`, `tls`, `grpc`, `http`, `http2`.
+                          values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                          `mysql`.
                         enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                         type: string
                       type:
                         description: >-

docs/generated/raw/crds/kuma.io_meshpassthroughs.yaml

@@ -65,13 +65,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

pkg/core/resources/apis/mesh/dataplane_helpers.go

@@ -27,6 +27,7 @@ const (
 	ProtocolHTTP2   = "http2"
 	ProtocolGRPC    = "grpc"
 	ProtocolKafka   = "kafka"
+	ProtocolMysql   = "mysql"
 )
 
 func ParseProtocol(tag string) Protocol {
@@ -43,6 +44,8 @@ func ParseProtocol(tag string) Protocol {
 		return ProtocolGRPC
 	case ProtocolKafka:
 		return ProtocolKafka
+	case ProtocolMysql:
+		return ProtocolMysql
 	default:
 		return ProtocolUnknown
 	}

pkg/core/resources/apis/mesh/dataplane_helpers_test.go

@@ -663,7 +663,7 @@ var _ = Describe("Dataplane", func() {
 			}),
 			Entry("mysql", testCase{
 				tag:      "mysql",
-				expected: ProtocolUnknown,
+				expected: ProtocolMysql,
 			}),
 			Entry("unknown", testCase{
 				tag:      "unknown",

pkg/plugins/policies/meshpassthrough/api/v1alpha1/meshpassthrough.go

@@ -30,7 +30,7 @@ type PassthroughMode string
 // +kubebuilder:validation:Enum=Domain;IP;CIDR
 type MatchType string
 
-// +kubebuilder:validation:Enum=tcp;tls;grpc;http;http2
+// +kubebuilder:validation:Enum=tcp;tls;grpc;http;http2;mysql
 type ProtocolType string
 
 const (
@@ -39,6 +39,7 @@ const (
 	GrpcProtocol  ProtocolType = "grpc"
 	HttpProtocol  ProtocolType = "http"
 	Http2Protocol ProtocolType = "http2"
+	MysqlProtocol ProtocolType = "mysql"
 )
 
 type Match struct {
@@ -48,7 +49,7 @@ type Match struct {
 	Value string `json:"value"`
 	// Port defines the port to which a user makes a request.
 	Port *uint32 `json:"port,omitempty"`
-	// Protocol defines the communication protocol. Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.
+	// Protocol defines the communication protocol. Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`, `mysql`.
 	// +kubebuilder:default=tcp
 	// +kubebuilder:validation:Optional
 	Protocol ProtocolType `json:"protocol"`

pkg/plugins/policies/meshpassthrough/api/v1alpha1/schema.yaml

@@ -34,13 +34,14 @@ properties:
                   type: integer
                 protocol:
                   default: tcp
-                  description: 'Protocol defines the communication protocol. Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                  description: 'Protocol defines the communication protocol. Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`, `mysql`.'
                   enum:
                     - tcp
                     - tls
                     - grpc
                     - http
                     - http2
+                    - mysql
                   type: string
                 type:
                   description: Type of the match, one of `Domain`, `IP` or `CIDR` is available.

pkg/plugins/policies/meshpassthrough/api/v1alpha1/testdata/full-invalid.input.yaml

@@ -45,3 +45,10 @@ default:
   - type: Domain
     value: "*.example.com"
     protocol: http
+  - type: Domain
+    value: mysql.server.com
+    port: 3306
+    protocol: mysql
+  - type: IP
+    value: 172.1.1.1
+    protocol: mysql

pkg/plugins/policies/meshpassthrough/api/v1alpha1/testdata/full-invalid.output.yaml

@@ -2,7 +2,7 @@ violations:
 - field: spec.default.appendMatch[0].port
   message: port must be a valid (1-65535)
 - field: spec.default.appendMatch[0].protocol
-  message: '"protocol" must be one of ["tcp", "tls", "grpc", "http", "http2"]'
+  message: '"protocol" must be one of ["tcp", "tls", "grpc", "http", "http2", "mysql"]'
 - field: spec.default.appendMatch[0].type
   message: provided type test is not supported, one of Domain, IP, or CIDR is supported
 - field: spec.default.appendMatch[1].protocol
@@ -24,3 +24,7 @@ violations:
   message: value google.com is already defiend for this port and protocol
 - field: spec.default.appendMatch[10].port
   message: wildcard domains doesn't work for all ports and layer 7 protocol
+- field: spec.default.appendMatch[11].protocol
+  message: protocol mysql is not supported for a domain
+- field: spec.default.appendMatch[12].port
+  message: port must be defined for Mysql protocol

pkg/plugins/policies/meshpassthrough/api/v1alpha1/testdata/valid.input.yaml

@@ -21,3 +21,7 @@ default:
     value: '*.example.com'
     port: 9090
     protocol: http
+  - type: IP
+    value: 172.1.1.1
+    port: 3306
+    protocol: mysql

pkg/plugins/policies/meshpassthrough/api/v1alpha1/validator.go

@@ -16,7 +16,7 @@ import (
 )
 
 var (
-	allMatchProtocols                = []string{string(TcpProtocol), string(TlsProtocol), string(GrpcProtocol), string(HttpProtocol), string(Http2Protocol)}
+	allMatchProtocols                = []string{string(TcpProtocol), string(TlsProtocol), string(GrpcProtocol), string(HttpProtocol), string(Http2Protocol), string(MysqlProtocol)}
 	notAllowedProtocolsOnTheSamePort = []ProtocolType{GrpcProtocol, HttpProtocol, Http2Protocol}
 	wildcardPartialPrefixPattern     = regexp.MustCompile(`^\*[^\.]+`)
 )
@@ -52,6 +52,9 @@ func validateDefault(conf Conf) validators.ValidationError {
 	}
 	uniqueDomains := map[portProtocol]map[string]bool{}
 	for i, match := range pointer.Deref(conf.AppendMatch) {
+		if match.Protocol == MysqlProtocol && match.Port == nil {
+			verr.AddViolationAt(validators.RootedAt("appendMatch").Index(i).Field("port"), "port must be defined for Mysql protocol")
+		}
 		if match.Port != nil && pointer.Deref[uint32](match.Port) == 0 || pointer.Deref[uint32](match.Port) > math.MaxUint16 {
 			verr.AddViolationAt(validators.RootedAt("appendMatch").Index(i).Field("port"), "port must be a valid (1-65535)")
 		}
@@ -90,8 +93,8 @@ func validateDefault(conf Conf) validators.ValidationError {
 				verr.AddViolationAt(validators.RootedAt("appendMatch").Index(i).Field("value"), "provided IP has incorrect value")
 			}
 		case "Domain":
-			if match.Protocol == "tcp" {
-				verr.AddViolationAt(validators.RootedAt("appendMatch").Index(i).Field("protocol"), "protocol tcp is not supported for a domain")
+			if match.Protocol == "tcp" || match.Protocol == "mysql" {
+				verr.AddViolationAt(validators.RootedAt("appendMatch").Index(i).Field("protocol"), fmt.Sprintf("protocol %s is not supported for a domain", match.Protocol))
 			}
 			if wildcardPartialPrefixPattern.MatchString(match.Value) {
 				verr.AddViolationAt(validators.RootedAt("appendMatch").Index(i).Field("value"), "provided DNS has incorrect value, partial wildcard is currently not supported")

pkg/plugins/policies/meshpassthrough/k8s/crd/kuma.io_meshpassthroughs.yaml

@@ -65,13 +65,15 @@ spec:
                         protocol:
                           default: tcp
                           description: 'Protocol defines the communication protocol.
-                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`.'
+                            Possible values: `tcp`, `tls`, `grpc`, `http`, `http2`,
+                            `mysql`.'
                           enum:
                           - tcp
                           - tls
                           - grpc
                           - http
                           - http2
+                          - mysql
                           type: string
                         type:
                           description: Type of the match, one of `Domain`, `IP` or

pkg/plugins/policies/meshpassthrough/plugin/v1alpha1/plugin.go

@@ -79,12 +79,12 @@ func applyToOutboundPassthrough(
 		// add clusters because they were not added in TransparentProxyGenerator
 		return addDefaultPassthroughClusters(rs, proxy.APIVersion)
 	}
-	if ctx.Mesh.Resource.Spec.IsPassthrough() && conf.PassthroughMode != nil && pointer.Deref[api.PassthroughMode](conf.PassthroughMode) == api.PassthroughMode("All") {
+	if ctx.Mesh.Resource.Spec.IsPassthrough() && conf.PassthroughMode != nil && pointer.Deref(conf.PassthroughMode) == api.PassthroughMode("All") {
 		// clusters were added in TransparentProxyGenerator, do nothing
 		return nil
 	}
 
-	if conf.PassthroughMode != nil && pointer.Deref[api.PassthroughMode](conf.PassthroughMode) == api.PassthroughMode("Matched") || conf.PassthroughMode == nil {
+	if conf.PassthroughMode != nil && pointer.Deref(conf.PassthroughMode) == api.PassthroughMode("Matched") || conf.PassthroughMode == nil {
 		removeDefaultPassthroughCluster(rs)
 		if len(pointer.Deref(conf.AppendMatch)) > 0 {
 			configurer := xds.Configurer{

pkg/plugins/policies/meshpassthrough/plugin/v1alpha1/plugin_test.go

@@ -506,6 +506,57 @@ var _ = Describe("MeshPassthrough", func() {
 			listenersGolden: "same-protocol.listener.golden.yaml",
 			clustersGolden:  "same-protocol.clusters.golden.yaml",
 		}),
+		Entry("mysql protocol", testCase{
+			resources: []*core_xds.Resource{
+				{
+					Name:   "outbound:passthrough:ipv4",
+					Origin: generator.OriginTransparent,
+					Resource: NewListenerBuilder(envoy_common.APIV3, "outbound:passthrough:ipv4").
+						Configure(OutboundListener("0.0.0.0", 15001, core_xds.SocketAddressProtocolTCP)).
+						Configure(FilterChain(NewFilterChainBuilder(envoy_common.APIV3, envoy_common.AnonymousResource).
+							Configure(TCPProxy("outbound_passthrough_ipv4", []envoy_common.Split{
+								plugins_xds.NewSplitBuilder().WithClusterName("outbound:passthrough:ipv4").WithWeight(100).Build(),
+							}...)),
+						)).MustBuild(),
+				},
+				{
+					Name:   "outbound:passthrough:ipv6",
+					Origin: generator.OriginTransparent,
+					Resource: NewListenerBuilder(envoy_common.APIV3, "outbound:passthrough:ipv6").
+						Configure(OutboundListener("::", 15001, core_xds.SocketAddressProtocolTCP)).
+						Configure(FilterChain(NewFilterChainBuilder(envoy_common.APIV3, envoy_common.AnonymousResource).
+							Configure(TCPProxy("outbound_passthrough_ipv6", []envoy_common.Split{
+								plugins_xds.NewSplitBuilder().WithClusterName("outbound:passthrough:ipv6").WithWeight(100).Build(),
+							}...)),
+						)).MustBuild(),
+				},
+			},
+			singleItemRules: core_rules.SingleItemRules{
+				Rules: []*core_rules.Rule{
+					{
+						Subset: []subsetutils.Tag{},
+						Conf: api.Conf{
+							AppendMatch: &[]api.Match{
+								{
+									Type:     api.MatchType("IP"),
+									Value:    "172.12.2.2",
+									Port:     pointer.To[uint32](3306),
+									Protocol: api.ProtocolType("mysql"),
+								},
+								{
+									Type:     api.MatchType("CIDR"),
+									Value:    "172.12.2.2/24",
+									Port:     pointer.To[uint32](3307),
+									Protocol: api.ProtocolType("mysql"),
+								},
+							},
+						},
+					},
+				},
+			},
+			listenersGolden: "mysql-protocol.listener.golden.yaml",
+			clustersGolden:  "mysql-protocol.clusters.golden.yaml",
+		}),
 		Entry("disabled on policy but enabled on mesh", testCase{
 			resources: []*core_xds.Resource{
 				{

pkg/plugins/policies/meshpassthrough/plugin/v1alpha1/testdata/basic.listener.golden.yaml

@@ -370,6 +370,9 @@ resources:
           statPrefix: meshpassthrough_http_80
       name: meshpassthrough_http_80
     listenerFilters:
+    - name: envoy.filters.listener.original_dst
+      typedConfig:
+        '@type': type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
     - name: envoy.filters.listener.tls_inspector
       typedConfig:
         '@type': type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
@@ -600,6 +603,9 @@ resources:
           statPrefix: meshpassthrough_http_80
       name: meshpassthrough_http_80
     listenerFilters:
+    - name: envoy.filters.listener.original_dst
+      typedConfig:
+        '@type': type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
     - name: envoy.filters.listener.tls_inspector
       typedConfig:
         '@type': type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector

pkg/plugins/policies/meshpassthrough/plugin/v1alpha1/testdata/cidr-http.listener.golden.yaml

@@ -127,6 +127,9 @@ resources:
           statPrefix: meshpassthrough_http_10_10_0_0_16_80
       name: meshpassthrough_http_10.10.0.0/16_80
     listenerFilters:
+    - name: envoy.filters.listener.original_dst
+      typedConfig:
+        '@type': type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
     - name: envoy.filters.listener.tls_inspector
       typedConfig:
         '@type': type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector

pkg/plugins/policies/meshpassthrough/plugin/v1alpha1/testdata/http-domains-aggregated.listener.golden.yaml

@@ -120,6 +120,9 @@ resources:
           statPrefix: meshpassthrough_http__
       name: meshpassthrough_http_*
     listenerFilters:
+    - name: envoy.filters.listener.original_dst
+      typedConfig:
+        '@type': type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
     - name: envoy.filters.listener.tls_inspector
       typedConfig:
         '@type': type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
@@ -249,6 +252,9 @@ resources:
           statPrefix: meshpassthrough_http__
       name: meshpassthrough_http_*
     listenerFilters:
+    - name: envoy.filters.listener.original_dst
+      typedConfig:
+        '@type': type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
     - name: envoy.filters.listener.tls_inspector
       typedConfig:
         '@type': type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector