Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: route-level MTLS #9322

Merged
merged 10 commits into from
May 6, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions apisix/init.lua
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ local str_sub = string.sub
local tonumber = tonumber
local type = type
local pairs = pairs
local ngx_re_match = ngx.re.match
local control_api_router

local is_http = false
Expand Down Expand Up @@ -179,6 +180,8 @@ function _M.http_ssl_phase()

local ok, err = router.router_ssl.match_and_set(api_ctx)
monkeyDluffy6017 marked this conversation as resolved.
Show resolved Hide resolved

ngx_ctx.matched_ssl = api_ctx.matched_ssl
ngx.log(ngx.WARN, require("inspect")(api_ctx.matched_ssl))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this

core.tablepool.release("api_ctx", api_ctx)
ngx_ctx.api_ctx = nil

Expand Down Expand Up @@ -310,12 +313,38 @@ local function verify_tls_client(ctx)
end


local function uri_matches_skip_mtls_route_patterns(ssl, uri)
for _, pat in ipairs(ssl.value.client.skip_mtls_uri_regex) do
if ngx_re_match(uri, pat, "jo") then
return true
end
end
end


local function verify_https_client(ctx)
local scheme = ctx.var.scheme
if scheme ~= "https" then
return true
end

local matched_ssl = ngx.ctx.matched_ssl
if matched_ssl.value.client
and matched_ssl.value.client.skip_mtls_uri_regex
and apisix_ssl.support_client_verification()
and (not uri_matches_skip_mtls_route_patterns(matched_ssl, ngx.var.uri)) then
local res = ctx.var.ssl_client_verify
if res ~= "SUCCESS" then
if res == "NONE" then
core.log.error("client certificate was not present")
else
core.log.error("client certificate verification is not passed: ", res)
end

return false
end
monkeyDluffy6017 marked this conversation as resolved.
Show resolved Hide resolved
end

local host = ctx.var.host
local matched = router.router_ssl.match_and_set(ctx, true, host)
if not matched then
Expand Down
9 changes: 9 additions & 0 deletions apisix/schema_def.lua
Original file line number Diff line number Diff line change
Expand Up @@ -767,6 +767,15 @@ _M.ssl = {
minimum = 0,
default = 1,
},
skip_mtls_uri_regex = {
type = "array",
minItems = 1,
uniqueItems = true,
items = {
description = "uri regular expression to skip mtls",
type = "string",
}
},
},
required = {"ca"},
},
Expand Down
7 changes: 6 additions & 1 deletion apisix/ssl/router/radixtree_sni.lua
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ local apisix_ssl = require("apisix.ssl")
local secret = require("apisix.secret")
local ngx_ssl = require("ngx.ssl")
local config_util = require("apisix.core.config_util")
local ngx = ngx
local ipairs = ipairs
local type = type
local error = error
Expand Down Expand Up @@ -229,7 +230,11 @@ function _M.match_and_set(api_ctx, match_only, alt_sni)
return false, "failed to parse client cert: " .. err
end

local ok, err = ngx_ssl.verify_client(parsed_cert, depth)
local reject_in_handshake =
(ngx.config.subsystem == "stream") or
(matched_ssl.value.client.skip_mtls_uri_regex == nil)
local ok, err = ngx_ssl.verify_client(parsed_cert, depth,
reject_in_handshake)
if not ok then
return false, err
end
Expand Down
Binary file added docs/assets/images/skip-mtls.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/en/latest/admin-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1160,6 +1160,7 @@ SSL resource request address: /apisix/admin/ssls/{id}
| keys | False | An array of private keys | Private keys to pair with the `certs`. | |
| client.ca | False | Certificate | Sets the CA certificate that verifies the client. Requires OpenResty 1.19+. | |
| client.depth | False | Certificate | Sets the verification depth in client certificate chains. Defaults to 1. Requires OpenResty 1.19+. | |
| client.skip_mtls_uri_regex | False | An array of regular expressions, in PCRE format | Used to match URI, if matched, this request bypasses the client certificate checking, i.e. skip the MTLS. | ["/hello[0-9]+", "/foobar"] |
| snis | True, only if `type` is `server` | Match Rules | A non-empty array of HTTPS SNI | |
| labels | False | Match Rules | Attributes of the resource specified as key-value pairs. | {"version":"v2","build":"16","env":"production"} |
| create_time | False | Auxiliary | Epoch timestamp (in seconds) of the created time. If missing, this field will be populated automatically. | 1602883670 |
Expand Down
121 changes: 121 additions & 0 deletions docs/en/latest/tutorials/client-to-apisix-mtls.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,127 @@ curl --resolve "test.com:9443:127.0.0.1" https://test.com:9443/anything -k --cer

Since we configured the [proxy-rewrite](../plugins/proxy-rewrite.md) plugin in the example, we can see that the response body contains the request body received upstream, containing the correct data.

## MTLS bypass based on regular expression matching against URI

APISIX allows configuring an URI whitelist to bypass MTLS.
If the URI of a request is in the whitelist, then the client certificate will not be checked.
Note that other URIs of the associated SNI will get HTTP 400 response
instead of alert error in the SSL handshake phase, if the client certificate is missing or invalid.

### Timing diagram

![skip mtls](https://raw.githubusercontent.com/apache/apisix/master/docs/assets/images/skip-mtls.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


### Example

```bash
curl http://127.0.0.1:9180/apisix/admin/routes/1 \
monkeyDluffy6017 marked this conversation as resolved.
Show resolved Hide resolved
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/*",
"upstream": {
"nodes": {
"httpbin.org": 1
}
}
}'

curl http://127.0.0.1:9180/apisix/admin/ssls/1 \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"cert": "'"$(<t/certs/mtls_server.crt)"'",
"key": "'"$(<t/certs/mtls_server.key)"'",
"snis": [
"*.apisix.dev"
],
"client": {
"ca": "'"$(<t/certs/mtls_ca.crt)"'",
"depth": 10,
"skip_mtls_uri_regex": [
"/anything.*"
]
}
}'

#
# if the client certificate is missing and the URI is not in the whitelist,
# then you'll get HTTP 400 response.
#
curl https://admin.apisix.dev:9443/uuid -v \
--resolve 'admin.apisix.dev:9443:127.0.0.1' --cacert t/certs/mtls_ca.crt
* Added admin.apisix.dev:9443:127.0.0.1 to DNS cache
* Hostname admin.apisix.dev was found in DNS cache
* Trying 127.0.0.1:9443...
* TCP_NODELAY set
* Connected to admin.apisix.dev (127.0.0.1) port 9443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: t/certs/mtls_ca.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=cn; ST=GuangDong; L=ZhuHai; CN=admin.apisix.dev; OU=ops
* start date: Dec 1 10:17:24 2022 GMT
* expire date: Aug 18 10:17:24 2042 GMT
* subjectAltName: host "admin.apisix.dev" matched cert's "admin.apisix.dev"
* issuer: C=cn; ST=GuangDong; L=ZhuHai; CN=ca.apisix.dev; OU=ops
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56246de24e30)
> GET /uuid HTTP/2
> Host: admin.apisix.dev:9443
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 400
< date: Fri, 21 Apr 2023 07:53:23 GMT
< content-type: text/html; charset=utf-8
< content-length: 229
< server: APISIX/3.2.0
<
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>openresty</center>
<p><em>Powered by <a href="https://apisix.apache.org/">APISIX</a>.</em></p></body>
</html>
* Connection #0 to host admin.apisix.dev left intact

#
# although the client certificate is missing, but the URI is in the whitelist,
# you get successful response.
#
curl https://admin.apisix.dev:9443/anything/foobar -i \
--resolve 'admin.apisix.dev:9443:127.0.0.1' --cacert t/certs/mtls_ca.crt
HTTP/2 200
content-type: application/json
content-length: 416
date: Fri, 21 Apr 2023 07:58:28 GMT
access-control-allow-origin: *
access-control-allow-credentials: true
server: APISIX/3.2.0
...
```

## Conclusion

If you don't want to use curl or test on windows, you can read this gist for more details. [APISIX mTLS for client to APISIX](https://gist.github.com/bzp2010/6ce0bf7c15c191029ed54724547195b4).
Expand Down
1 change: 1 addition & 0 deletions docs/zh/latest/admin-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1168,6 +1168,7 @@ SSL 资源请求地址:/apisix/admin/ssls/{id}
| keys | 否 | 私钥字符串数组 | `certs` 对应的证书私钥,需要与 `certs` 一一对应。 | |
| client.ca | 否 | 证书 | 设置将用于客户端证书校验的 `CA` 证书。该特性需要 OpenResty 为 1.19 及以上版本。 | |
| client.depth | 否 | 辅助 | 设置客户端证书校验的深度,默认为 1。该特性需要 OpenResty 为 1.19 及以上版本。 | |
| client.skip_mtls_uri_regex | 否 | PCRE正则表达式数组 | 用来匹配请求的 URI,如果匹配,则该请求将绕过客户端证书的检查,也就是跳过 MTLS。 | ["/hello[0-9]+", "/foobar"] |
| snis | 是 | 匹配规则 | 非空数组形式,可以匹配多个 SNI。 | |
| labels | 否 | 匹配规则 | 标识附加属性的键值对。 | {"version":"v2","build":"16","env":"production"} |
| create_time | 否 | 辅助 | epoch 时间戳,单位为秒。如果不指定则自动创建。 | 1602883670 |
Expand Down
116 changes: 116 additions & 0 deletions docs/zh/latest/tutorials/client-to-apisix-mtls.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,122 @@ curl --resolve "test.com:9443:127.0.0.1" https://test.com:9443/anything -k --cer

由于我们在示例中配置了 `proxy-rewrite` 插件,我们可以看到响应体中包含上游收到的请求体,包含了正确数据。

## 基于对 URI 正则表达式匹配,绕过 MTLS

APISIX 允许配置 URI 白名单以便绕过 MTLS。如果请求的 URI 在白名单内,客户端证书将不被检查。注意,如果针对白名单外的 URI 发请求,而该请求缺乏客户端证书或者提供了非法客户端证书,会得到 HTTP 400 响应,而不是在 SSL 握手阶段被拒绝。

### 时序图

![skip mtls](https://raw.githubusercontent.com/apache/apisix/master/docs/assets/images/skip-mtls.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto


### 例子

```bash
monkeyDluffy6017 marked this conversation as resolved.
Show resolved Hide resolved
curl http://127.0.0.1:9180/apisix/admin/routes/1 \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"uri": "/*",
"upstream": {
"nodes": {
"httpbin.org": 1
}
}
}'

curl http://127.0.0.1:9180/apisix/admin/ssls/1 \
-H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
"cert": "'"$(<t/certs/mtls_server.crt)"'",
"key": "'"$(<t/certs/mtls_server.key)"'",
"snis": [
"*.apisix.dev"
],
"client": {
"ca": "'"$(<t/certs/mtls_ca.crt)"'",
"depth": 10,
"skip_mtls_uri_regex": [
"/anything.*"
]
}
}'

#
# 如果没提供客户端证书,而 URI 又不在白名单内,会得到 HTTP 400 响应。
#
curl https://admin.apisix.dev:9443/uuid -v \
--resolve 'admin.apisix.dev:9443:127.0.0.1' --cacert t/certs/mtls_ca.crt
* Added admin.apisix.dev:9443:127.0.0.1 to DNS cache
* Hostname admin.apisix.dev was found in DNS cache
* Trying 127.0.0.1:9443...
* TCP_NODELAY set
* Connected to admin.apisix.dev (127.0.0.1) port 9443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: t/certs/mtls_ca.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=cn; ST=GuangDong; L=ZhuHai; CN=admin.apisix.dev; OU=ops
* start date: Dec 1 10:17:24 2022 GMT
* expire date: Aug 18 10:17:24 2042 GMT
* subjectAltName: host "admin.apisix.dev" matched cert's "admin.apisix.dev"
* issuer: C=cn; ST=GuangDong; L=ZhuHai; CN=ca.apisix.dev; OU=ops
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x56246de24e30)
> GET /uuid HTTP/2
> Host: admin.apisix.dev:9443
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 400
< date: Fri, 21 Apr 2023 07:53:23 GMT
< content-type: text/html; charset=utf-8
< content-length: 229
< server: APISIX/3.2.0
<
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>openresty</center>
<p><em>Powered by <a href="https://apisix.apache.org/">APISIX</a>.</em></p></body>
</html>
* Connection #0 to host admin.apisix.dev left intact

#
# 虽然没提供客户端证书,但是 URI 在白名单内,请求会被成功处理和响应。
#
curl https://admin.apisix.dev:9443/anything/foobar -i \
--resolve 'admin.apisix.dev:9443:127.0.0.1' --cacert t/certs/mtls_ca.crt
HTTP/2 200
content-type: application/json
content-length: 416
date: Fri, 21 Apr 2023 07:58:28 GMT
access-control-allow-origin: *
access-control-allow-credentials: true
server: APISIX/3.2.0
...
```

## 总结

想了解更多有关 Apache APISIX 的 mTLS 功能介绍,可以阅读:[TLS 双向认证](../mtls.md)。
Loading