kong upstream healthcheck stop after a target(with hostname) return 404

[luguohong]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

kong 2.7.0

Current Behavior

when i upgrade kong to 2.7.0 from 2.2.0，i create a new service, route, upstream, and then the healthcheck's config as below:

{
	"hash_on_header": null,
	"host_header": null,
	"id": "ddaea45f-6c4e-451f-96fb-50aa204dad8e",
	"created_at": 1642749019,
	"hash_on_cookie": null,
	"hash_on_cookie_path": "/",
	"hash_fallback": "none",
	"hash_fallback_header": null,
	"name": "service.xxx.v1.0.0",
	"slots": 1000,
	"hash_on": "none",
	"healthchecks": {
		"active": {
			"type": "http",
			"timeout": 1,
			"http_path": "/console/demo/kong/test",
			"https_sni": null,
			"https_verify_certificate": true,
			"healthy": {
				"successes": 1,
				"http_statuses": [200, 302],
				"interval": 1
			},
			"concurrency": 10,
			"unhealthy": {
				"tcp_failures": 1,
				"timeouts": 1,
				"http_failures": 1,
				"interval": 3,
				"http_statuses": [429, 500, 501, 502, 503, 504, 505, 404]
			}
		},
		"passive": {
			"type": "http",
			"unhealthy": {
				"http_failures": 3,
				"http_statuses": [429, 500, 502, 503],
				"tcp_failures": 0,
				"timeouts": 0
			},
			"healthy": {
				"http_statuses": [200, 201, 202, 203, 204, 205, 206, 207, 208, 226, 300, 301, 302, 303, 304, 305, 306, 307, 308],
				"successes": 0
			}
		},
		"threshold": 0
	},
	"tags": null,
	"algorithm": "round-robin",
	"client_certificate": null
}

and the target is 'xxx.hostname.com:47893' ,Hostname is interpreted as 3 IP (as A records) (found through the dig command: dig xxx.hostname.com)

then i set kong log level to debug

at first, i see the healthcheck's log is normal(the target with healthcheck path return 404) and the target's health status is unhealthy as i hope, after 3 or 5 minutes, the healthcheck log stop and the target's health status switch to health!!!

then no matter what I set as target(ip or host, health or unhealth),the target show health,and the healthcheck log do not work

However, when I set IP as target, active healthcheck works properly

by the way:

1. i have searched existing issues and found 3 related issues
   Kong Marking unhealthy target healthy again on deletion of target or addition of new target
   Healthcheck stopped after update Upstream
   [healthchecks] failed setting peer status: no peer found by name and address
   Especially issue 7163
2. the target(xxx.hostname.com:47893) was set in other upstream(but other healthcheck has different http path so the target in other upstream is health), I don't know if this will affect each other

Expected Behavior

As mentioned above, I hope the active healthcheck can work normally when the target is hostname

Steps To Reproduce

As mentioned above
1. a new upstream（config is above）
2. set a target（hostname）
3. turn on active healthcheck（config is above）
4. maybe，should set the same target in other, and the other upstream's healthcheck must config the target to health

Anything else?

if want more infomation,please contact me

[locao]

Hi @luguohong! Can you please share some more details?

• Is xxx.hostname.com an SRV record?
• Are there any error level messages in the logs?
• Is the health checker unusable for any other target or only for xxx.hostname.com after the issue happens?

Thank you!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[luguohong]

@locao
Is xxx.hostname.com an SRV record?
the hostname is an cname record(hostname b)
the hostname b is A record
Are there any error level messages in the logs?
no other error msg.
and we turn the log level to debug, and found that no healthcheck log showed
Is the health checker unusable for any other target or only for xxx.hostname.com after the issue happens?
we just test a hostname, and a ip:port