kong runtime not effective immediately after kic update the config by calling /config?hash_check=1

[wuzhuoquan]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 2.4.1

Current Behavior

We use kong and KIC in k8s cluster, they works well mostly.
But sometimes, we found that after we scale down the service deployment replicas, that means some pods are deleted and remove from the service endpoint, KIC call the /config API of kong to modify the configurations in kong, after that, we found that kong runtime not effect the configuration immediately, and still send request to the deleted pod, and we found some Connection problem.

E.G.
12:28:15 the pod with ip 172.21.143.79 is removed from endpoint in k8s

but kong still send request the this ip address in 12:29:34, that means this pod's ip is not remove from the upstream list yet.

Expected Behavior

after the pod's ip has been removed from the endpoint, kong should not sent request to this pod any more.

Steps To Reproduce

It's not easy to reproduce the problem, This problem occurs randomly.

Anything else?

No response

[StarlightIbuki]

@randmonkey Could you take a look?

[randmonkey]

@wuzhuoquan Please confirm the version of Kong and KIC.
We have fixed a bug where KIC would send config to removed gateway instances(pods) in KIC 3.2. Please try that version and confirm if the issue is fixed.

[wuzhuoquan]

@wuzhuoquan Please confirm the version of Kong and KIC. We have fixed a bug where KIC would send config to removed gateway instances(pods) in KIC 3.2. Please try that version and confirm if the issue is fixed.

KIC version is 1.3.1