Pod has no outbound connection on 0.9.4 + RS3 Windows

[yuedai]

Is this a request for help?:
yes

---

Is this an ISSUE or FEATURE REQUEST? (choose one):
Issue

What version of acs-engine?:
0.9.4

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
Kubernetes

What happened:
I use 0.9.4 to provision a windows cluster, with RS3 windows. I created a deployment and the pod could be started successfully. But out-going call are all failed on pod.
e.g. I "kubectl exec" to one pod and try to nslookup, dns server is totally not reachable.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):
This is my API model:

{
"apiVersion": "vlabs",
"properties": {
"orchestratorProfile": {
"orchestratorType": "Kubernetes",
"orchestratorRelease": "1.8",
"orchestratorVersion": "1.8.2",
"kubernetesConfig": {
"kubernetesImageBase": "gcrio.azureedge.net/google_containers/",
"clusterSubnet": "10.244.0.0/16",
"dnsServiceIP": "10.0.0.10",
"serviceCidr": "10.0.0.0/16",
"networkPolicy": "none",
"nonMasqueradeCidr": "10.0.0.0/8",
"maxPods": 110,
"dockerBridgeSubnet": "172.17.0.1/16",
"nodeStatusUpdateFrequency": "10s",
"ctrlMgrNodeMonitorGracePeriod": "40s",
"ctrlMgrPodEvictionTimeout": "5m0s",
"ctrlMgrRouteReconciliationPeriod": "10s",
"gchighthreshold": 85,
"gclowthreshold": 80,
"etcdVersion": "2.2.5",
"etcdDiskSizeGB": "128",
"addons": [
{
"name": "tiller",
"enabled": true,
"containers": [
{
"name": "tiller",
"cpuRequests": "50m",
"memoryRequests": "150Mi",
"cpuLimits": "50m",
"memoryLimits": "150Mi"
}
]
},
{
"name": "kubernetes-dashboard",
"enabled": true,
"containers": [
{
"name": "kubernetes-dashboard",
"cpuRequests": "300m",
"memoryRequests": "150Mi",
"cpuLimits": "300m",
"memoryLimits": "150Mi"
}
]
}
]
}
},
"masterProfile": {
"count": 1,
"dnsPrefix": "cci-1709",
"vmSize": "Standard_D2_v2",
"osDiskSizeGB": 30,
"firstConsecutiveStaticIP": "10.240.255.5",
"storageProfile": "ManagedDisks",
"oauthEnabled": false,
"preProvisionExtension": null,
"extensions": [],
"distro": "ubuntu"
},
"agentPoolProfiles": [
{
"name": "cci1709",
"count": 1,
"vmSize": "Standard_D5_v2",
"osDiskSizeGB": 200,
"osType": "Windows",
"availabilityProfile": "AvailabilitySet",
"storageProfile": "ManagedDisks",
"distro": "ubuntu",
"fqdn": "",
"preProvisionExtension": null,
"extensions": []
},
{
"name": "side",
"count": 1,
"vmSize": "Standard_D3_v2",
"osDiskSizeGB": 30,
"osType": "Linux",
"availabilityProfile": "AvailabilitySet",
"storageProfile": "ManagedDisks",
"distro": "ubuntu",
"fqdn": "",
"preProvisionExtension": null,
"extensions": []
}
],
"linuxProfile": {
"adminUsername": "",
"ssh": {
"publicKeys": [
{
"keyData": ""
}
]
}
},
"windowsProfile": {
"adminUsername": "",
"adminPassword": ""
},
"servicePrincipalProfile": {
"clientId": "",
"secret": ""
},
"certificateProfile": {
"caCertificate": "",
"caPrivateKey": "",
"apiServerCertificate": "",
"apiServerPrivateKey": "",
"clientCertificate": "",
"clientPrivateKey": "",
"kubeConfigCertificate": "",
"kubeConfigPrivateKey": ""
}
}
}

Anything else we need to know:

[jackfrancis]

@JiangtianLi is does this issue overlap with work in #1810?

Or does it appear to be a transient issue where a cluster had no outbound access?

[JiangtianLi]

@yuedai What does Resolve-DnsName www.bing.com output in your container?

@jackfrancis This is a known random issue with Windows DNS and has been serviced to windows update.

[yuedai]

It's timeout. The Dns server (10.0.0.10) is not reachable. On another 2016 cluster (from 0.8.0 acs-engine) it works fine.

[JiangtianLi]

@yuedai Does the output from Resolve-DnsName say "No DNS servers configured for local system"? If so, it is the local DNS config issue, not the no response from DNS server issue. And it has been fixed in Windows. Will update when the fix is in acs-engine deployment.

[yuedai]

I think there is Dns configuration, but it’s just not reachable. I also tried to use kubectl exec to open a console and tried nslookup www.bing.com<http://www.bing.com>. The tool just returned a set of timeout. Also tracert 10.0.0.10 from the console failed as well. Is this a known issue? From: Jiangtian Li [mailto:notifications@github.com] Sent: 2017年12月6日 11:27 To: Azure/acs-engine <acs-engine@noreply.github.com> Cc: yuedai <daiyue@hotmail.com>; Mention <mention@noreply.github.com> Subject: Re: [Azure/acs-engine] Pod has no outbound connection on 0.9.4 + RS3 Windows (#1881) @yuedai<https://github.com/yuedai> Does the output from Resolve-DnsName say "No DNS servers configured for local system"? If so, it is the local DNS config issue, not the no response from DNS server issue. And it has been fixed in Windows. Will update when the fix is in acs-engine deployment. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1881 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABO5D1OzmHB6wlyPskkk6V5lTCgEaZS9ks5s9uqGgaJpZM4Q1ux7>.

[JiangtianLi]

@yuedai Although DNS server shows up in ipconfig /all inside windows container, there is a race condition to make the config actually failed. So if you use Resolve-DnsName and see "No DNS servers configured for local system" message, that'll confirm it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contribution. Note that acs-engine is deprecated--see https://github.com/Azure/aks-engine instead.