sklearn-mnist inference endpoint test timeout and failed

[Devinwong]

Hi, I've been following this https://github.com/Azure/AML-Kubernetes/blob/master/docs/simple-flow.md to setup and deploy a sample sklearn-mnist inference endpoint.

I've tried on 3 different configurations.

1. ✅AKS -> works as expected
2. ✅AKS connected to Arc -> works as expected (attached the compute target as Arc cluster, not AKS cluster, in order to test Arc route)
3. ❌minikube local cluster on WSL Ubuntu-20.04 (with Win11, MS working machine, connected to MS VPN) connected to Arc -> deployment succeeded. But when trying to test the endpoint with AML Studio UI, it hit timeout and eventually failed.

Here are more details for case 3.

Collected the following kubectl logs. kubectl_logs.txt

1. kubectl get pods -A => all the pods are running
2. kubectl logs blue-sklearn-mnist-minikube-58f5fd4cf7-lgvmf => the pod for inference runtime is running without errors.
3. az ml online-endpoint invoke -
   n sklearn-mnist-minikube -r sample-request.json => using the CLI to test the endpoint also timeout, like the AML studio UI test timeout.

ps. Created a local endpoint and tested with command

It was classified as 8 correctly.

What could be wrong with case 3?

[zetiaatgithub]

Sorry for the inconvenient. From the logs you shared, the scoring endpoint is "http://10.106.30.249/api/v1/endpoint/sklearn-mnist-minikube/score" and the error is "Connection to 10.106.30.249 timed out". So, it's a network connection issue. Could you please check if "10.106.30.249" is accessible from your local environment?