forked from kubeflow/pipelines
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Request Logging in Control Plane * change executor to logger in release scripts * change executor into logger * initial inference logger commit * initial Q version of inference logger * Add initial cloudevents code * add cloudevents and opencensus vendor additions * update cloudevents and create example * add log_type and sample * Add logging to transformers and explainers * Add validation tests * Allow default log url to be knative default broker * Bring inference logger annotations into single method * add model_url and update examples * refactor to logger * Update logger Dockerfile * Fix compile errors after rebase * Logger files, and test * Remove sample and add modelId * Updates from review * Updates from review * Updates from testing * Update from review comments * update workflow to replace executor with logger * regenerate tf2openapi * Updates from review * review updates * Updates from review
- Loading branch information
1 parent
a5f1eff
commit cfd3905
Showing
41 changed files
with
1,190 additions
and
68 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| # Inference Logger Demo | ||
|
|
||
| First let us create a message dumper KNative service which will print out the Cloud Events it receives. | ||
| We will use the following resource yaml: | ||
|
|
||
| ``` | ||
| apiVersion: serving.knative.dev/v1alpha1 | ||
| kind: Service | ||
| metadata: | ||
| name: message-dumper | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - image: gcr.io/knative-releases/github.com/knative/eventing-sources/cmd/event_display | ||
| ``` | ||
|
|
||
| Let's apply the resource to the cluster: | ||
|
|
||
| ``` | ||
| kubectl create -f message-dumper.yaml | ||
| ``` | ||
|
|
||
| We can now create a sklearn predictor with a logger which points at the message dumper. The yaml is shown below. | ||
|
|
||
| ``` | ||
| apiVersion: "serving.kubeflow.org/v1alpha2" | ||
| kind: "InferenceService" | ||
| metadata: | ||
| name: "sklearn-iris" | ||
| spec: | ||
| default: | ||
| predictor: | ||
| minReplicas: 1 | ||
| logger: | ||
| url: http://message-dumper.default/ | ||
| mode: all | ||
| sklearn: | ||
| storageUri: "gs://kfserving-samples/models/sklearn/iris" | ||
| ``` | ||
|
|
||
| Let's apply this yaml: | ||
|
|
||
| ``` | ||
| kubectl create -f sklearn-logging.yaml | ||
| ``` | ||
|
|
||
| We can now send a request to the sklearn model: | ||
|
|
||
| ``` | ||
| MODEL_NAME=sklearn-iris | ||
| INPUT_PATH=@./iris-input.json | ||
| CLUSTER_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}') | ||
| SERVICE_HOSTNAME=$(kubectl get inferenceservice sklearn-iris -o jsonpath='{.status.url}' | cut -d "/" -f 3) | ||
| curl -v -H "Host: ${SERVICE_HOSTNAME}" http://$CLUSTER_IP/v1/models/$MODEL_NAME:predict -d $INPUT_PATH -H "Content-Type: application/json" | ||
| ``` | ||
|
|
||
| Expected Output | ||
|
|
||
| ``` | ||
| * Trying 169.63.251.68... | ||
| * TCP_NODELAY set | ||
| * Connected to 169.63.251.68 (169.63.251.68) port 80 (#0) | ||
| > POST /models/sklearn-iris:predict HTTP/1.1 | ||
| > Host: sklearn-iris.default.svc.cluster.local | ||
| > User-Agent: curl/7.60.0 | ||
| > Accept: */* | ||
| > Content-Length: 76 | ||
| > Content-Type: application/x-www-form-urlencoded | ||
| > | ||
| * upload completely sent off: 76 out of 76 bytes | ||
| < HTTP/1.1 200 OK | ||
| < content-length: 23 | ||
| < content-type: application/json; charset=UTF-8 | ||
| < date: Mon, 20 May 2019 20:49:02 GMT | ||
| < server: istio-envoy | ||
| < x-envoy-upstream-service-time: 1943 | ||
| < | ||
| * Connection #0 to host 169.63.251.68 left intact | ||
| {"predictions": [1, 1]} | ||
| ``` | ||
|
|
||
| If we now check the logs of the message dumper: | ||
|
|
||
| ``` | ||
| kubectl logs $(kubectl get pod -l serving.knative.dev/service=message-dumper -o jsonpath='{.items[0].metadata.name}') user-container | ||
| ``` | ||
|
|
||
| Expected output: | ||
|
|
||
| ``` | ||
| ☁️ cloudevents.Event | ||
| Validation: valid | ||
| Context Attributes, | ||
| cloudEventsVersion: 0.1 | ||
| eventType: org.kubeflow.serving.inference.request | ||
| source: http://localhost:8080/ | ||
| eventID: 462af46b-d582-4f3a-9f2a-6851d4143e3d | ||
| eventTime: 2019-10-21T12:12:49.82518115Z | ||
| contentType: application/json | ||
| Data, | ||
| { | ||
| "instances": [ | ||
| [ | ||
| 6.8, | ||
| 2.8, | ||
| 4.8, | ||
| 1.4 | ||
| ], | ||
| [ | ||
| 6.0, | ||
| 3.4, | ||
| 4.5, | ||
| 1.6 | ||
| ] | ||
| ] | ||
| } | ||
| ☁️ cloudevents.Event | ||
| Validation: valid | ||
| Context Attributes, | ||
| cloudEventsVersion: 0.1 | ||
| eventType: org.kubeflow.serving.inference.response | ||
| source: http://localhost:8080/ | ||
| eventID: 462af46b-d582-4f3a-9f2a-6851d4143e3d | ||
| eventTime: 2019-10-21T12:12:49.83269988Z | ||
| contentType: application/json; charset=UTF-8 | ||
| Data, | ||
| { | ||
| "predictions": [ | ||
| 1, | ||
| 1 | ||
| ] | ||
| } | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "instances": [ | ||
| [6.8, 2.8, 4.8, 1.4], | ||
| [6.0, 3.4, 4.5, 1.6] | ||
| ] | ||
| } |
Oops, something went wrong.