Add support to deploy SKLearn and XGBoost models with MLServer #2450

adriangonz · 2020-09-17T14:52:59Z

What this PR does / why we need it:

Add support to deploy SKLearn and XGBoost models compatible with the V2 Dataplane proposed by NVIDIA / KFServing, by leveraging MLServer.

On most cases, the user will be able to enable the V2 dataplane by specifying kfserving as the protocol on their SKLEARN_SERVER and XGBOOST_SERVER deployments. For example, one can do:

apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: xgboost
spec:
  name: iris
  protocol: kfserving
  predictors:
  - graph:
      children: []
      implementation: XGBOOST_SERVER
      modelUri: gs://seldon-models/xgboost/iris
      name: classifier
    name: default
    replicas: 1

To Do

~~Add examples with V2 protocol~~ (moved to Add SKLearn and XGBoost examples for MLServer / V2 Dataplane #2479)

Which issue(s) this PR fixes:

Fixes #1648

Special notes for your reviewer:

This PR is based on #2023. Once that one gets merged, I'll rebase this one to master to clean up Git's history.

Does this PR introduce a user-facing change?:

Added support to use the V2 Dataplane with SKLearn and XGBoost models.

review-notebook-app · 2020-09-17T14:53:04Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

adriangonz · 2020-09-17T14:54:16Z

/cc @cliveseldon @RafalSkolasinski @axsaucedo

There are still a few details left on this PR, like rebasing once #2023 gets merged, but I think that the rest is ready for review. It would be great to get your thoughts!

seldondev · 2020-09-22T09:25:01Z

Tue Sep 22 09:24:59 UTC 2020
The logs for [pr-build] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=3

seldondev · 2020-09-22T09:25:01Z

Tue Sep 22 09:25:00 UTC 2020
The logs for [lint] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=4

seldondev · 2020-09-28T10:39:09Z

Mon Sep 28 10:39:06 UTC 2020
The logs for [integration] [15] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/15.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=15

seldondev · 2020-09-28T10:39:20Z

Mon Sep 28 10:39:19 UTC 2020
The logs for [pr-build] [13] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/13.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=13

seldondev · 2020-09-28T10:39:22Z

Mon Sep 28 10:39:21 UTC 2020
The logs for [lint] [14] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/14.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=14

adriangonz · 2020-09-28T11:03:34Z

/test lint
/test pr-build

seldondev · 2020-09-28T11:04:56Z

Mon Sep 28 11:04:55 UTC 2020
The logs for [lint] [16] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/16.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=16

adriangonz · 2020-09-28T13:49:51Z

/retest

seldondev · 2020-09-28T13:51:57Z

Mon Sep 28 13:51:56 UTC 2020
The logs for [pr-build] [17] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/17.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=17

seldondev · 2020-09-28T13:52:00Z

Mon Sep 28 13:51:59 UTC 2020
The logs for [integration] [18] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/18.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=18

adriangonz · 2020-09-29T09:38:38Z

/test integration

seldondev · 2020-09-29T09:40:22Z

Tue Sep 29 09:40:19 UTC 2020
The logs for [integration] [21] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/21.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=21

seldondev · 2020-09-29T09:40:26Z

Tue Sep 29 09:40:24 UTC 2020
The logs for [lint] [20] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/20.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=20

seldondev · 2020-09-29T09:40:36Z

Tue Sep 29 09:40:34 UTC 2020
The logs for [pr-build] [19] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/19.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=19

adriangonz · 2020-09-29T10:49:43Z

/test integration

seldondev · 2020-09-29T10:52:10Z

Tue Sep 29 10:52:05 UTC 2020
The logs for [integration] [24] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/24.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=24

seldondev · 2020-09-29T10:52:14Z

Tue Sep 29 10:52:11 UTC 2020
The logs for [lint] [23] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/23.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=23

seldondev · 2020-09-29T10:52:15Z

Tue Sep 29 10:52:13 UTC 2020
The logs for [pr-build] [22] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/22.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=22

adriangonz · 2020-09-29T13:20:31Z

/test notebooks

seldondev · 2020-09-29T13:22:28Z

Tue Sep 29 13:22:27 UTC 2020
The logs for [notebooks] [25] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/25.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=25

RafalSkolasinski

Nice one, solid work!

seldondev · 2020-09-29T14:54:20Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RafalSkolasinski

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [RafalSkolasinski]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

adriangonz · 2020-09-30T13:31:52Z

/test notebooks

seldondev · 2020-09-30T13:33:32Z

Wed Sep 30 13:33:31 UTC 2020
The logs for [notebooks] [26] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/26.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=26

seldondev · 2020-09-30T15:44:02Z

Wed Sep 30 15:43:56 UTC 2020
The logs for [pr-build] [27] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/27.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=27

seldondev · 2020-09-30T15:44:19Z

Wed Sep 30 15:44:18 UTC 2020
The logs for [lint] [28] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-2450/28.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-2450 --build=28

seldondev added do-not-merge/work-in-progress size/XXL labels Sep 17, 2020

seldondev requested review from axsaucedo, ukclivecox and RafalSkolasinski September 17, 2020 14:54

FogDong mentioned this pull request Sep 20, 2020

Triton server support with kfserving protocol #2460

Closed

adriangonz force-pushed the add-mlserver branch from aff0b4f to 7a8e9f2 Compare September 22, 2020 09:23

Adrian Gonzalez-Martin added 19 commits September 23, 2020 10:11

Start adding MLServer pre-packaged server

d78b2a3

Add tests for env vars

da77cef

Add optional protocols entry to prepack configmap

c9d0f02

Update fixture configmap and add tests for image

ccaa288

Add test for container

3f88345

Build container for mlserver

9c0065e

Add integration test for mlserver deployments

fd84f96

Add deepcopy methods for protocolsConfig

e6a7af9

Fix tests for mlserver

ce8d564

Fix conversion from int32 to string

8d47b89

Add model name and version

5889a1b

Add mlserver image to configmap

083b180

Add some sample resources

1810330

Add test for v2 sklearn server

c7cfa16

Set model version to v1 by default

1d6ffe3

Add helper method to PredictorServerConfig to extract the image

2148510

Use new helper method in defaulter

1e5c9a8

Add helper method to inject mlserver params into an existing container

00474d9

Add test for xgboost

64ee9fb

Add mlserver image to XGBoost config

816c5e5

Handle case where configmap is not ready yet

0c76496

RafalSkolasinski approved these changes Sep 29, 2020

View reviewed changes

seldondev assigned RafalSkolasinski Sep 29, 2020

seldondev added the lgtm label Sep 29, 2020

seldondev added the approved label Sep 29, 2020

seldondev merged commit fc4904b into SeldonIO:master Sep 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support to deploy SKLearn and XGBoost models with MLServer #2450

Add support to deploy SKLearn and XGBoost models with MLServer #2450

adriangonz commented Sep 17, 2020 •

edited

Loading

review-notebook-app bot commented Sep 17, 2020

adriangonz commented Sep 17, 2020

seldondev commented Sep 22, 2020

seldondev commented Sep 22, 2020

seldondev commented Sep 28, 2020

seldondev commented Sep 28, 2020

seldondev commented Sep 28, 2020

adriangonz commented Sep 28, 2020

seldondev commented Sep 28, 2020

adriangonz commented Sep 28, 2020

seldondev commented Sep 28, 2020

seldondev commented Sep 28, 2020

adriangonz commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

adriangonz commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

adriangonz commented Sep 29, 2020

seldondev commented Sep 29, 2020

RafalSkolasinski left a comment

seldondev commented Sep 29, 2020

adriangonz commented Sep 30, 2020

seldondev commented Sep 30, 2020

seldondev commented Sep 30, 2020

seldondev commented Sep 30, 2020

Add support to deploy SKLearn and XGBoost models with MLServer #2450

Add support to deploy SKLearn and XGBoost models with MLServer #2450

Conversation

adriangonz commented Sep 17, 2020 • edited Loading

review-notebook-app bot commented Sep 17, 2020

adriangonz commented Sep 17, 2020

seldondev commented Sep 22, 2020

seldondev commented Sep 22, 2020

seldondev commented Sep 28, 2020

seldondev commented Sep 28, 2020

seldondev commented Sep 28, 2020

adriangonz commented Sep 28, 2020

seldondev commented Sep 28, 2020

adriangonz commented Sep 28, 2020

seldondev commented Sep 28, 2020

seldondev commented Sep 28, 2020

adriangonz commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

adriangonz commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

seldondev commented Sep 29, 2020

adriangonz commented Sep 29, 2020

seldondev commented Sep 29, 2020

RafalSkolasinski left a comment

Choose a reason for hiding this comment

seldondev commented Sep 29, 2020

adriangonz commented Sep 30, 2020

seldondev commented Sep 30, 2020

seldondev commented Sep 30, 2020

seldondev commented Sep 30, 2020

adriangonz commented Sep 17, 2020 •

edited

Loading