Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add GPU Support for k8s-model-server on Kubeflow #194

Closed
bkungfoo opened this issue Feb 1, 2018 · 5 comments
Closed

Add GPU Support for k8s-model-server on Kubeflow #194

bkungfoo opened this issue Feb 1, 2018 · 5 comments
Labels
area/inference
Milestone
Kubecon Europe

Comments

@bkungfoo
Copy link

bkungfoo commented Feb 1, 2018

As a feature request, adding GPU support for TF-serving in Kubeflow can help speed up batch predictions that can take a long time to run on a CPU-only machine. Unfortunately, existing docker images, such as the one from the TF-serving repository, do not build correctly (or are not well-maintained): https://github.com/tensorflow/serving/blob/master/tensorflow_serving/tools/docker/Dockerfile.devel-gpu

It would be good to have kubeflow maintain a working GPU docker image that can be deployed with the k8s-model-server.
@DjangoPeng
Copy link
Member

Cool! It's very helpful to speed up the inferencing and prediction work.

@jlewi
Copy link
Contributor

jlewi commented Feb 2, 2018

Also related #50

/cc @Fix42

@flx42
Copy link

flx42 commented Feb 2, 2018

It's @flx42 :)

@DjangoPeng DjangoPeng mentioned this issue Feb 6, 2018
Merged
@jlewi jlewi added this to the Kubecon Europe milestone Feb 10, 2018
@jlewi
Copy link
Contributor

jlewi commented Feb 10, 2018

As part of this issue we should create an Argo workflow to build and test this Docker image as in #214.

jlewi pushed a commit that referenced this issue Feb 16, 2018
@DjangoPeng @jlewi
feat(model-server): add Dockerfile of model-server with gpu support (#…
e8704c8 
…210)

Related to #194
    * Automated building and E2E testing will be added in a follow on PR.

How to test
You could train a mnist model with https://github.com/tensorflow/serving/blob/master/tensorflow_serving/example/mnist_saved_model.py.

Then mount the mnist saved model into tensorflow model server container and launch the gRPC server with mnist model.

Finally, you could launch a gRPC client with https://github.com/tensorflow/serving/blob/master/tensorflow_serving/example/mnist_client.py.
@jlewi
Copy link
Contributor

jlewi commented Feb 24, 2018

Opened up:

#291 E2E test for TFServing with GPUs.
#292 Add GPU support to ksonnet packages.

@jlewi jlewi closed this as completed Feb 24, 2018
yanniszark pushed a commit to arrikto/kubeflow that referenced this issue Feb 15, 2021
@YujiOshima @k8s-ci-robot
Fix CI (kubeflow#194)
0bfa23b 
* add -o xtrace

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>

* use client-cert instead password

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>

* delete get-credentials

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>

* delete unnecessary line

Signed-off-by: YujiOshima <yuji.oshima0x3fd@gmail.com>
@dchourasia dchourasia mentioned this issue Jul 29, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/inference
Projects
None yet
Milestone
Kubecon Europe
Development

No branches or pull requests
4 participants
@jlewi @flx42 @bkungfoo @DjangoPeng