Add the vendor directory to the repository. #222

zjj2wry · 2017-12-14T03:39:24Z

This change is

add vendor package into project, golang
- Checking in vendor is a common convention. For example, a variety of projects in kubernetes check in vendor.
- This should help developers by allowing them to skip updating the deps and doing the manual step of fixing some of the dependencies.
remove subpackage vendor from .gitignore

zjj2wry · 2017-12-14T05:31:50Z

/test tf-k8s-presubmit

zjj2wry · 2017-12-14T08:01:55Z

/test tf-k8s-presubmit

zjj2wry · 2017-12-14T08:41:41Z

@jlewi Do you know this CI status, seems i pass all test, but status is failed. thanks

jlewi · 2017-12-14T19:37:43Z

Examples of checking in vendor
https://github.com/kubernetes/kubernetes
https://github.com/kubernetes/client-go/tree/master/vendor
https://github.com/ksonnet/ksonnet
https://github.com/kubernetes/test-infra

Some counter examples
https://github.com/grpc/grpc-go
https://github.com/GoogleCloudPlatform/google-cloud-go
https://github.com/google/glog
https://github.com/kubernetes/helm

It looks like our total repo size is ~125 MB and ~110 MB of this vendor.
But you have to download all 125 MB if you want to fully build it; so a smaller repo size doesn't really make cloning and building the code any faster.

So following the K8s convention and checking in vendor makes sense to me.

I'll investigate the test failure.

jlewi · 2017-12-14T19:48:16Z

Looks like there's a bug in how we report failures and failed steps in our pipeline are not being reported to caicloud.

To see the actual error you'd have to look at Airflow and get the logs. You don't have permission for that but that's something we should fix.

Here's the error; looks like an issue with some dependencies not being found.

[2017-12-14 08:29:19,475] {base_task_runner.py:98} INFO - Subtask: INFO|2017-12-14T08:29:18|/opt/tensorflow_k8s/py/util.py|36| Running: go install github.com/tensorflow/k8s/cmd/tf_operator 
[2017-12-14 08:29:19,475] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2017-12-14 08:29:19,475] {base_task_runner.py:98} INFO - Subtask: INFO|2017-12-14T08:29:19|/opt/tensorflow_k8s/py/util.py|69| Subprocess output:
[2017-12-14 08:29:19,476] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/github.com/howeyc/gopass/terminal.go:5:8: cannot find package "golang.org/x/crypto/ssh/terminal" in any of:
[2017-12-14 08:29:19,476] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/crypto/ssh/terminal (vendor tree)
[2017-12-14 08:29:19,476] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/crypto/ssh/terminal (from $GOROOT)
[2017-12-14 08:29:19,477] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/crypto/ssh/terminal (from $GOPATH)
[2017-12-14 08:29:19,477] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/k8s.io/client-go/plugin/pkg/client/auth/gcp/gcp.go:30:2: cannot find package "golang.org/x/net/context" in any of:
[2017-12-14 08:29:19,478] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/net/context (vendor tree)
[2017-12-14 08:29:19,478] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/net/context (from $GOROOT)
[2017-12-14 08:29:19,478] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/net/context (from $GOPATH)
[2017-12-14 08:29:19,479] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/k8s.io/apimachinery/pkg/util/net/http.go:33:2: cannot find package "golang.org/x/net/http2" in any of:
[2017-12-14 08:29:19,479] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/net/http2 (vendor tree)
[2017-12-14 08:29:19,480] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/net/http2 (from $GOROOT)
[2017-12-14 08:29:19,480] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/net/http2 (from $GOPATH)
[2017-12-14 08:29:19,480] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/github.com/PuerkitoBio/purell/purell.go:17:2: cannot find package "golang.org/x/net/idna" in any of:
[2017-12-14 08:29:19,481] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/net/idna (vendor tree)
[2017-12-14 08:29:19,481] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/net/idna (from $GOROOT)
[2017-12-14 08:29:19,482] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/net/idna (from $GOPATH)
[2017-12-14 08:29:19,483] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/k8s.io/client-go/plugin/pkg/client/auth/gcp/gcp.go:31:2: cannot find package "golang.org/x/oauth2" in any of:
[2017-12-14 08:29:19,483] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/oauth2 (vendor tree)
[2017-12-14 08:29:19,483] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/oauth2 (from $GOROOT)
[2017-12-14 08:29:19,484] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/oauth2 (from $GOPATH)
[2017-12-14 08:29:19,484] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/k8s.io/client-go/plugin/pkg/client/auth/gcp/gcp.go:32:2: cannot find package "golang.org/x/oauth2/google" in any of:
[2017-12-14 08:29:19,485] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/oauth2/google (vendor tree)
[2017-12-14 08:29:19,485] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/oauth2/google (from $GOROOT)
[2017-12-14 08:29:19,485] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/oauth2/google (from $GOPATH)
[2017-12-14 08:29:19,488] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/github.com/PuerkitoBio/purell/purell.go:18:2: cannot find package "golang.org/x/text/secure/precis" in any of:
[2017-12-14 08:29:19,489] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/text/secure/precis (vendor tree)
[2017-12-14 08:29:19,489] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/text/secure/precis (from $GOROOT)
[2017-12-14 08:29:19,489] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/text/secure/precis (from $GOPATH)
[2017-12-14 08:29:19,490] {base_task_runner.py:98} INFO - Subtask: /var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/github.com/PuerkitoBio/purell/purell.go:19:2: cannot find package "golang.org/x/text/unicode/norm" in any of:
[2017-12-14 08:29:19,490] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/github.com/tensorflow/k8s/vendor/golang.org/x/text/unicode/norm (vendor tree)
[2017-12-14 08:29:19,490] {base_task_runner.py:98} INFO - Subtask: 	/usr/local/go/src/golang.org/x/text/unicode/norm (from $GOROOT)
[2017-12-14 08:29:19,491] {base_task_runner.py:98} INFO - Subtask: 	/var/lib/data/runs/tf_k8s_tests/2017-12-14T08_29_02/go/src/golang.org/x/text/unicode/norm (from $GOPATH)

zjj2wry · 2017-12-17T15:56:22Z

@jlewi thanks, this log very helpful.

zjj2wry · 2017-12-17T16:27:53Z

@jlewi golang.com/golang/* is deprecated, i update it to golang.org/x/*. this can work, ptal

add -v flag in glide install ignore subpackage vendor

21045bf

zjj2wry changed the title ~~Rep dependency~~ remove subpackage vendor Dec 14, 2017

zjj2wry force-pushed the rep_dependency branch from d186f4f to 21045bf Compare December 14, 2017 08:19

add vendor package

77cde29

jlewi mentioned this pull request Dec 14, 2017

Test failures aren't properly reported in Gubernator #229

Closed

update 'golang.com/golang/*' to 'golang.org/x/*' in dependency

36ea7ec

jlewi approved these changes Dec 18, 2017

View reviewed changes

jlewi changed the title ~~remove subpackage vendor~~ Add the vendor directory to the repository. Dec 18, 2017

jlewi merged commit c8bcb9d into kubeflow:master Dec 18, 2017

jose5918 mentioned this pull request Feb 1, 2018

Releaser incorrectly tags images as "dirty" #321

Closed

jlewi mentioned this pull request Apr 6, 2018

Create a POC of an app to simplify deployment of Kubeflow kubeflow/kubeflow#595

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the vendor directory to the repository. #222

Add the vendor directory to the repository. #222

zjj2wry commented Dec 14, 2017 •

edited by jlewi

Loading

zjj2wry commented Dec 14, 2017

zjj2wry commented Dec 14, 2017

zjj2wry commented Dec 14, 2017

jlewi commented Dec 14, 2017

jlewi commented Dec 14, 2017

zjj2wry commented Dec 17, 2017

zjj2wry commented Dec 17, 2017

Add the vendor directory to the repository. #222

Add the vendor directory to the repository. #222

Conversation

zjj2wry commented Dec 14, 2017 • edited by jlewi Loading

zjj2wry commented Dec 14, 2017

zjj2wry commented Dec 14, 2017

zjj2wry commented Dec 14, 2017

jlewi commented Dec 14, 2017

jlewi commented Dec 14, 2017

zjj2wry commented Dec 17, 2017

zjj2wry commented Dec 17, 2017

zjj2wry commented Dec 14, 2017 •

edited by jlewi

Loading