Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Specify service account for kaniko jobs #352

Merged
merged 19 commits into from
Feb 24, 2023
Merged

Specify service account for kaniko jobs #352

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
c7adb69
Add capability to specify kaniko service account
shydefoo Feb 20, 2023
d2fdfc6
Add kaniko service account to chart
shydefoo Feb 20, 2023
8eea1f8
Modify batch predictor docker file
shydefoo Feb 21, 2023
ff92a6c
Add test for sa, fix linting
shydefoo Feb 21, 2023
427fba3
Merge remote-tracking branch 'upstream/main' into enable-wi-kaniko
shydefoo Feb 21, 2023
667c674
Add kaniko service account to helm chart
shydefoo Feb 21, 2023
be8e758
Increase memory to kserve
shydefoo Feb 21, 2023
2fc9733
Add kaniko service account template
shydefoo Feb 21, 2023
876c9df
Merge remote-tracking branch 'upstream/main' into enable-wi-kaniko
shydefoo Feb 22, 2023
1fe39d7
Fix merlin chart
shydefoo Feb 22, 2023
56ca8e2
Set kaniko sa name based on create boolean
shydefoo Feb 22, 2023
9af217c
Fix test case issue
shydefoo Feb 22, 2023
a13dcc0
Upgrade pip in e2e script
shydefoo Feb 22, 2023
59f1a32
Use python 3.10.6
shydefoo Feb 22, 2023
0442b78
Increase kserve memory overlay
shydefoo Feb 23, 2023
7f248d8
Reduce kserve memory requested
shydefoo Feb 23, 2023
c87b118
Remove changes to e2e test
shydefoo Feb 23, 2023
7ce63ac
Specify python version in e2e test
shydefoo Feb 23, 2023
d103439
Set pytest number of workers to 4
shydefoo Feb 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/merlin.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ jobs:
LOCAL_REGISTRY: "dev.localhost"
INGRESS_HOST: "127.0.0.1.nip.io"
MLP_CHART_VERSION: 0.3.4
E2E_PYTHON_VERSION: "3.10"
E2E_PYTHON_VERSION: "3.10.6"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version is specified to as leaving it as 3.10 gives this error: see this run https://github.com/gojek/merlin/actions/runs/4249958378/jobs/7390728519

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. @ariefrahmansyah I think it's got to do with how the CI use cache key https://github.com/gojek/merlin/blob/36c63d3775b5b08d518dde87d35385c5c8cb00e0/.github/workflows/merlin.yml#L365 , I am not sure why we use ${{ matrix.python-version }}, this task doesn't have matrix right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate more on why specifying E2E_PYTHON_VERSION to 3.10.6 solve the issue?

-> I am not sure why we use ${{ matrix.python-version }}, this task doesn't have matrix right?
Yeah, we should change the cache key to not using matrix.python-version, but still have to be different from other python jobs cache key.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure on this. This is the error that happens consistently when 3.10 is used:

Notice:  A new release of pip is available: 23.0 -> 23.0.1
Notice:  To update, run: pip install --upgrade pip
Installing dependencies from Pipfile...
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.10/x64/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/cli/options.py", line 56, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/decorators.py", line [84](https://github.com/gojek/merlin/actions/runs/4249958378/jobs/7390728519#step:14:85), in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/cli/command.py", line 233, in install
    do_install(
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/core.py", line 2064, in do_install
    do_init(
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/core.py", line 1333, in do_init
    do_install_dependencies(
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/core.py", line [86](https://github.com/gojek/merlin/actions/runs/4249958378/jobs/7390728519#step:14:87)1, in do_install_dependencies
    batch_install(
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/core.py", line 732, in batch_install
    deps_to_install = [
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/core.py", line 733, in <listcomp>
    dep for dep in deps_to_install if not project.environment.is_satisfied(dep)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/project.py", line 304, in environment
    self._environment = self.get_environment(allow_global=allow_global)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/project.py", line 2[90](https://github.com/gojek/merlin/actions/runs/4249958378/jobs/7390728519#step:14:91), in get_environment
    environment = Environment(
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/environment.py", line 79, in __init__
    self._base_paths = self.get_paths()
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/environment.py", line 390, in get_paths
    c = subprocess_run(command)
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pipenv/utils/processes.py", line 75, in subprocess_run
    return subprocess.run(
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/subprocess.py", line [97](https://github.com/gojek/merlin/actions/runs/4249958378/jobs/7390728519#step:14:98)1, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/subprocess.py", line 1847, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/.local/share/virtualenvs/sdk-ywGzPJ0F/bin/python'

There was a similar issue raised here, pypa/pipenv#5075, but the solution is not clear to me. Since the python version here is 3.10.10, i thought of specifying a fixed version and it seems to be able to install the dependencies.

steps:
- uses: actions/checkout@v2
with:
Expand Down
2 changes: 2 additions & 0 deletions api/cmd/api/setup.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,7 @@ func initImageBuilder(cfg *config.Config) (webserviceBuilder imagebuilder.ImageB
ContextSubPath: cfg.ImageBuilderConfig.ContextSubPath,
BuildTimeoutDuration: timeout,
KanikoImage: cfg.ImageBuilderConfig.KanikoImage,
KanikoServiceAccount: cfg.ImageBuilderConfig.KanikoServiceAccount,
Resources: cfg.ImageBuilderConfig.Resources,
Tolerations: cfg.ImageBuilderConfig.Tolerations,
NodeSelectors: cfg.ImageBuilderConfig.NodeSelectors,
Expand All @@ -158,6 +159,7 @@ func initImageBuilder(cfg *config.Config) (webserviceBuilder imagebuilder.ImageB
ContextSubPath: cfg.ImageBuilderConfig.PredictionJobContextSubPath,
BuildTimeoutDuration: timeout,
KanikoImage: cfg.ImageBuilderConfig.KanikoImage,
KanikoServiceAccount: cfg.ImageBuilderConfig.KanikoServiceAccount,
Resources: cfg.ImageBuilderConfig.Resources,
Tolerations: cfg.ImageBuilderConfig.Tolerations,
NodeSelectors: cfg.ImageBuilderConfig.NodeSelectors,
Expand Down
1 change: 1 addition & 0 deletions api/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@ type ImageBuilderConfig struct {
DockerRegistry string `envconfig:"IMG_BUILDER_DOCKER_REGISTRY"`
BuildTimeout string `envconfig:"IMG_BUILDER_TIMEOUT" default:"10m"`
KanikoImage string `envconfig:"IMG_BUILDER_KANIKO_IMAGE" default:"gcr.io/kaniko-project/executor:v1.6.0"`
KanikoServiceAccount string `envconfig:"IMG_BUILDER_KANIKO_SERVICE_ACCOUNT"`
Resources ResourceRequestsLimits `envconfig:"IMG_BUILDER_RESOURCES"`
// How long to keep the image building job resource in the Kubernetes cluster. Default: 2 days (48 hours).
Retention time.Duration `envconfig:"IMG_BUILDER_RETENTION" default:"48h"`
Expand Down
2 changes: 2 additions & 0 deletions api/pkg/imagebuilder/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ type Config struct {
BuildTimeoutDuration time.Duration
// Kaniko docker image
KanikoImage string
// Kaniko kubernetes service account
KanikoServiceAccount string
// Kubernetes resource request and limits for kaniko
Resources cfg.ResourceRequestsLimits
// Tolerations for Jobs Specification
Expand Down
71 changes: 43 additions & 28 deletions api/pkg/imagebuilder/imagebuilder.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,6 @@ func (c *imageBuilder) createKanikoJobSpec(project mlp.Project, model *models.Mo
fmt.Sprintf("--context=%s", baseImageTag.BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", version.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", baseImageTag.ImageName),
fmt.Sprintf("--build-arg=%s=%s", gacEnvKey, saFilePath),
fmt.Sprintf("--destination=%s", imageRef),
"--cache=true",
"--single-snapshot",
Expand All @@ -360,6 +359,37 @@ func (c *imageBuilder) createKanikoJobSpec(project mlp.Project, model *models.Mo
}

activeDeadlineSeconds := int64(c.config.BuildTimeoutDuration / time.Second)
var volume []v1.Volume
var volumeMount []v1.VolumeMount
var envVar []v1.EnvVar

// If kaniko service account is not set, use kaniko secret
if c.config.KanikoServiceAccount == "" {
kanikoArgs = append(kanikoArgs,
fmt.Sprintf("--build-arg=%s=%s", gacEnvKey, saFilePath))
volume = []v1.Volume{
{
Name: kanikoSecretName,
VolumeSource: v1.VolumeSource{
Secret: &v1.SecretVolumeSource{
SecretName: kanikoSecretName,
},
},
},
}
volumeMount = []v1.VolumeMount{
{
Name: kanikoSecretName,
MountPath: "/secret",
},
}
envVar = []v1.EnvVar{
{
Name: gacEnvKey,
Value: saFilePath,
},
}
}

resourceRequirements := RequestLimitResources{
Request: Resource{
Expand All @@ -380,7 +410,7 @@ func (c *imageBuilder) createKanikoJobSpec(project mlp.Project, model *models.Mo
},
}

return &batchv1.Job{
job := &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{
Name: kanikoPodName,
Namespace: c.config.BuildNamespace,
Expand All @@ -400,39 +430,24 @@ func (c *imageBuilder) createKanikoJobSpec(project mlp.Project, model *models.Mo
RestartPolicy: v1.RestartPolicyNever,
Containers: []v1.Container{
{
Name: containerName,
Image: c.config.KanikoImage,
Args: kanikoArgs,
VolumeMounts: []v1.VolumeMount{
{
Name: kanikoSecretName,
MountPath: "/secret",
},
},
Env: []v1.EnvVar{
{
Name: gacEnvKey,
Value: saFilePath,
},
},
Name: containerName,
Image: c.config.KanikoImage,
Args: kanikoArgs,
VolumeMounts: volumeMount,
Env: envVar,
Resources: resourceRequirements.Build(),
TerminationMessagePolicy: v1.TerminationMessageFallbackToLogsOnError,
},
},
Volumes: []v1.Volume{
{
Name: kanikoSecretName,
VolumeSource: v1.VolumeSource{
Secret: &v1.SecretVolumeSource{
SecretName: kanikoSecretName,
},
},
},
},
Volumes: volume,
Tolerations: c.config.Tolerations,
NodeSelector: c.config.NodeSelectors,
},
},
},
}, nil
}
if c.config.KanikoServiceAccount != "" {
job.Spec.Template.Spec.ServiceAccountName = c.config.KanikoServiceAccount
}
return job, nil
}
149 changes: 141 additions & 8 deletions api/pkg/imagebuilder/imagebuilder_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,61 @@ var (
},
MaximumRetry: jobBackOffLimit,
}
configWithSa = Config{
BuildNamespace: buildNamespace,
ContextSubPath: "python/pyfunc-server",
BaseImages: cfg.BaseImageConfigs{
"3.7.*": cfg.BaseImageConfig{
ImageName: "gojek/base-image:1",
BuildContextURI: buildContextURL,
DockerfilePath: "./Dockerfile",
},
"3.8.*": cfg.BaseImageConfig{
ImageName: "gojek/base-image:2",
BuildContextURI: buildContextURL,
DockerfilePath: "./Dockerfile",
},
"3.9.*": cfg.BaseImageConfig{
ImageName: "gojek/base-image:3",
BuildContextURI: buildContextURL,
DockerfilePath: "./Dockerfile",
},
"3.10.*": cfg.BaseImageConfig{
ImageName: "gojek/base-image:4",
BuildContextURI: buildContextURL,
DockerfilePath: "./Dockerfile",
},
},
DockerRegistry: dockerRegistry,
BuildTimeoutDuration: timeout,
ClusterName: "my-cluster",
GcpProject: "test-project",
Environment: "dev",
KanikoImage: "gcr.io/kaniko-project/executor:v1.1.0",
Resources: cfg.ResourceRequestsLimits{
Requests: cfg.Resource{
CPU: "500m",
Memory: "1Gi",
},
Limits: cfg.Resource{
CPU: "500m",
Memory: "1Gi",
},
},
Tolerations: []v1.Toleration{
{
Key: "image-build-job",
Value: "true",
Operator: v1.TolerationOpEqual,
Effect: v1.TaintEffectNoSchedule,
},
},
NodeSelectors: map[string]string{
"cloud.google.com/gke-nodepool": "image-building-job-node-pool",
},
MaximumRetry: jobBackOffLimit,
KanikoServiceAccount: "kaniko-sa",
}

defaultResourceRequests = RequestLimitResources{
Request: Resource{
Expand Down Expand Up @@ -214,11 +269,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -266,6 +321,84 @@ func TestBuildImage(t *testing.T) {
wantImageRef: fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID),
config: config,
},
{
name: "success: no existing job, use K8s Service account",
args: args{
project: project,
model: model,
version: modelVersion,
},
existingJob: nil,
wantCreateJob: &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{
Name: fmt.Sprintf("%s-%s-%s", project.Name, model.Name, modelVersion.ID),
Namespace: config.BuildNamespace,
Labels: map[string]string{
"gojek.com/app": model.Name,
"gojek.com/orchestrator": "merlin",
"gojek.com/stream": project.Stream,
"gojek.com/team": project.Team,
"gojek.com/environment": config.Environment,
"gojek.com/component": "image-builder",
},
},
Spec: batchv1.JobSpec{
Completions: &jobCompletions,
BackoffLimit: &jobBackOffLimit,
TTLSecondsAfterFinished: &jobTTLSecondAfterComplete,
ActiveDeadlineSeconds: &timeoutInSecond,
Template: v1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"gojek.com/app": model.Name,
"gojek.com/orchestrator": "merlin",
"gojek.com/stream": project.Stream,
"gojek.com/team": project.Team,
"gojek.com/environment": config.Environment,
"gojek.com/component": "image-builder",
},
},
Spec: v1.PodSpec{
RestartPolicy: v1.RestartPolicyNever,
Containers: []v1.Container{
{
Name: containerName,
Image: "gcr.io/kaniko-project/executor:v1.1.0",
Args: []string{
fmt.Sprintf("--dockerfile=%s", config.BaseImages[modelVersion.PythonVersion].DockerfilePath),
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
},
Resources: defaultResourceRequests.Build(),
TerminationMessagePolicy: v1.TerminationMessageFallbackToLogsOnError,
},
},
Tolerations: []v1.Toleration{
{
Key: "image-build-job",
Operator: v1.TolerationOpEqual,
Value: "true",
Effect: v1.TaintEffectNoSchedule,
},
},
NodeSelector: map[string]string{
"cloud.google.com/gke-nodepool": "image-building-job-node-pool",
},
ServiceAccountName: "kaniko-sa",
},
},
},
Status: batchv1.JobStatus{},
},
wantDeleteJobName: "",
wantImageRef: fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID),
config: configWithSa,
},
{
name: "success: no existing job, tolerations is not set",
args: args{
Expand Down Expand Up @@ -314,11 +447,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -442,11 +575,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -580,10 +713,10 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -712,11 +845,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -811,11 +944,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -913,11 +1046,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down Expand Up @@ -1003,11 +1136,11 @@ func TestBuildImage(t *testing.T) {
fmt.Sprintf("--context=%s", config.BaseImages[modelVersion.PythonVersion].BuildContextURI),
fmt.Sprintf("--build-arg=MODEL_URL=%s/model", modelVersion.ArtifactURI),
fmt.Sprintf("--build-arg=BASE_IMAGE=%s", config.BaseImages[modelVersion.PythonVersion].ImageName),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
fmt.Sprintf("--destination=%s", fmt.Sprintf("%s/%s-%s:%s", config.DockerRegistry, project.Name, model.Name, modelVersion.ID)),
"--cache=true",
"--single-snapshot",
fmt.Sprintf("--context-sub-path=%s", config.ContextSubPath),
fmt.Sprintf("--build-arg=GOOGLE_APPLICATION_CREDENTIALS=%s", "/secret/kaniko-secret.json"),
},
VolumeMounts: []v1.VolumeMount{
{
Expand Down
Loading