Kustomize doesn't fetch YAML from HTTPS URLs

[jpeach]

Based on the documentation , I was expecting that I could use a https:// URL as a YAML resource, but kustomize doesn't seem to find it (or doesn't follow the GitHub redirects?).

$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/knative/serving/releases/download/v0.11.0/serving-crds.yaml
- https://github.com/knative/serving/releases/download/v0.11.0/serving-core.yaml

$ kustomize build .
2020/01/10 16:29:09 evalsymlink failure on '/private/var/folders/d9/tpzrvxbs7s98zqy4c5yq8bk40000gr/T/kustomize-632648700/releases/download/v0.11.0/serving-crds.yaml' : lstat /private/var/folders/d9/tpzrvxbs7s98zqy4c5yq8bk40000gr/T/kustomize-632648700/releases: no such file or directory
Error: accumulating resources: accumulating resources from 'https://github.com/knative/serving/releases/download/v0.11.0/serving-crds.yaml': evalsymlink failure on '/Users/jpeach/src/kustomize/knative/base/https:/github.com/knative/serving/releases/download/v0.11.0/serving-crds.yaml' : lstat /Users/jpeach/src/kustomize/knative/base/https:: no such file or directory

$ kustomize version
{Version:3.5.3 GitCommit:5ba90fe5ef1dc4599e359edd41d1d0e6373b247d BuildDate:2019-12-18T03:07:49+00:00 GoOs:darwin GoArch:amd64}

[yujunz]

AFAIK, remote target is limited to a few git repo URL scheme.

You may have a look at https://github.com/kubernetes-sigs/kustomize/blob/master/examples/goGetterGeneratorPlugin.md for your use case.

[piyushGoyal2]

+1 .
We also want to segregate sidecars and init containers common across applications. We were planning to have following section in all applications base folders with Kustomization.yaml having

patches:

• path: patchSidecarSplunk.yaml
  target:
  kind: Deployment

but referring to one single file based on Github URL doesn't seem possible.

@yujunz

[jpeach]

I was reading this documentation, which made me think that all kinds of go-getter URLs were supported generally.

So this sounds like "behaves correctly"?

[yujunz]

@jpeach the documents says all url should be go-getter conformant. But not all go-getter url are supported. Related discussion in #923

[yujunz]

@piyushGoyal2 you are right about single file cannot be referenced

We may play some trick in go-getter plugin to make it work for generator

https://github.com/kubernetes-sigs/kustomize/blob/master/plugin/someteam.example.com/v1/gogetter/GoGetter_test.go#L72

But it does not apply for transformers like patches.

[piyushGoyal2]

@yujunz - Does it always look out for kustomization.yaml in remote repository? My assumption was that using go-getter, it just clones the repo as a sub directory in the current folder and then rest of the kustomize logic applies on top of it. Maybe I don't need to download particular files. I could clone the remote repository with sub folders and then give the path of cloned repo sub folder to include in the patches.

Looks like a pretty strong use case to me. What would you suggest?

[yujunz]

The go-getter plugin does not expect kustomization, you may override the command.

The builtin resource generator does looking for a directory with kustomization.yaml file.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[daurnimator]

I would expect to be able to reference a remote resource.

[yujunz]

I would expect to be able to reference a remote resource.

It should be supported in version >3.5.5 now.

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.