Build an offline version of the Devfile registry image #112
Conversation
amisevsk
requested review from
l0rd,
nickboldt,
rhopp,
skabashnyuk and
vparfonov
as code owners
amisevsk
force-pushed
the
cache-projects
branch
from
6c1d9c3
to
3a61f90
Compare
|
Update: the last commit adds an attempt to determine the registry's service IP and port from k8s env vars when the required environment variable is not set. This should work in many cases but is rather fragile. I haven't tested it for workspaces in another namespace, etc. |
amisevsk
force-pushed
the
cache-projects
branch
from
3a61f90
to
684d8c8
Compare
|
Squashed and rebased onto master now that #110 is merged; PR is ready for review cc: @l0rd @nickboldt |
| # In addition, this script will perform the necessary set up for the offline | ||
| # devfile registry, replacing placeholders in all devfiles based off environment | ||
| # variable | ||
| # CHE_DEVFILE_HTTPS_ENDPOINT |
There was a problem hiding this comment.
So I assume that in airgap mode, from the operator, we would also have to add this variable in the new devfile registry config map, based on the external URL of the devfile registry ?
@l0rd do you confirm ?
cc @tomgeorge
There was a problem hiding this comment.
Yes, that's accurate; we need the environment variable to be able to rewrite devfiles to point at exposed endpoints. Without the environment variable we do try to use k8s-provisioned env vars to determine service IP and route, but I haven't tested this much and it may be unreliable.
There was a problem hiding this comment.
@davidfestal yes that's the weakness of this solution. There is no reliable way to figure the value automatically.
There was a problem hiding this comment.
In particular, without the env var set, we try to go off of
URL="http://${CHE_DEVFILE_REGISTRY_SERVICE_HOST}:${CHE_DEVFILE_REGISTRY_SERVICE_PORT}"
which depends on the devfile registry server being called che-devfile-registry, and may not work in some scenarios (I don't know the internals of networking on OpenShift).
|
@l0rd @nickboldt So I assume that if we would like to use the devfile registry with embedded samples inside for the airgap mode, we should first have built that with the |
|
I suspect for CRW 2 we will only build & publish the offline/airgap option. Yes, it's a bigger image to download than the online flavour, but it's also more guaranteed to work as all the plugins/projects will be self-contained in the devfile and plugin reg containers. |
|
I agree with @nickboldt. It will be the default downstream (and we should consider to make it the default upstream as well c.f. this comment) |
|
@amisevsk I think we should update the container build section in the README file as well and the "Customizing the devfile and plugin-in registries" section in the doc. But considered that this PR may block other PRs we can merge it and address the doc in a separate one. |
+1 |
|
@l0rd I added some docs to the README.md and opened issue eclipse-che/che#14866 to track the larger documentation task |
There was a problem hiding this comment.
I like this solution.
However...
For this solution to be viable, all the sample projects to be bundled need to have their license/copyright info correctly configured.
See https://issues.jboss.org/browse/CRW-317?focusedCommentId=13798222&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13798222 for a list of the problems with the sample projects' licenses/copyright info.
amisevsk
force-pushed
the
cache-projects
branch
from
c56b9cd
to
f8a47b0
Compare
|
Rebased on master to fix merge conflict. |
Che issue for this: eclipse-che/che#14790 |
Enable users to build an offline version of the devfile registry, which
contains .zip files for all projects. The offline registry should be
launched with environment variable CHE_DEVFILE_HTTPS_ENDPOINT set to the
public URL of the devfile registry for it to properly serve project zip
files. If the environment variable is not set at startup, the registry
will try to resolve its cluster IP and port from Kubernetes-provisioned
environment variables.
The regular (non-offline) build of the registry now requires docker
builds to target 'registry', e.g.
docker build [...] --target registry
offline builds can instead target 'offline-registry'
Signed-off-by: Angel Misevski <amisevsk@redhat.com>
amisevsk
force-pushed
the
cache-projects
branch
from
f8a47b0
to
a3fc742
Compare
Signed-off-by: Angel Misevski <amisevsk@redhat.com>
amisevsk
force-pushed
the
cache-projects
branch
from
4bc0e9e
to
38035c2
Compare
Signed-off-by: Angel Misevski <amisevsk@redhat.com>
* Enable building offline version of registry
Enable users to build an offline version of the devfile registry, which
contains .zip files for all projects. The offline registry should be
launched with environment variable CHE_DEVFILE_HTTPS_ENDPOINT set to the
public URL of the devfile registry for it to properly serve project zip
files. If the environment variable is not set at startup, the registry
will try to resolve its cluster IP and port from Kubernetes-provisioned
environment variables.
The regular (non-offline) build of the registry now requires docker
builds to target 'registry', e.g.
docker build [...] --target registry
offline builds can instead target 'offline-registry'
Signed-off-by: Angel Misevski <amisevsk@redhat.com>
* Enable building offline version of registry
Enable users to build an offline version of the devfile registry, which
contains .zip files for all projects. The offline registry should be
launched with environment variable CHE_DEVFILE_HTTPS_ENDPOINT set to the
public URL of the devfile registry for it to properly serve project zip
files. If the environment variable is not set at startup, the registry
will try to resolve its cluster IP and port from Kubernetes-provisioned
environment variables.
The regular (non-offline) build of the registry now requires docker
builds to target 'registry', e.g.
docker build [...] --target registry
offline builds can instead target 'offline-registry'
Signed-off-by: Angel Misevski <amisevsk@redhat.com>
Important Note: this PR is set to target the
override-imagesbranch until #110 is merged, as it also requires modification of the container entrypoint. Once PR #110 is merged, I will update target branch to master.What does this PR do?
Enables building an offline version of the devfile registry.
As in the similar feature for the plugin registry, this is done by specifying
--target offline-registryduring the docker build (with the current online registry being built by--target registry).The offline registry downloads all project zip files for projects hosted on github and stores them in the
/resourcesdir. Devfiles are updated to contain a placeholder, which is filled from an env var (CHE_DEVFILE_HTTPS_ENDPOINT) that must be set at runtime.What issues does this PR fix or reference?
eclipse-che/che#14733
Additional Info
Tested on OpenShift 3.11, manually setting the required environment variable. It works generally as desired, except I encountered issue eclipse-che/che#14800.
Caveat
For this solution to be viable, all the sample projects to be bundled need to have their license/copyright info correctly configured.
See https://issues.jboss.org/browse/CRW-317?focusedCommentId=13798222&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13798222 for a list of the problems with the sample projects' licenses/copyright info.