-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
osdocs-626 preparing for disconnected installation #16678
Conversation
The preview will be available shortly at: |
@umohnani8, will you PTAL? |
installing/installing_disconnected/installing-disconnected-preparations.adoc
Outdated
Show resolved
Hide resolved
|
||
.Next steps | ||
|
||
//I need to figure out how to sequence the disconnected and proxy prep. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These aren't inherently ordered, you just need to setup both at some point and feed their respective settings into the install-config.yaml
. The only wrinkle is that if each need additionalTrustBundle
entries, you'd need to append the second entry, but if only one or the other needs additional trust, you need to create a new additionalTrustBundle
instead of appending. That's not all that complicated in practice, but might require some careful wording to avoid confusion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm making a follow-up JIRA to review this sequencing after the proxy content merges: https://jira.coreos.com/browse/OSDOCS-693
|
||
If you choose to perform a disconnected installation on a cloud platform, you | ||
still require access to its cloud APIs and will still require internet access | ||
to use your cluster. Depending on your network, you might require less internet |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
... and will still require internet access to use your cluster
Can you unpack this? I expected access to cloud APIs to be all you'd need.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking, I might have misunderstood, but don't you need internet access to use the cloud APIs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AWS populates routes to some of its endpoints, so you can reach them from inside the VPC, even if you've blocked all "external" traffic. Or something like that. Some things (like IAM) don't get this treatment, so the cred operator would want a way to get out to AWS's external IAM endpoint.
|
||
Disconnected clusters have the following additional limitations and restrictions: | ||
|
||
* Cluster settings always displays errors because the cluster cannot determine |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Cluster settings" is not very specific. Maybe "The ClusterVersion status will include Unable to retrieve available updates
unless you update its spec.upstream
."? And hmm, maybe we want to patch the ClusterVersion object to empty that property, since it's optional. Not sure if the best place to patch it is clobbering this in a multi-part install or to patch the object post-install. Probably more reliable to patch post-install, since then you wouldn't be impacted in the unlikely event that the installer renames its generated manifest file. But then there will be a window of time before the patch when you will still see the error. Or maybe clearing the upstream
field will still give you errors. @abhinavdahiya, thoughts about the best-practices config for "I don't want to upgrade this cluster ever"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you tell me more about this patch? Do we want users to patch this field? Can I just tell them that there's an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Telling them ClusterVersion will report an error (more specific than what you have now) is fine, but if it was my cluster I'd patch to remove it ;).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking, do you have information about patching spec.upstream
somewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be:
$ oc patch clusterversion version --type json -p '{"op": "remove", "path": "/spec/upstream"}'
But let me sanity check, and I'll report back.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So for "I want to watch for updates in a different location":
$ oc patch clusterversion version --type json -p '[{"op": "add", "path": "/spec/upstream", "value": "https://example.com/your/new/location"}]'
And for "I don't want to check for upgrades, I'll either never upgrade (!) or manually upgrade (e.g. via oc adm upgrade ...
)":
$ oc patch clusterversion version --type json -p '[{"op": "remove", "path": "/spec/channel"}, {"op": "remove", "path": "/spec/upstream"}]'
in which case ClusterVersion will have a status.conditions
entry like:
- lastTransitionTime: "2019-09-21T22:36:52Z"
message: The update channel has not been configured.
reason: NoChannel
status: "False"
type: RetrievedUpdates
which seems appropriate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you file a bug about that console issue? I think that more text would certainly help there.
I'm making a follow-up issue to describe patching the CVO. I don't see any modifications to it in our current doc set, and with our docs workflow, it will be easier to get reviews on the proposed change on a separate PR.
---- | ||
"auths": { | ||
... | ||
"<bastion_host_name>:5000": { <1> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't have to be :5000
, your local registry could serve HTTPS on the default 443. You could use <local_registry_authority>
to lean on the URI nomenclature or <local_registry_host_port>
because we don't actually want the authority's optional userinfo "@"
here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
5000 seems popular for "I'm running some dev thing". For production, I'd expect a registry running on 443, but I don't think these docs are where we want to walk folks through production registries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wking, we need to provide enough information that they could take this example and configure a production registry. I'll make it generic.
Is there any other information that would be valid in a production environment that we don't have yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any other information that would be valid in a production environment that we don't have yet?
If you mirror to an existing, production registry, you probably just need the oc adm release mirror ...
step and imageContentSources
block. You won't need anything for additionalTrustBundle
to support the mirror registry, because it will already have a cert signed by a RHCOS-trusted CA. If the mirrored images are public on the mirror registry, you won't need a mirror token in your pull secret. I dunno, there's not all that much else going on...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm glad that it's that easy.
} | ||
---- | ||
|
||
. Edit the new file and add a section that describes your registry to it: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we care about the cloud.redhat.com pull-secret entries? You'll need those to create the mirror, and to submit telemetry. So maybe for setups where you had a proxy. But for a truly disconnected install I don't see how the cluster would actually use them, which would allow simpler docs like:
Create a pull secret containing only your local registry authentication information...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably not. I'll make a separate module with the simpler workflow.
|
||
. Review the | ||
link:https://quay.io/repository/openshift-release-dev/ocp-release?tab=tags[Quay.io] | ||
repository to determine the version of {product-title} that you want to install. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm. Is this the UX we want? I'd expect folks would want to use the most-recent release in their channel, which you can get with:
$ curl -sH 'Accept:application/json' 'https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.1' | jq -r '.nodes | sort_by(.version | sub(".*[.]"; "") | tonumber)[-1] | .version + "\t" + .payload'
4.1.16 quay.io/openshift-release-dev/ocp-release@sha256:61ed953962d43cae388cb3c544b4cac358d4675076c2fc0befb236209d5116f7
replacing stable-4.1
with your favorite channel. Or use []
instead of [-1]
to get a list of all the choices. And we could do that filtering in JavaScript if we wanted something live in the served docs. Or are we not allowed to rely on JavaScript or something?
Lacking JavaScript, I'd expect there would be a customer-portal link we could point folks at with a list of supported versions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you have that customer portal link?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@openshift/team-documentation, PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of comments and otherwise LGTM!
636ca5e
to
ced98c6
Compare
/cherrypick enterprise-4.2 |
@kalexand-rh: #16678 failed to apply on top of branch "enterprise-4.2":
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/cherrypick enterprise-4.2 |
@kalexand-rh: new pull request created: #16823 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
<1> For `bastion_host_name`, specify the registry domain name | ||
that you specified in your certificate, and for `<local_registry_host_port>`, | ||
specify the port that your mirror registry uses to serve content. | ||
<2> For `<credentials>`, specify the base64-encoded user name and password for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What command to generate the base64-encoded string?
Actually a easier way to generate this pull secret json file.
# podman login --authfile ~/pullsecret_config.json upshift.mirror-registry.qe.devcluster.openshift.com:5000 Username: dummy Password: Login Succeeded!
# cat ~/pullsecret_config.json { "auths": { "upshift.mirror-registry.qe.devcluster.openshift.com:5000": { "auth": "ZHVtbXk6ZHVtbXk=" } } }
@@ -0,0 +1,45 @@ | |||
[id="installing-restricted-networks-preparations"] | |||
= Creating a mirror registry for installation in a restricted network |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is better to highlight disconnected install only support UPI install, but not IPI install.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll clarify that in line 8.
The export landed with the module in ced98c6 (osdocs-626 preparing for disconnected installation, 2019-08-26, openshift#16678). But these are local variables that we expand when calling the mirror command. The mirror command receives them from its command line arguments; we don't need export [1] to expose them in the mirror command's environment as well. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#export
The OCP_RELEASE stuff landed in ced98c6 (osdocs-626 preparing for disconnected installation, 2019-08-26, openshift#16678). Since then, 4.2.14 and later (including some 4.1 releases) have been suffixing the Quay tags with architecture information [1]. By pivoting to digests pulled from the release errata, we decouple these docs from tag names. Also pull the repo portion of the pullspec out of the errata, so there's less hard-coding in the docs. Checking the to-disk flow using the 4.5.11 oc to mirror the 4.5.11 release image: $ oc version --client Client Version: 4.5.11 $ oc adm release mirror --to-dir=/tmp/mirror quay.io/openshift-release-dev/ocp-release@sha256:4d048ae1274d11c49f9b7e70713a072315431598b2ddbb512aee4027c422fe3e info: Mirroring 110 images to file://openshift/release ... ... Update image: openshift/release:4.5.11 To upload local images to a registry, run: oc image mirror --from-dir=/tmp/mirror 'file://openshift/release:4.5.11*' REGISTRY/REPOSITORY Configmap signature file /tmp/mirror/config/signature-sha256-4d048ae1274d11c4.yaml created I've gone with :4.* in the 'image mirror' examples, because I don't want folks to have to bother creating an environment variable, and we have no roadmap for a backwards-incompatible OpenShift v5 today. The reference to 'oc adm release info ...' should mitigate concerns about folks who want to map from by-digest pullspecs back to the release metadata [2]. I'm not linking to an existing section on that command, because the only section we have around that today is _unused_topics/exploring-cvo.adoc. There are some other mentions, e.g. with --changelog in modules/cli-administrator-other.adoc and with --commits in modules/querying-operator-status-after-installation.adoc, but an inline comment seemed simpler here than trying to link to any of those locations with enough guidance for the reader to re-orient once they followed the link. I also updated the very similar modules/update-mirror-repository.adoc with content that minimally diverged from the installation module. It might be worth unifying these modules with conditionals for imageContentSources vs. ImageContentSourcePolicy, but I'm punting on that for now. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1794466 [2]: openshift#19266 (comment)
https://jira.coreos.com/browse/OSDOCS-626