Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

asset/ignition: add event on bootstrap completion #459

Merged
merged 3 commits into from
Oct 15, 2018
Merged

asset/ignition: add event on bootstrap completion #459

merged 3 commits into from
Oct 15, 2018

Conversation

crawford
Copy link
Contributor

@crawford crawford commented Oct 12, 2018

In order to allow the installer to eventually track the installation
progress, this adds a reporting service which inserts an event when the
bootstrap process has completed. This event can be used to determine
when it is safe to remove the bootstrap node, since it signifies that
the bootstrap control plane has pivoted to the scheduled control plane
and that all of the OpenShift manifests have been inserted.

This no longer enables tectonic.service because progress.service is
enabled and lists both tectonic.service and bootkube.service as
dependencies. systemd will start all three services as a result.

In an ideal world, this would make use of sd-notify to signal the
completion of prerequisites but this doesn't work for two reasons.
First, runc doesn't support passing the extra file descriptor used by
sd-notify 1. Second, systemd doesn't support retrying activations when
a dependency has failed 2. This change makes use of "breadcrumb" files
to signal completion as a workaround.

In the creation of the event, date is invoked and passed a custom
format. This was necessary because Golang requires the T separator in
the time package 3 but GNU's date uses a space separator. ISO8601
was also tried, but Golang doesn't seem to parse that correctly either
(date uses "+0000" to specify timezone but Golang expects "+00:00").

@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 12, 2018
@crawford
Copy link
Contributor Author

From my tests:

$ kubectl --kubeconfig=crawford-libvirt/auth/kubeconfig -n kube-system get events bootstrap-complete
LAST SEEN   FIRST SEEN   COUNT     NAME                 KIND      SUBOBJECT   TYPE      REASON    SOURCE                                  MESSAGE                                             
7m          7m           1         bootstrap-complete                                             bootstrap, crawford-libvirt-bootstrap   cluster bootstrapping has completed

This was way more difficult than it should have been.


const (
// ReportSystemdContents is a service that reports the bootstrap progress
// via a ConfigMap.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

via an Event

@abhinavdahiya
Copy link
Contributor

/approve

The content package only contains bootstrap assets. It's more clear that
that is the case if that package is under the bootstrap package, rather
than being adjacent to it.
Adding a line break before and after the closing and opening grave ticks
makes this a little more readable and is the style that the rest of the
codebase uses. This does result in an empty line at the top and bottom
of the file, which is why this style cannot be applied to the shell
scripts.
pkg/asset/ignition/bootstrap/content/report.go Outdated Show resolved Hide resolved
pkg/asset/ignition/bootstrap/content/report.go Outdated Show resolved Hide resolved
pkg/asset/ignition/bootstrap/content/report.go Outdated Show resolved Hide resolved
In order to allow the installer to eventually track the installation
progress, this adds a reporting service which inserts an event when the
bootstrap process has completed. This event can be used to determine
when it is safe to remove the bootstrap node, since it signifies that
the bootstrap control plane has pivoted to the scheduled control plane
and that all of the OpenShift manifests have been inserted.

This no longer enables `tectonic.service` because `progress.service` is
enabled and lists both `tectonic.service` and `bootkube.service` as
dependencies. systemd will start all three services as a result.

In an ideal world, this would make use of sd-notify to signal the
completion of prerequisites but this doesn't work for two reasons.
First, runc doesn't support passing the extra file descriptor used by
sd-notify [1]. Second, systemd doesn't support retrying activations when
a dependency has failed [2]. This change makes use of "breadcrumb" files
to signal completion as a workaround.

In the creation of the event, `date` is invoked and passed a custom
format. This was necessary because Golang requires the `T` separator in
the time package [3] but GNU's `date` uses a space separator. ISO8601
was also tried, but Golang doesn't seem to parse that correctly either
(`date` uses "+0000" to specify timezone but Golang expects "+00:00").

[1]: opencontainers/runc#1807
[2]: systemd/systemd#1312
[3]: golang/go#25937
@abhinavdahiya
Copy link
Contributor

/retest

@wking
Copy link
Member

wking commented Oct 15, 2018

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 15, 2018
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhinavdahiya, crawford, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [abhinavdahiya,crawford,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 7cc49c8 into openshift:master Oct 15, 2018
@crawford crawford deleted the progress branch October 15, 2018 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants