refactor: implement warnings as notices #486

benhoyt · 2024-08-16T03:55:51Z

This deletes the legacy warnings code and re-implements pebble warnings and pebble okay using notices as warnings (as we originally intended). The /v1/warnings endpoints have been removed, and pebble warnings now hits /v1/notices?types=warning.

On every request, instead of returning the count of warnings and the latest timestamp, we only return the timestamp of the latest (most recent last-repeated time). This way the server doesn't have to track which notices have been okayed -- the client can compare against its stored timestamp and see if the server-reported timestamp is later.

So there are a few breaking changes in this PR:

Client.WarningsSummary() (count int, timestamp time.Time) is now Client.LatestWarningTime() time.Time, as there's no count anymore
GET /v1/warnings (list warnings) is gone: pebble warnings now uses GET /v1/notices?types=warning
POST /v1/warnings (okay warnings) are gone: pebble okay now just updates the local state file, and doesn't use the API/client at all
Similarly, Client.Warnings() and Client.Okay() are gone; use Client.Notices() and maintain local client-side state, respectively
State.Warnf is still around, but it's now just a helper that calls State.AddNotice with the right type (warning) and key (message)

Currently Pebble itself doesn't record any warnings, so you can't test easily without making a code change. To perform manual tests, I hacked the API endpoint used by pebble notify to record warning notices instead of custom ones:

diff --git a/internals/daemon/api_notices.go b/internals/daemon/api_notices.go
index 8cd2745..7b62f19 100644
--- a/internals/daemon/api_notices.go
+++ b/internals/daemon/api_notices.go
@@ -201,9 +201,9 @@ func v1PostNotices(c *Command, r *http.Request, user *UserState) Response {
        if payload.Type != "custom" {
                return BadRequest(`invalid type %q (can only add "custom" notices)`, payload.Type)
        }
-       if !customKeyRegexp.MatchString(payload.Key) {
-               return BadRequest(`invalid key %q (must be in "example.com/path" format)`, payload.Key)
-       }
+       //if !customKeyRegexp.MatchString(payload.Key) {
+       //      return BadRequest(`invalid key %q (must be in "example.com/path" format)`, payload.Key)
+       //}
        if len(payload.Key) > maxNoticeKeyLength {
                return BadRequest("key must be %d bytes or less", maxNoticeKeyLength)
        }
@@ -228,7 +228,7 @@ func v1PostNotices(c *Command, r *http.Request, user *UserState) Response {
        st.Lock()
        defer st.Unlock()
 
-       noticeId, err := st.AddNotice(user.UID, state.CustomNotice, payload.Key, &state.AddNoticeOptions{
+       noticeId, err := st.AddNotice(nil, state.WarningNotice, payload.Key, &state.AddNoticeOptions{
                Data:        data,
                RepeatAfter: repeatAfter,
        })

Fixes #475.

hpidcock

LGTM. A few comments. Looking forward to the follow-up review that removes the mutex lock on the daemon response code for warnings.

hpidcock · 2024-08-18T23:53:28Z

internals/overlord/state/notices.go

+func (s *State) LatestWarningTime() time.Time {
+	s.reading()
+
+	// TODO(benhoyt): optimise with an in-memory atomic cache when warnings are added (or pruned),


Are you going to follow up with this PR soon? It seems like then we can finally get rid of the mutex lock in the daemon response code.

Yes, it's my plan to do a follow-up PR shortly after this one.

hpidcock · 2024-08-18T23:57:00Z

internals/overlord/state/notices.go

+		RepeatAfter: warningRepeatAfter,
+	})
+	if err != nil {
+		panic(fmt.Sprintf("internal error: error adding warning notice: %v", err))


This would seem to only panic when the message is an empty string. That seems OK.

hpidcock · 2024-08-19T00:17:45Z

internals/cli/cmd_okay.go

 		}
 	}

-	last, err := lastWarningTimestamp()
+	err = saveCLIState(cmd.socketPath, state)


I think, at some point, we need to improve saveCLIState to atomically write the state file (and to include conflict resolution).

hpidcock · 2024-08-19T00:20:02Z

internals/cli/cmd_warnings.go

 		w.Flush()
 	}

+	state.WarningsLastListed = warnings[len(warnings)-1].LastRepeated


Nice detail.

IronCore864

Some questions/comments on this feature in general after testing:

client/notices.go

client/warnings.go

internals/overlord/state/notices.go

internals/cli/cmd_okay.go

client/client.go

change-update already existed before this PR.

@hpidcock

#17991 Per discussion with @hpidcock, update Juju 3.6 to use the just-released Pebble v1.16.0 ([changelog](https://github.com/canonical/pebble/releases/tag/v1.16.0)). Summary of main changes: * Improvements to how services with dependencies are started and stopped (using "lanes"), so that independent services are started in parallel, and dependent services start up serially. Error handling is also improved. Fix in canonical/pebble#437. * A fix for a tricky bug in pebble exec, so it doesn't lose output in interactive mode. Fix in canonical/pebble#466! prdesc Reduce the size of exec tasks by not storing the execSetup (which includes all environment variables) in state. This speeds up (and shrinks the data) when serialising state to disk during state.Unlock. Fix is in canonical/pebble#478. prdesc Ensure Pebble doesn't hang (with no error message) when the state file directory is read-only or otherwise inaccessible. Fix in canonical/pebble#481. * Re-implement warnings using notices. This was always the intention since the notices feature was added (it was designed as a superset of warnings), but we'd never gotten to it. Lots of nice code deletion in canonical/pebble#486. ## QA steps Deploy a K8s charm, like `snappass-test`, and ensure `pebble version --client` on the workload reports v1.16.0.

refactor: implement warnings as notices

844fa8a

benhoyt requested review from IronCore864 and hpidcock August 16, 2024 03:57

hpidcock approved these changes Aug 19, 2024

View reviewed changes

IronCore864 reviewed Aug 19, 2024

View reviewed changes

client/notices.go Show resolved Hide resolved

client/warnings.go Show resolved Hide resolved

internals/overlord/state/notices.go Show resolved Hide resolved

internals/cli/cmd_okay.go Show resolved Hide resolved

client/client.go Show resolved Hide resolved

IronCore864 approved these changes Aug 19, 2024

View reviewed changes

benhoyt added 3 commits August 20, 2024 13:11

okay command no longer needs the client at all

29629c6

uncomment "change-update" and "warning" notice types in docs

754866a

change-update already existed before this PR.

fix --all flag for "pebble warnings"

abfe0fc

benhoyt merged commit 0dd6abc into canonical:master Aug 20, 2024
17 checks passed

benhoyt deleted the warnings-as-notices branch August 20, 2024 05:09

benhoyt mentioned this pull request Aug 29, 2024

chore: update Pebble version to v1.16.0 juju/juju#17991

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: implement warnings as notices #486

refactor: implement warnings as notices #486

benhoyt commented Aug 16, 2024 •

edited

Loading

hpidcock left a comment

hpidcock Aug 18, 2024

benhoyt Aug 19, 2024

hpidcock Aug 18, 2024

hpidcock Aug 19, 2024

hpidcock Aug 19, 2024

IronCore864 left a comment •

edited

Loading

refactor: implement warnings as notices #486

refactor: implement warnings as notices #486

Conversation

benhoyt commented Aug 16, 2024 • edited Loading

hpidcock left a comment

Choose a reason for hiding this comment

hpidcock Aug 18, 2024

Choose a reason for hiding this comment

benhoyt Aug 19, 2024

Choose a reason for hiding this comment

hpidcock Aug 18, 2024

Choose a reason for hiding this comment

hpidcock Aug 19, 2024

Choose a reason for hiding this comment

hpidcock Aug 19, 2024

Choose a reason for hiding this comment

IronCore864 left a comment • edited Loading

Choose a reason for hiding this comment

benhoyt commented Aug 16, 2024 •

edited

Loading

IronCore864 left a comment •

edited

Loading