Move queue management to dispatcher

[michel-laterman]

What does this PR do?

Move queue management actions to the dispatcher from the fleet-server in order to help with future work to add a retry mechanism. Add a PersistedQueue type which wrap the ActionQueue to make persisting the queue simpler for the consumer.

Queuing was added as part of #419

Why is it important?

Retry actions on edge will be implemented in the dispatcher by interacting with the queue. Moving all of the control flow for the queue to the dispatcher makes it more straightforward to understand and debug

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Follow guide in #419

Related issues

• Relates Auto-retry upgrades on the edge #778

[mergify]

This pull request does not have a backport label. Could you fix it @michel-laterman? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[elasticmachine]

❕ Build Aborted

Either there was a build timeout or someone aborted the build.

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-09-13T13:54:19.819+0000

• Duration: 122 min 18 sec

Test stats 🧪

Test	Results
Failed	0
Passed	4923
Skipped	17
Total	4940

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages.

• run integration tests : Run the Elastic Agent Integration tests.

• run end-to-end tests : Generate the packages and run the E2E Tests.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	97.333% (73/75)	👍
Files	67.521% (158/234)	👍
Classes	68.192% (313/459)	👍 0.069
Methods	52.132% (868/1665)	👎 -0.063
Lines	38.515% (9381/24357)	👍 0.026
Conditionals	100.0% (0/0)	💚

[michel-laterman]

rebasing on github is mesy I'll make a new PR for v2 without any of the garbage

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b queue-in-dispatch upstream/queue-in-dispatch
git merge upstream/main
git push upstream queue-in-dispatch

[michel-laterman]

Uses fixes from the v2 attempt: #1109

[blakerouse]

Change looks good.

[blakerouse]

Instead of this forloop copy should work.

copy(actions, resp.Actions)

[michel-laterman]

ok, loop was the original implementation. i'll change to copy

[michel-laterman]

/test

[michel-laterman]

/test

[cmacknz]

The Mac build being broken is a known issue and is not specific to this PR, main just updated to switch to Github actions which should help. There is an open investigation with the CI team about why the Mac workers aren't reliable.

[michel-laterman]

Closing, v2 PR was merged as the feature has been pushed to 8.6