FEATURE: Introduce ContentRepositoryMaintainer and restore projection cli commands

[mhsdesign]

Adjustments for the subscription engine: #5321

Introduces a dedicated ContentRepositoryMaintainer as replacement for ContentRepository::setUp and the ProjectionService/ ProjectionCatchupService (from 01f7f8f)

The ContentRepositoryMaintainer

Set up and manage a content repository

Initialisation / Tear down

The method setUp sets up the content repository like event store and subscription database tables.
It is non-destructive.

Resetting a content repository with prune method will purge the event stream and reset all subscription states.

Status information

The status of the content repository e.g. if a setup is required or if all subscriptions are active and their position
can be examined with status

The event store status is available via ContentRepositoryStatus::$eventStoreStatus, and the subscription status
via ContentRepositoryStatus::$subscriptionStatus. Further documentation in SubscriptionStatusCollection.

Subscriptions (mainly projections)

This maintainer offers also the public API to interact with the subscription catchup. In the happy path,
no interaction is necessary, as ContentRepository::handle() triggers the subscriptions after applying the events.

Special cases:

Replay

For initialising on a new database - which contains events already - a replay will make sure that the subscriptions
are emptied and reapply the events. This can be triggered via replaySubscription or replayAllSubscriptions

And after registering a new subscription a setup as well as a replay of this subscription is also required.

Reactivate

In case a subscription is detached but is reinstalled a reactivation is needed via reactivateSubscription

Also in case a subscription runs into the error status, its code needs to be fixed, and it can also be attempted to be reactivated.

Note that in both cases a subscription replay would also work, but with the difference that the subscription is reset as well.

use Neos\Error\Messages\Error;

/** @api */
class ContentRepositoryMaintainer
{
    public function setUp(): Error|null;
    public function status(): ContentRepositoryStatus;
    public function replaySubscription(SubscriptionId $subscriptionId, \Closure|null $progressCallback = null): Error|null;
    public function replaySubscriptions(\Closure|null $progressCallback = null): Error|null;
    public function retrySubscription(SubscriptionId $subscriptionId, \Closure|null $progressCallback = null): Error|null;
    public function prune(): Error|null;
}

---

Also a few naming adjustments were made to the original ProjectionStatus introduced via: #4846

The idea was then that a projection could notice that it needs a replay, which is not possible as it would need access to the event store. Also with the subscription status the ProjectionStatus is more of a setup status or schema / migration status and has been renamed to ProjectionSetupStatus.

Furthermore the ProjectionSubscriptionStatus is now a first level concern like the ProjectionSubscriber in theSubscriptionStatusCollection.
If its extended to allow other subscribers another status type should be introduced.

And a special DetachedSubscriptionStatus was added as the projection setup status cannot be calculated and when extending the system to support multiple subscribers, we cannot know their original classification but have to use a special empty placeholder like: DetachedSubscriptionStatus. This also makes the $projectionStatus === null detached case more explicit when using status.

Lastly the lately added method pruneAllWorkspacesAndContentStreamsFromEventStream has been inlined into CR Maintainer

Upgrade instructions

Review instructions

Checklist

• Code follows the PSR-2 coding style
• Tests have been created, run and adjusted as needed
• The PR is created against the lowest maintained branch
• Reviewer - PR Title is brief but complete and starts with FEATURE|TASK|BUGFIX
• Reviewer - The first section explains the change briefly for change-logs
• Reviewer - Breaking Changes are marked with !!! and have upgrade-instructions

[mhsdesign]

the ContentRepositoryMaintainer allows us to make the subscription engine internal which is important to be able to extend it and to not expose too many concepts at first.

[kitsunet]

I mean looks nice and all, but do we really need this?

[mhsdesign]

I fine tuned the catchupProjection behaviour so it throws if setup was not run for the projection.

On the other-hand #4746 (comment) explicitly defines that setup should not reactivate errored projections and not reattach detached projections as well. But thats just how the original inventors did it:

Detached -> must be reactivated / or reset and then catchup (replay)
Error -> must be reactivated / or replay

Current state:

Detached -> must be setup and then catchup / or replay
Error -> must be setup and then catchup / or replay

---

Maybe it would be really better to say a detached projection MUST be replayed (because the data is probably stale) -> no a reactivate would be better!.

So i thought of this again, instead of offering catchupProjection we have a dedicated reactivateProjection which is a new subscription engine process like https://github.com/patchlevel/event-sourcing/blob/b8591c56b21b049f46bead8e7ab424fd2afe9917/src/Subscription/Engine/DefaultSubscriptionEngine.php#L624

[mhsdesign]

Okay, regarding the naming discussion for the cli commands

so currently in 9.0 this is the feature set:

./flow cr:projectionreplay
./flow cr:projectionreplayall

which i am about to extend with a new command to retry projections in ERROR

./flow cr:projectionreactivate

the thing is do we want to continue using the projection naming here or as startet by Bastian originally use "subscription::

./flow cr:subscriptionreplay
./flow cr:subscriptionreplayall
./flow cr:subscriptionreactivate

(actually his orignal draft was a little different and low level like this:)

./flow cr:subscriptionsreset && ./flow cr:subscriptionscatchup

and to have it all we could just remove the projection suffix, effectively reverting 3c154a0 again:

./flow cr:replay
./flow cr:replayall
./flow cr:reactivate or even ./flow cr:reactivatesubscriber as its not to be used often

---

The question is for the future: will the projectionreplayall command then only replay the projections or other subscribers as well? Only replaying the projections would leave the others unactivated in which case the intention is maybe too literal. A general replay just to setup things makes sense right?

---

After discussing this with christian we opted in for option Drölf: Keep the old projection* commands as deprecated and forward them, but introduce a new subscription controller with shorter namings:

./flow subscription:replay
./flow subscription:replayall
./flow subscription:reactivate

Regarding that the maintainer could just expose reset and boot directly, we argued that this leakes to much implementation outwards for now and in case we find this necessary at some point we can still deprecated replayAll in the maintainer service and implement it differently. A replayAll for cli makes definitely sense especially how often it is needed during the betas.