GitRepositoryReconciler no-op clone improvements #724

Generic error is an attempt to avoid creating new error type for every new unique scenario. It can be used to configure and build custom error handling behavior, logging and event recording at present. Contextual errors, Stalling and Waiting error, have special meaning for the reconciliation results. But the Event error type can be replaced with Generic error with some specific configurations. The Event error is kept for a gradual migation to Generic error. Similarly, the Generic error can be used to easily create new error handling behaviors. The error Config can be used to configure any of the errors, including contextual errors, without altering their contextual meaning, to modify how they are handled. The error constructors configure the errors with common default configurations. These configurations can be modified to alter the behavior. Signed-off-by: Sunny <darkowlzz@protonmail.com>

ErrorActionHandler processes the reconciliation error results based on their configurations. It performs actions like logging and event recording based on the error configuration. More actions can be accommodated in the future with more error configurations. It can be a replacement for RecordContextualError() which does the same operations but can't be configured much. Signed-off-by: Sunny <darkowlzz@protonmail.com>

Add Generic error in RuntimeResultBuilder and ComputeReconcileResult implementation with consideration to the error configurations. Safeguards are added in the runtime result builder to ensure default requeue after interval is set when is's set to zero or unset. Signed-off-by: Sunny <darkowlzz@protonmail.com>

For gradual migration to Generic error, update only the GitRepo reconciler to use Generic error. Replace the Waiting error for git no change scenario with a Generic error with proper no-op, early return, error configurations. This ensures that the no-op only results in log and K8s native events at normal level. Fixes a reconciliation issue when recovering from a failure state (with previous success state and artifact in the storage) and optimized git clone feature is on, which results in failure to persist as the git optimization prevented full reconciliation due to already existing artifact and removal of failure negative conditions on the object status. In order to allow failure recovery, the git clone optimizations are now only applied when the object is already in a ready state. Signed-off-by: Sunny <darkowlzz@protonmail.com>

Introduce a new field in the GitRepositoryReconciler to set the enabled features. This makes it test friendly compared to using global flags for setting and checking flags in the tests. Enable default feature gates in all the GitRepo reconciler tests. Add test cases for reconcileSource() to test the behavior of optimized git clone when the Repo is ready and not ready. This ensures that the full reconciliation is not skipped when GitRepo is not ready. Signed-off-by: Sunny <darkowlzz@protonmail.com>

Introduce concrete and partial commits. Concrete commits have all the information from remote including the hash and commit content. Partial commits are based on locally available copy of a repo, they may only contain the commit hash and reference. IsConcreteCommit() can be used to find out if a given commit is based on local information or full remote repo information. Update go-git and libgit2 branch/tag clone optimization to return a partial commit and no error. Update and simplify the go-git and libgit2 tests for the same. Signed-off-by: Sunny <darkowlzz@protonmail.com>

Introduce contentConfigChecksum in the GitRepository.Status to track the configurations that affect the content of the artifact. It is used to detect a change in the configuration that requires rebuilding the whole artifact. This helps skip the reconciliation early when we find out that the remote repository has not changed. Moves fetching the included repositories in reconcileSource() to collect enough information in reconcileSource() to be able to decide if the full reconciliation can be skipped. This results in reconcileInclude() to just copy artifact to the source build directory. Introduce a gitCheckout() method to perform construction of all the git checkout options and perform the checkout operation. This helps to easily perform checkout multiple times when we need it in reconcileSource(). When we check with the remote repository if there's an update, and find out that there's no update, we check if any other configurations that affect the source content has changed, like includes, ignore rules, etc. If there's a change, we need to perform a full checkout of the remote repository in order to fetch the complete source. The git checkout no-op optimization is enabled in this method based on the presence of an artifact in the storage. The failure notification handler is modifed to handle the recovery of a no-op reconcile failure and create a notification message accordingly with the partial commit. Signed-off-by: Sunny <darkowlzz@protonmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitRepositoryReconciler no-op clone improvements #724

GitRepositoryReconciler no-op clone improvements #724

Commits on May 20, 2022