Skip to content
This repository has been archived by the owner on May 23, 2023. It is now read-only.

Enable multi_ack and multi_ack_detailed capabilities #4

Merged
merged 1 commit into from
Oct 19, 2022

Conversation

pjbgf
Copy link
Member

@pjbgf pjbgf commented Oct 18, 2022

Enabling multi_ack and multi_ack_detailed allows the use of go-git against Azure DevOps.
Those capabilities are not fully implemented, however they have no impact for "fresh new fetches", which are the primary use case for Flux.

⚠️ Using this version to deal with existing repositories or to call additional fetch/pull operations against Azure DevOps will result in:

empty git-upload-pack given

This change was tested against a range of Git Service Providers to ensure no regression was being introduced.
Note that the tests were completed on the scope of Source and Image Automation Controllers, which only use a subset of the Git features implemented by go-git.

https-aws-codecommit
https-bitbucket
https-devops-flux-testing
https-gitlab
https-google-code
ssh-ecdsa-bitbucket
ssh-ecdsa-gitlab
ssh-ecdsa-google
ssh-ed25519-bitbucket
ssh-ed25519-gitlab
ssh-ed25519-google
ssh-rsa-aws-codecommit
ssh-rsa-bitbucket
ssh-rsa-devops
ssh-rsa-gitlab
ssh-rsa-google

The blue line in the graph below represents rate(go_memstats_alloc_bytes_total{app="source-controller"}[$__rate_interval]) and shows a stable memory profile throughout the 6h. The same has been observed since the beginning of the tests (+19h).
image

For the purposes of the tests, all the repositories had their spec.gitImplementation set to go-git, the controller has its libgit2 managed transport disabled and OptimizedGitClones=false. The controller was set to have 40 workers to quickly iterate through the 35 GitRepository objects. Most target sources have at least one large file (~100mb). The reconciliation intervals are set to 30s, forcing the reconciliations to be overlapping every so often.

Additional memory and GC related settings:

      - op: replace
        path: /spec/template/spec/containers/0/resources/limits/memory
        value: 1200Mi
      - op: replace
        path: /spec/template/spec/containers/0/resources/requests/memory
        value: 1200Mi
      - op: add
        path: /spec/template/spec/containers/0/env/0
        value:
          name: GOMEMLIMIT
          value: "800MiB"
     - op: add
        path: /spec/template/spec/containers/0/env/0
        value:
          name: GOGC
          value: "100"

For completeness, an additional graph without go_memstats_alloc_bytes_total so it is easier to track GC duration and goroutine numbers:
image

@pjbgf pjbgf added this to the GA milestone Oct 18, 2022
@pjbgf pjbgf requested a review from aryan9600 October 18, 2022 08:56
Signed-off-by: Paulo Gomes <pjbgf@linux.com>
@pjbgf pjbgf force-pushed the reenable-multiack branch from 285d8c0 to dcf3dce Compare October 18, 2022 09:02
Copy link
Member

@aryan9600 aryan9600 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! 🚀
we should probably change the PR title to something like Enable multiack capability?

@pjbgf pjbgf changed the title Add support to Azure DevOps Enable multi_ack and multi_ack_detailed capability Oct 19, 2022
@pjbgf pjbgf changed the title Enable multi_ack and multi_ack_detailed capability Enable multi_ack and multi_ack_detailed capabilities Oct 19, 2022
@pjbgf pjbgf merged commit b33dae7 into master Oct 19, 2022
@pjbgf pjbgf deleted the reenable-multiack branch October 19, 2022 10:13
@mestadler
Copy link

Can we look at Gitea as well?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants