Add Azure DevOps VersionController #476

rerikson · 2024-04-22T20:19:54Z

What does this change

This adds a new VersionController implementation for Azure DevOps

For example if it introduces a new flag or modifies a commands output, give an example of you running the command and showing real output here.

Redacted Command

 multi-gitter  run ./examples/general/replace.sh  --conflict-strategy replace --git-type cmd -m "test message" --token <faketoken> -g https://dev.azure.com/<my-org> -p azuredevops -R <ProjectName>/<RepoName> reviewers test@mydomain.com

Redacted Output

INFO[0001] Running on 1 repositories                    
INFO[0001] Cloning and running script                    repo=<MyProject>/<MyRepo>
INFO[0003] Pushing changes to remote                     repo=<MyProject>/<MyRepo>
INFO[0004] Creating pull request                         repo=<MyProject>/<MyRepo>

What issue does it fix

Closes #475

If there is not an existing issue, please make sure we have context on why this change is needed. .

Notes for the reviewer

go is not my main language, but hopefully this is pretty close to what you would be looking for in a new VersionController for AzureDevops. I tried my best to follow existing implementations, but I'm happy to make any changes!

Checklist

[x ] Made sure the PR follows the CONTRIBUTING.md guidelines
[ x] Tests if something new is added

lindell · 2024-04-23T10:25:24Z

Thanks for the PR! I will take a closer look in the coming days.

Just briefly browsing the cmd/ change I noticed the naming of things in Azure Devops seems to colide with the naming of GitLab. Just to make sure I understand the terminology correctly. Is this how things are named?

Github	GitLab	Azure Devops
Repository	Project	Repository
Organization	Group (hierarchical)	Project

So in Azure Devops the only levels are Repositories that can be within one project? There are no stacking projects, and no higher level (organizations etc)?

rerikson · 2024-04-23T16:06:40Z

Hey thanks for the question! I tried to make the naming in the code match the naming in Azure DevOps and you're right it does collide with GitLab. The chart you made of Github/GitLab/Azure DevOps is correct. Azure DevOps does have the concept of organizations, but each organization would have a different base url and is pretty much a totally separate entity. There are no stacked/sub projects either.

lindell

Just had time to look at some initial code in cmd. If you want to TAL, please do, otherwise feel free to wait until I have time to look at the rest of the code.

cmd/platform.go

internal/scm/azuredevops/azuredevops.go

rerikson · 2024-05-02T18:18:24Z

Hey @lindell, no rush on the rest of the code review, but just wanted to let you know I pushed up a commit to address the comments so far.

lindell

Hello again,

Nice that you had time to look at the first comment. I've had very little time to work on this the last week, but I've been able to go through about half of the PR. In general I think it's great since you are not usually working with Go.
I publish the new comments, but as the last time, please feel free to ignore them until the entire review is done if you want.

If there is one comment to take a look at, it's about getting repositories by name, and not fetching all of them.

internal/scm/azuredevops/azuredevops.go

lindell · 2024-04-30T05:09:56Z

internal/scm/azuredevops/azuredevops.go

+		return nil, err
+	}
+
+	locationClient := location.NewClient(context.Background(), connection)


Using a context.Background here does mean that this request will not be cancelable (by pressing Ctrl+C or similar).

I think we should either create a new client for each time we need it, with a cache (cache seems to already be implemented internally in the library). But with a help method, func (a *AzureDevOps) connection(ctx context.Context) .

Or make sure we send in a context in the creation.

I think I understand what you mean, but let me make sure. Are you suggesting I defer creating the clients until I actually need to use them inside the various methods: ForkRepository, GetRepositories, etc., pass the context into the NewClient method at that time, and store it on the *AzureDevOps reference?

Yes, that should probably be the way to do it 🙂

lindell · 2024-05-01T18:02:27Z

internal/scm/azuredevops/azuredevops.go

+	projectMap := make(map[string]uuid.UUID)
+
+	if fork {
+		projects, err := coreClient.GetProjects(context.Background(), core.GetProjectsArgs{})


Will this always return all projects? Even if there are thousands of them? If not I think it might be cleaner to get the name when needed (with the same cache structure), slightly more inefficient in some scenarios, and slightly less in others (with very new forks).

Yeah, that will return all projects even if there are thousands. Are you thinking I should change this to a search by project name on demand?

Yes, I think that would be the way to do it if the API allows

internal/scm/azuredevops/azuredevops.go

lindell · 2024-05-03T04:49:46Z

internal/scm/azuredevops/azuredevops.go

+}
+
+func (a *AzureDevOps) getRepos(ctx context.Context) (*[]git.GitRepository, error) {
+	allRepos, err := a.gitClient.GetRepositories(ctx, git.GetRepositoriesArgs{})


There is no way of getting a repository by name? (project + name)

Will skip to review this function until I know.

I looked and there is a way of getting repositories by name as long as you also have the project name

In that case (if we can avoid fetching all repositories in the entire org). We should instead do a fetching for only the projects / repositories defined.

internal/scm/azuredevops/azuredevops.go

…o add-azuredevops

internal/scm/azuredevops/azuredevops.go

lindell · 2024-05-09T10:47:46Z

internal/scm/azuredevops/azuredevops.go

+		return err
+	}
+
+	newObjectID := "0000000000000000000000000000000000000000"


Please add a comment of what is happening here

Added a comment but let me know if it's not sufficient. I pulled this value from the docs here: https://learn.microsoft.com/en-us/rest/api/azure/devops/git/refs/update-refs?view=azure-devops-rest-7.1&tabs=HTTP

Please add the link to the docs as well and we should be good here

lindell · 2024-05-09T10:50:35Z

Tried to review through vscode, apparently it posts every comment and not together. Sorry if this was spammy to you.

lindell

I'm now done with the review, thanks again for the PR!

Please let me know if you have any follow up questions, or if you need any assistance with fixing any of the comments. I can't guarantee any timeline, but I'm happy to help.

lindell · 2024-05-09T18:45:34Z

internal/scm/azuredevops/azuredevops.go

+	if err == nil {
+		return a.convertRepo(existingFork), nil
+	}
+


This will not be corect the correct behaviour if the error is for example a network error. (A new fork will be requested).

Something similar to this need to be made:

Suggested change

if err == nil {

return a.convertRepo(existingFork), nil

}

if err == nil {

return a.convertRepo(existingFork), nil

} else if !errors.Is(err, core.NotFoundError) {

return nil, err

}

Note: I have not verified how to check notfound errors (core.NotFoundError does not exist) with this API. But it can either be a constant error (use errors.Is), or a type error (use errors.As)

I found a way to check not found errors that I think makes sense and pushed that up. Let me know if you think I should do something else please.

lindell · 2024-05-09T18:50:02Z

internal/scm/azuredevops/azuredevops.go

+		np = r.ownerName
+	}
+
+	repoName := r.name + "." + strings.ReplaceAll(currentUser, " ", ".")


Forks can only be made to the current user in Azure devops? Not to projects (like to organizations in GitHub)

Forks can actually only be made to Projects in Azure devops. In the lines above, I am appending the username to the end of the repo name so that multiple forks of the repo can be forked into the same project. The project to fork the repo into gets passed into this method through the newProject variable.

Might this behavior be something users want to configure? I'm fine if you want to start with this, and potentially add it later.

internal/scm/azuredevops/azuredevops_test.go

internal/scm/azuredevops/repository.go

Co-authored-by: Johan Lindell <johan@lindell.me>

…itter into add-azuredevops

Co-authored-by: Johan Lindell <johan@lindell.me>

rerikson · 2024-06-28T20:37:35Z

Thank you very much for the thorough PR and being kind to my go code! Also, sorry it took so long for me to get back to this. I'll try and get any remaining changes fixed much faster. Any changes I pushed up I closed the conversation for and I left open any conversations that seemed like further discussion might be needed.

I think we're getting close on this!

lindell

Sorry for the long review time. Got it in the beginning of a longer vacation, and had forgot about it when I came back. Should be faster to respond to any subsequent messages.

lindell · 2024-09-09T18:58:18Z

internal/scm/azuredevops/azuredevops.go

+		return nil, err
+	}
+
+	locationClient := location.NewClient(context.Background(), connection)


Yes, that should probably be the way to do it 🙂

lindell · 2024-09-09T18:59:24Z

internal/scm/azuredevops/azuredevops.go

+	projectMap := make(map[string]uuid.UUID)
+
+	if fork {
+		projects, err := coreClient.GetProjects(context.Background(), core.GetProjectsArgs{})


Yes, I think that would be the way to do it if the API allows

lindell · 2024-09-09T19:07:36Z

internal/scm/azuredevops/azuredevops.go

+}
+
+func (a *AzureDevOps) getRepos(ctx context.Context) (*[]git.GitRepository, error) {
+	allRepos, err := a.gitClient.GetRepositories(ctx, git.GetRepositoriesArgs{})


In that case (if we can avoid fetching all repositories in the entire org). We should instead do a fetching for only the projects / repositories defined.

internal/scm/azuredevops/azuredevops.go

lindell · 2024-09-09T19:10:16Z

internal/scm/azuredevops/azuredevops.go

+		return err
+	}
+
+	newObjectID := "0000000000000000000000000000000000000000"


Please add the link to the docs as well and we should be good here

lindell · 2024-09-09T19:12:15Z

internal/scm/azuredevops/azuredevops.go

+		np = r.ownerName
+	}
+
+	repoName := r.name + "." + strings.ReplaceAll(currentUser, " ", ".")


Might this behavior be something users want to configure? I'm fine if you want to start with this, and potentially add it later.

rerikson and others added 4 commits April 22, 2024 19:08

fix

e90c59a

Merge https://github.com/lindell/multi-gitter into add-azuredevops

6cea5f8

go mod

87dc1c0

Add update

6286ec4

lindell reviewed Apr 24, 2024

View reviewed changes

fix initial PR comments

d82b912

lindell reviewed May 3, 2024

View reviewed changes

Merge branch 'master' of https://github.com/rerikson/multi-gitter int…

b1a5059

…o add-azuredevops

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

internal/scm/azuredevops/azuredevops.go Outdated Show resolved Hide resolved

lindell reviewed May 9, 2024

View reviewed changes

rwerikson and others added 6 commits June 27, 2024 11:21

fix issues from inital commit

f8440e7

Update internal/scm/azuredevops/azuredevops.go

1b10990

Co-authored-by: Johan Lindell <johan@lindell.me>

More PR comment fixes

1b20431

Merge branch 'add-azuredevops' of https://github.com/rerikson/multi-g…

bbe4a38

…itter into add-azuredevops

fix test

c1092ae

Co-authored-by: Johan Lindell <johan@lindell.me>

change url parse method

5f059d7

Co-authored-by: Johan Lindell <johan@lindell.me>

fix additional issues

9fa8e28

logic fix

0e944a0

lindell reviewed Sep 9, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Azure DevOps VersionController #476

Add Azure DevOps VersionController #476

rerikson commented Apr 22, 2024

lindell commented Apr 23, 2024

rerikson commented Apr 23, 2024

lindell left a comment

rerikson commented May 2, 2024

lindell left a comment

lindell Apr 30, 2024

rerikson Jun 28, 2024

lindell Sep 9, 2024

lindell May 1, 2024

rerikson Jun 28, 2024

lindell Sep 9, 2024

lindell May 3, 2024

rerikson Jun 28, 2024

lindell Sep 9, 2024

lindell May 9, 2024

rerikson Jun 28, 2024

lindell Sep 9, 2024

lindell commented May 9, 2024

lindell left a comment

lindell May 9, 2024

rerikson Jun 28, 2024

lindell May 9, 2024

rerikson Jun 28, 2024

lindell Sep 9, 2024

rerikson commented Jun 28, 2024

lindell left a comment

lindell Sep 9, 2024

lindell Sep 9, 2024

lindell Sep 9, 2024

lindell Sep 9, 2024

lindell Sep 9, 2024

Add Azure DevOps VersionController #476

Are you sure you want to change the base?

Add Azure DevOps VersionController #476

Conversation

rerikson commented Apr 22, 2024

What does this change

What issue does it fix

Notes for the reviewer

Checklist

lindell commented Apr 23, 2024

rerikson commented Apr 23, 2024

lindell left a comment

Choose a reason for hiding this comment

rerikson commented May 2, 2024

lindell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lindell commented May 9, 2024

lindell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rerikson commented Jun 28, 2024

lindell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment