Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUGFIX] Force pushing do not work with protected branches #124

Closed
wants to merge 1 commit into from

Conversation

grebois
Copy link

@grebois grebois commented Aug 27, 2018

Error: "I couldn't merge this branch: failed to push merged changes, check my logs!"

GitLab: You are not allowed to force push code to a protected branch on this project.

As mentioned here: https://about.gitlab.com/2014/11/26/keeping-your-code-protected/

Note that even Maintainer is not able to force push to or delete a protected branch.

Error: "I couldn't merge this branch: failed to push merged changes, check my logs!"

`GitLab: You are not allowed to force push code to a protected branch on this project.`

As mentioned here: https://about.gitlab.com/2014/11/26/keeping-your-code-protected/

`Note that even Maintainer is not able to force push to or delete a protected branch.`
@Jellby
Copy link

Jellby commented Aug 27, 2018

Another reason for using a merge strategy?

@JaimeLennox
Copy link
Contributor

Marge only force pushes to feature branches. Do you have a reason for keeping these protected? Usually you would only protect your master branch.

@grebois
Copy link
Author

grebois commented Aug 27, 2018

@JaimeLennox quiet the contrary, why would you not use marge to merge to your master branch? This heavily limits the usability and forces avoidable manual intervention in the CI/CD.

@Jellby What do you mean? it happens on fast-forward and merge commits, tried both.

@jcpetruzza
Copy link
Contributor

@grebois Of course, the idea is to use marge to merge to the master branch. But for merging to master, no git push --force is used, just a call to the GitLab api to merge the branch.

So a force-push is only used when rebasing the changes in the merge-request branch. And the question would be why to protect those branches since they are ephemeral.

@grebois
Copy link
Author

grebois commented Aug 27, 2018

@jcpetruzza in my particular use case, im using the environment branches approach:

https://docs.gitlab.com/ee/workflow/gitlab_flow.html#environment-branches-with-gitlab-flow

In this case, all release branches, like pre-production and production are protected:

image

So I would like to have Marge merge this for me, but she cant, am I missing something?

@Jellby
Copy link

Jellby commented Aug 28, 2018

@grebois marge-bot has a use-merge-strategy option with which it should never try to rebase the source branch, but rather merge it with the target. However, even in that case she will try a no-op push to the source branch (see #112).

@jcpetruzza
Copy link
Contributor

@grebois So, if I understand correctly, you are asking marge to merge a request from Pre-production (protected) to Production, and by the time she tries to do this, Production has moved and is ahead of Pre-production and you get a failure when she tries to rebase Pre-production on top of Production (for which the push-force is needed, unless you prefer merge-commits, as @Jellby suggests).

I'm not sure why your Production branch is ahead of Pre-production in the first-place, though. Maybe you are squashing the commits when moving from Pre-production to Production? What would you want marge to do in that scenario? I mean, even if you were to turn off push-force, it seems to me that moving commits from Production to Pre-production goes against the workflow you are trying to implement

@grebois
Copy link
Author

grebois commented Aug 28, 2018

@jcpetruzza let me show you, I have master (protected) and production (protected) both up-to-date:

screen shot 2018-08-28 at 11 47 11

Now I do a simple commit to master, which makes production behind by one:

screen shot 2018-08-28 at 11 48 01

I create a merge request:

screen shot 2018-08-28 at 11 48 42

and assign it to marge:

screen shot 2018-08-28 at 11 49 12

Minutes later I got, I'm broken on the inside, please somebody fix me..

screen shot 2018-08-28 at 11 51 09

So I go to the logs and I get:

2018-08-28 09:49:31,761 INFO Fetching merge requests assigned to me in mgreboi/pipeline-service...
2018-08-28 09:49:31,937 INFO Running git clone --origin=origin git@gitlab.*********.com:mgreboi/pipeline-service.git /tmpu1g7x8vq/tmpuv1vk8t8
2018-08-28 09:49:32,642 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 config user.email gitlab@dms.ca2go.com
2018-08-28 09:49:32,648 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 config user.name 'Marge Bot'
2018-08-28 09:49:32,652 INFO Got 1 requests to merge;
2018-08-28 09:49:32,652 INFO Attempting to merge the oldest MR...
2018-08-28 09:49:32,652 INFO Processing !6 - 'Testing Marge Merge'
2018-08-28 09:49:32,972 INFO Ensuring MR !6 is mergeable
2018-08-28 09:49:33,117 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 fetch --prune origin
2018-08-28 09:49:33,504 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 checkout -B master origin/master --
2018-08-28 09:49:33,534 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 rebase origin/production
2018-08-28 09:49:33,627 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 rev-parse HEAD
2018-08-28 09:49:33,632 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 rev-parse origin/production
2018-08-28 09:49:33,636 INFO Adding trailers for MR !6
2018-08-28 09:49:33,783 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 filter-branch --force --msg-filter 'TRAILERS='"'"'Reviewed-by: '"'"' python3 /nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/trailerfilter.py' origin/production..master
2018-08-28 09:49:33,981 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 rev-parse HEAD
2018-08-28 09:49:33,986 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 checkout master --
2018-08-28 09:49:33,992 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 diff-index --quiet HEAD
2018-08-28 09:49:33,998 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 ls-files --others
2018-08-28 09:49:34,003 INFO Running git -C /tmpu1g7x8vq/tmpuv1vk8t8 push --force origin master:master
2018-08-28 09:49:34,603 WARNING git returned 1
2018-08-28 09:49:34,603 WARNING stdout: b''
2018-08-28 09:49:34,604 WARNING stderr: b"remote: GitLab: You are not allowed to force push code to a protected branch on this project.        \nTo gitlab.*********.com:mgreboi/pipeline-service.git\n ! [remote rejected] master -> master (pre-receive hook declined)\nerror: failed to push some refs to 'git@gitlab.*********.com:mgreboi/pipeline-service.git'\n"
2018-08-28 09:49:34,604 ERROR Unexpected Exception
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 181, in git
    return _run(*command, env=env, check=True, timeout=timeout_seconds)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 207, in _run
    retcode, process.args, output=stdout, stderr=stderr,
subprocess.CalledProcessError: Command '[b'git', b'-C', b'/tmpu1g7x8vq/tmpuv1vk8t8', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 258, in update_from_target_branch_and_push
    repo.push(source_branch, source_repo_url=source_repo_url, force=True)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 147, in push
    self.git('push', force_flag, source, '%s:%s' % (branch, branch))
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 186, in git
    raise GitError(err)
marge.git.GitError: Command '[b'git', b'-C', b'/tmpu1g7x8vq/tmpuv1vk8t8', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 269, in update_from_target_branch_and_push
    raise CannotMerge('failed to push rebased changes, check my logs!')
marge.job.CannotMerge: failed to push rebased changes, check my logs!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 54, in update_merge_request_and_accept
    source_repo_url=source_repo_url,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 282, in update_from_target_branch_and_push
    assert source_repo_url is not None
AssertionError
2018-08-28 09:49:34,827 INFO Unassigning from MR !6
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 181, in git
    return _run(*command, env=env, check=True, timeout=timeout_seconds)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 207, in _run
    retcode, process.args, output=stdout, stderr=stderr,
subprocess.CalledProcessError: Command '[b'git', b'-C', b'/tmpu1g7x8vq/tmpuv1vk8t8', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 258, in update_from_target_branch_and_push
    repo.push(source_branch, source_repo_url=source_repo_url, force=True)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 147, in push
    self.git('push', force_flag, source, '%s:%s' % (branch, branch))
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 186, in git
    raise GitError(err)
marge.git.GitError: Command '[b'git', b'-C', b'/tmpu1g7x8vq/tmpuv1vk8t8', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 269, in update_from_target_branch_and_push
    raise CannotMerge('failed to push rebased changes, check my logs!')
marge.job.CannotMerge: failed to push rebased changes, check my logs!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/bin/.marge.app-wrapped", line 4, in <module>
    marge.app.main()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/app.py", line 261, in main
    marge_bot.start()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 41, in start
    self._run(repo_manager)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 59, in _run
    projects,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 97, in _process_projects
    self._process_merge_requests(repo_manager, project, merge_requests)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 164, in _process_merge_requests
    merge_job.execute()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 54, in update_merge_request_and_accept
    source_repo_url=source_repo_url,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 282, in update_from_target_branch_and_push
    assert source_repo_url is not None
AssertionError

@Jellby So I go and do the same with use-merge-strategy:

screen shot 2018-08-28 at 11 54 03

Same error, and in the logs:

2018-08-28 09:52:51,296 INFO Fetching merge requests assigned to me in mgreboi/pipeline-service...
2018-08-28 09:52:51,481 INFO Running git clone --origin=origin git@gitlab.*********.com:mgreboi/pipeline-service.git /tmpyp5xyhr0/tmpai7py4f9
2018-08-28 09:52:52,069 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 config user.email gitlab@dms.ca2go.com
2018-08-28 09:52:52,075 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 config user.name 'Marge Bot'
2018-08-28 09:52:52,081 INFO Got 1 requests to merge;
2018-08-28 09:52:52,081 INFO Attempting to merge the oldest MR...
2018-08-28 09:52:52,082 INFO Processing !6 - 'Testing Marge Merge'
2018-08-28 09:52:52,378 INFO Ensuring MR !6 is mergeable
2018-08-28 09:52:52,545 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 fetch --prune origin
2018-08-28 09:52:52,898 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 checkout -B master origin/master --
2018-08-28 09:52:52,907 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 merge origin/production
2018-08-28 09:52:52,912 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 rev-parse HEAD
2018-08-28 09:52:52,916 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 rev-parse origin/production
2018-08-28 09:52:52,920 INFO Adding trailers for MR !6
2018-08-28 09:52:53,118 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 filter-branch --force --msg-filter 'TRAILERS='"'"'Reviewed-by: '"'"' python3 /nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/trailerfilter.py' origin/production..master
2018-08-28 09:52:53,315 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 rev-parse HEAD
2018-08-28 09:52:53,320 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 checkout master --
2018-08-28 09:52:53,327 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 diff-index --quiet HEAD
2018-08-28 09:52:53,333 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 ls-files --others
2018-08-28 09:52:53,338 INFO Running git -C /tmpyp5xyhr0/tmpai7py4f9 push --force origin master:master
2018-08-28 09:52:53,954 WARNING git returned 1
2018-08-28 09:52:53,954 WARNING stdout: b''
2018-08-28 09:52:53,955 WARNING stderr: b"remote: GitLab: You are not allowed to force push code to a protected branch on this project.        \nTo gitlab.*********.com:mgreboi/pipeline-service.git\n ! [remote rejected] master -> master (pre-receive hook declined)\nerror: failed to push some refs to 'git@gitlab.*********.com:mgreboi/pipeline-service.git'\n"
2018-08-28 09:52:53,955 ERROR Unexpected Exception
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 181, in git
    return _run(*command, env=env, check=True, timeout=timeout_seconds)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 207, in _run
    retcode, process.args, output=stdout, stderr=stderr,
subprocess.CalledProcessError: Command '[b'git', b'-C', b'/tmpyp5xyhr0/tmpai7py4f9', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 258, in update_from_target_branch_and_push
    repo.push(source_branch, source_repo_url=source_repo_url, force=True)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 147, in push
    self.git('push', force_flag, source, '%s:%s' % (branch, branch))
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 186, in git
    raise GitError(err)
marge.git.GitError: Command '[b'git', b'-C', b'/tmpyp5xyhr0/tmpai7py4f9', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 267, in update_from_target_branch_and_push
    raise CannotMerge('failed to push merged changes, check my logs!')
marge.job.CannotMerge: failed to push merged changes, check my logs!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 54, in update_merge_request_and_accept
    source_repo_url=source_repo_url,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 282, in update_from_target_branch_and_push
    assert source_repo_url is not None
AssertionError
2018-08-28 09:52:54,241 INFO Unassigning from MR !6
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 181, in git
    return _run(*command, env=env, check=True, timeout=timeout_seconds)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 207, in _run
    retcode, process.args, output=stdout, stderr=stderr,
subprocess.CalledProcessError: Command '[b'git', b'-C', b'/tmpyp5xyhr0/tmpai7py4f9', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 258, in update_from_target_branch_and_push
    repo.push(source_branch, source_repo_url=source_repo_url, force=True)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 147, in push
    self.git('push', force_flag, source, '%s:%s' % (branch, branch))
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/git.py", line 186, in git
    raise GitError(err)
marge.git.GitError: Command '[b'git', b'-C', b'/tmpyp5xyhr0/tmpai7py4f9', b'push', b'--force', b'origin', b'master:master']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 267, in update_from_target_branch_and_push
    raise CannotMerge('failed to push merged changes, check my logs!')
marge.job.CannotMerge: failed to push merged changes, check my logs!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/bin/.marge.app-wrapped", line 4, in <module>
    marge.app.main()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/app.py", line 261, in main
    marge_bot.start()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 41, in start
    self._run(repo_manager)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 59, in _run
    projects,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 97, in _process_projects
    self._process_merge_requests(repo_manager, project, merge_requests)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 164, in _process_merge_requests
    merge_job.execute()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 54, in update_merge_request_and_accept
    source_repo_url=source_repo_url,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 282, in update_from_target_branch_and_push
    assert source_repo_url is not None
AssertionError

Marge is a maintainer of the project:

screen shot 2018-08-28 at 11 59 36

and I am doing Merge Commits, it also happens with fast-forward

screen shot 2018-08-28 at 12 02 11

not sure what I'm doing wrong if this is not the expected behavior.

@jcpetruzza
Copy link
Contributor

@grebois Ok, I think this makes sense. So, your master branch should by construction never be behind production, which means that when marge rebases master onto production, nothing should change. It then does a push-force regardless since, assuming the branch is not protected, that will be a no-op anyway.

For a rebase-based workflow, maybe the right thing to do would be not to do the push-force if after rebasing the head of the source branch hasn't moved (i.e., make the no-op explicit) since that will then also work on protected branches. However. this may not won't work for a merge-based workflow, since after the merge, a merge commit will have been created anyway, so pushing is necessary (am I right here, @Jellby?)

So in my opinion a more robust solution could perhaps be:

  • Always attempt the push-force
  • If it fails because the branch is protected, optimistically try to do a normal push
  • if this also fails, fail with a better error message

@jcpetruzza
Copy link
Contributor

@grebois In any case, the reason you are using marge for this is for uniformity since you use it for the previous stages of development, right? I mean, otherwise, you could just accept the merge request directly, or am I missing something?

@Jellby
Copy link

Jellby commented Aug 28, 2018

For a rebase-based workflow, maybe the right thing to do would be not to do the push-force if after rebasing the head of the source branch hasn't moved (i.e., make the no-op explicit) since that will then also work on protected branches. However. this may not won't work for a merge-based workflow, since after the merge, a merge commit will have been created anyway, so pushing is necessary (am I right here, @Jellby?)

I'm not sure what happens with a "merge commit", what I'd recommend is the "fast-forward" option in GitLab, and the use-merge-strategy in marge-bot. If the MR is not fast-forwardable, marge should abort and not merge it. The MR creator can then merge with the target branch and push, and reassign to marge. If we expect marge to do this by herself, then yes, she'd need push permission to the source branch in order to push the new merge (but not forced!)

So in my opinion a more robust solution could perhaps be:
* Always attempt the push-force
* If it fails because the branch is protected, optimistically try to do a normal push
* if this also fails, fail with a better error message

I still think there should be a prior check as in #112: If there is no need, don't try pushing at all. Then, if a push is needed, I'd rather check first if it requires -force or not.

@JaimeLennox
Copy link
Contributor

I agree we should have a more appropriate error.

@grebois The force push is needed here. Marge has the ability to add trailers (as explained in https://github.com/smarkets/marge-bot#adding-reviewed-by-tested-and-part-of-to-commit-messages), and the process of doing this rewrites the commits; hence the force push is required to update the repo with the rewritten commits.

I can see from the logs you posted that Marge is trying to add the "Reviewed-By" trailer; if you would prefer not to have a force push, then I think not setting any options for trailers should solve this. Can you try running Marge like this and see if this is still an issue?

@grebois
Copy link
Author

grebois commented Aug 29, 2018

@JaimeLennox still with fast-forward and

docker run --name marge-bot -v /srv/marge-bot/:/srv/marge-bot -e MARGE_AUTH_TOKEN_FILE=/srv/marge-bot/marge-bot.token -e MARGE_SSH_KEY_FILE=/srv/marge-bot/marge-bot-ssh-key smarkets/marge-bot --ci-timeout 30m --use-merge-strategy --gitlab-url=

I get:

2018-08-29 12:10:58,748 INFO Got 1 requests to merge;
2018-08-29 12:10:58,750 INFO Attempting to merge the oldest MR...
2018-08-29 12:10:58,750 INFO Processing !6 - 'Testing Marge Merge'
2018-08-29 12:10:59,032 INFO Ensuring MR !6 is mergeable
2018-08-29 12:10:59,171 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt fetch --prune origin
2018-08-29 12:10:59,523 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt checkout -B master origin/master --
2018-08-29 12:10:59,531 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt merge origin/production
2018-08-29 12:10:59,538 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt rev-parse HEAD
2018-08-29 12:10:59,542 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt rev-parse origin/production
2018-08-29 12:10:59,546 INFO Adding trailers for MR !6
2018-08-29 12:10:59,547 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt checkout master --
2018-08-29 12:10:59,554 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt diff-index --quiet HEAD
2018-08-29 12:10:59,559 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt ls-files --others
2018-08-29 12:10:59,563 INFO Running git -C /tmpsuegoy4h/tmpm5h90vxt push --force origin master:master
2018-08-29 12:10:59,931 ERROR Unexpected Exception
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 54, in update_merge_request_and_accept
    source_repo_url=source_repo_url,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 282, in update_from_target_branch_and_push
    assert source_repo_url is not None
AssertionError
2018-08-29 12:11:00,147 INFO Unassigning from MR !6
Traceback (most recent call last):
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/bin/.marge.app-wrapped", line 4, in <module>
    marge.app.main()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/app.py", line 261, in main
    marge_bot.start()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 41, in start
    self._run(repo_manager)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 59, in _run
    projects,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 97, in _process_projects
    self._process_merge_requests(repo_manager, project, merge_requests)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/bot.py", line 164, in _process_merge_requests
    merge_job.execute()
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 24, in execute
    self.update_merge_request_and_accept(approvals)
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/single_merge_job.py", line 54, in update_merge_request_and_accept
    source_repo_url=source_repo_url,
  File "/nix/store/77wrcwqx8p2wbw3n0v5v4lsvh7bwn01w-python3.6-marge-0.7.0/lib/python3.6/site-packages/marge/job.py", line 282, in update_from_target_branch_and_push
    assert source_repo_url is not None
AssertionError

it always force pushes :/

@Jellby
Copy link

Jellby commented Aug 29, 2018

It always force-pushes indeed. In my tests I couldn't get a failure when manually trying to force-push a no-op to a protected branch. But then again, I didn't get a failure when trying to push a no-op to a branch where I have no push permission. However, the latter failed for me with marge-bot, so I guess the former is failing for you. It must be something related to the git settings...

@JaimeLennox
Copy link
Contributor

JaimeLennox commented Aug 29, 2018

The force push is a no-op at this point. I don't see the error from GitLab anymore, so I think this is a separate issue - and probably related to the fact that you are merging from master to somewhere else (which is a case we don't consider that often).

It seems we assume that if you're merging from master, then you are also merging from a forked repo. This is obviously not always the case, and should be fixed, but it's unrelated to the force pushing (which is fine at this stage).

@grebois
Copy link
Author

grebois commented Aug 29, 2018

@JaimeLennox I get the same error between two randomly protected branches.

@Jellby You were right, it was related to git setting, now it works without "Reject unsigned commits" which was for some reason enabled, any way to make Marge sign the commits?

@Jellby
Copy link

Jellby commented Aug 30, 2018

So in my opinion a more robust solution could perhaps be:
* Always attempt the push-force
* If it fails because the branch is protected, optimistically try to do a normal push
* if this also fails, fail with a better error message

I still think there should be a prior check as in #112: If there is no need, don't try pushing at all. Then, if a push is needed, I'd rather check first if it requires -force or not.

Or at least do a final check before failing at the end: Is the commit I want to push identical to the current remote one? Yes: issue a warning. No: fail

@JaimeLennox
Copy link
Contributor

I've double checked that force pushing to a protected branch is possible without errors as long as it's a fast forward.

@grebois in the workflow you describe, this would therefore not be an issue, provided each successive branch is simply behind the one prior. Of course, if you do cherry-picks across them then this would not work, as a rebase would be required and Marge would then need to force push.

I've therefore added a separate issue (#128) in favour of this to respond with a more appropriate message for protected branches, and will close this MR as I don't see the solution here as relevant to the issue at hand.

Int terms of signing commits, it would be relatively easy to get Marge to do so (by providing a GPG key), but I'm not sure this makes sense. If Marge doesn't need to rewrite commits, then there isn't an issue, as the commits will stay signed; if Marge does need to rewrite commits, then she would have to use her own key, which of course means the commits are no longer signed by the original authors - to me, this defeats the purpose of doing so.

@JaimeLennox JaimeLennox closed this Sep 2, 2018
JaimeLennox added a commit that referenced this pull request Sep 2, 2018
Fixes an issue uncovered by #124 where Marge throws an assertion error
if the source/target branches are from the same project and the source
branch is master. We assume that merges will always be into master,
which is obviously not always the case; you may have a workflow
requiring you to merge from master into another branch (e.g. a
staging/production branch).
@grebois grebois deleted the patch-1 branch September 18, 2018 16:01
@grebois
Copy link
Author

grebois commented Sep 18, 2018

@JaimeLennox @Jellby thanks a lot for your overall effort on this project and your constant and insightful follow up on this particular topic, great work!

aschmolck pushed a commit that referenced this pull request Oct 1, 2018
Fixes an issue uncovered by #124 where Marge throws an assertion error
if the source/target branches are from the same project and the source
branch is master. We assume that merges will always be into master,
which is obviously not always the case; you may have a workflow
requiring you to merge from master into another branch (e.g. a
staging/production branch).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants