Fix: Set correct branch when it's not specified in the config #5844

bytrangle · 2021-09-27T09:27:00Z

Summary
Currently, the default branch in Github, Gitlab and Bitbucket is set to be main instead of master.

However, when people use Netlify CMS and don't specify the branch name to push content in the config file, Netlify CMS will automatically push to branch master.

Since this branch may not exist, users will get an error on the frontend: "API_ERROR: Branch not found".

Test plan

In the admin/config.yml file, specify a valid Github repo path to push content to.
Run yarn start.
Go to Neltify CMS dashboard, create a new post and hit Publish. You shouldn't get an error. Checking back on the repo that you specified in the config, you should see the new post.

Checklist

Please add a x inside each checkbox:

I have read the contribution guidelines.
Code is formatted via running yarn format.
Tests are passing via running yarn test.
The status checks are successful (continuous integration). Those can be seen below.

bytrangle · 2021-09-29T14:44:38Z

Please hold off on reviewing until I've pushed the unit tests.

erezrokah

Thanks @bytrangle, great work as always.

I've added some comments, please let me know your thoughts.
Also, are we planning to do this for other backends too?

erezrokah · 2021-09-30T11:13:16Z

packages/netlify-cms-backend-github/src/implementation.tsx

+      if (!masterBranch) {
+        // Get default branch of the repo
+        const defaultBranch = await this.api!.newGetDefaultBranch();
+        this.branch = defaultBranch || '';


Suggested change

this.branch = defaultBranch || '';

this.branch = defaultBranch || 'master';

Should we default to master to ensure the property is set (this will avoid calling setDefaultBranch again if authenticate is re-invoked).

Two things:

I didn't consider the scenario that authenticate can be revoked. I will change it as requested.Good point.

I was going to apply the same approach to other backend, but then I realized that would mean writing some repetitive code. That will open up more room for errors and make it harder to track changes in the future. So what do you think about adding a helper in Netlify Utils to get a default branch and call that in each respective backend?

2. Netlify Utils to get a default branch and call that in each respective backend?

Sounds good! How about adding the helpers to https://github.com/netlify/netlify-cms/blob/master/packages/netlify-cms-lib-util/src/backendUtil.ts?

Should it be backendUtils or API.ts is the place?
And when do you need these changes? I normally don't work on open source on weekend, but if you need it soon, I can work on this issue on Sunday.

Either backendUtils or API.ts work (or both) depending on the common logic you extract.

I normally don't work on open source on weekend, but if you need it soon, I can work on this issue on Sunday.

I definitely don't need it soon, and in any case no need to change your work schedule for us. We are the ones who should adjust our schedule to contributors.

packages/netlify-cms-backend-github/src/implementation.tsx

erezrokah · 2021-09-30T11:14:55Z

packages/netlify-cms-backend-github/src/API.ts

@@ -1188,6 +1189,16 @@ export default class API {
    return result;
  }

+  async newGetDefaultBranch() {
+    try {


How about we drop the try/catch and let the calling function handle any errors (it already has a try/catch)?

This requestConfig object will be passed to a helper for making API request

Include switch clause to construct API urls for different backends

…ured The property is needed so that we'll only set default branch when branch prop is missing in config

…config This is needed so that this property won't be empty when authorization is revoked.

Reason: Requesting information from a public repo doesn't require token

bytrangle · 2021-10-12T09:21:58Z

Sorry that it took such a long time to update this PR. Writing a helper that can accommodate all the major git hosts turn out to be more complex than I had expected.

bytrangle · 2021-10-27T09:59:55Z

Hello @erezrokah . Let me ask a noob question: How do I rerun the Github action workflow? Or is it only possible for team members?

erezrokah · 2021-10-27T10:26:27Z

No noob questions - do you have this button when you open the action?

erezrokah · 2021-10-27T10:27:53Z

Regardless, I invited you to collaborate on the repo, long overdue ❤️

stale · 2023-04-26T09:26:38Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

martinjagodic · 2023-10-16T09:49:42Z

@bytrangle are you still interested in moving this forward?

paulRbr · 2024-02-28T09:55:49Z

I think this PR is interesting to be merged upstream. I was going to suggest a much more simple change (changing the default value of the branch to main) but I think this PR is better.

martinjagodic · 2024-02-28T11:53:08Z

@paulRbr yes, we would like to have this merged, but we need to solve those merge conflicts first.

bytrangle · 2024-03-14T13:00:49Z

@martinjagodic Hi, I'm interested in working on this again. However, I may have to ask you about resolving merging conflicts later :).

martinjagodic · 2024-03-14T13:03:15Z

@bytrangle awesome! The conflicts emerged when we renamed the packages from netlify-cms-* to decap-cms-*. If a rename does not suffice, I am happy to answer questions/help out.

bytrangle · 2024-04-01T10:08:00Z

Hi @demshy . Is there anything else I need to do?

demshy · 2024-04-02T07:04:26Z

Sorry @bytrangle I somehow overlooked this one. I added a missing import that made the unit tests fail. I will gladly merge this one into a release this week

* feat: add helper for fetching default branch from Github * feat: add method for setting default branch * fix: set default branch after user has authenticated successfully * fix: format code * feat: add unit test for getting default branch name * feat: add helpers for parsing API responses * feat(lib-util): add helper for constructing request headers * feat(lib-util): add helper for constructing full URL for API request * feat(lib-util): store base URLs for each backend * feat(lib-util): add type annotation for the request config This requestConfig object will be passed to a helper for making API request * feat(lib-util): add helper for handle API request error * feat(lib-util): add config for making api request * feat(lib-util): add api request generator * feat(lib-util): add helper for getting default branch name Include switch clause to construct API urls for different backends * feat(lib-util): export method for getting default branch name * feat(gh-backend): add a boolean property to check if branch is configured The property is needed so that we'll only set default branch when branch prop is missing in config * feat(gh-backend): set prop `branch` as `master` when it's missing in config This is needed so that this property won't be empty when authorization is revoked. * feat(gh-backend): set branch name when it's missing in config * feat(gitlab-backend): set branch when it's not in the config * feat(bitbucket-backend): set branch when it's not specified in config * feat(lib-util): allow token type to be undefined Reason: Requesting information from a public repo doesn't require token * fix: format codes * feat(github): removed setDefaultBranch function Reason: Default branch is already set when calling `authenticate` function * feat(github): remove function for getting default branch * fix (github): removed GithubRepo object because it was never used * fix (gitlab test): Repeat response for getting project info 2 times Reason: The endpoint for getting Gitlab project info is called twice. Need to specify the number of times to repeat the same response as 2, or Nock will throw an error. * fix(gitlab test): add property `default_branch` to project response REASON: Getting a single project through `/projects/:id` returns an object which contains `default_branch` property, among many other props. The mock response needs to convey that. * fix(gitlab test): reformat codes * feat(lib util api): change function name Change from `constructUrl` to `constructUrlWithParams` to indicate that the returned url may contain query string * feat(lib-util api): Change variable name for storing API roots Change from `rootApi` to `apiRoots` to indicate that the varible contains multiple values * feat(lib-util api): Add varialbe for storing endpoint constants * feat(lib-util api): Change the returned value for `getDefaultBranchName` Reason: There's no `null` value for default_branch * feat(api test): Import Nock module for mocking API requests * feat(api test): Add default values for mocking API Default values include: default branch name, mock tokens and mock repo slug * feat(api test): Add mock response to getting a single repo * feat(api test): Add function for itnercepting request to get single repo * feat(api test): Add test for gettingDefaultBranchName * feat(lib-util): reformat codes * feat(jest config): add moduleNameMapper for GitHub and BitBucket Required for the test that checks setDefaultBranchName is called in lib-util to work * feat(lib-util test): return some modules from backend core for testing * feat(lib-util test): add owner login value for Github's repo response The authenticate method of Github API wrapper extracts owner.login from repo resp * feat(lib-util test): change access level for Gitlab to 30 Reason: If access level is under 30, Gitlab package will throw error * feat(lib-util test): add mock response for getting a user The authenticate method of each backend requires this * feat(lib-util test): add default config for backend field * feat(lib-util test): rewrite function for mocking API * feat(lib-util test): rewrite function for mocking request for repo * test(lib-util): rewrite test for the function getDefaultBranchName * test(lib-util): add function for resolving backend * test(lib-util): import 'set' module from Lodash * test(lib-util): add helper for constructing url path for each backend * test(lib-util): add function for intercepting API request to authenticate * test(lib-util): import each backend module * test(lib-util): add tests that check each backend calls getDefaultBranchName * style: format files * fix: query branch name before setting Github API service * fix: reformat implementation module of Github backend * fix: remove importing of getDefaultBranchName from lib * fix: removed test for getDefaultBranchName in lib packages * fix: removed unused vars in api test for lib package * feat: retrieve default branch before creating Bitbucket AI instance * fix: reformat codes in Bitbucket implementation module * fix: add missing import --------- Co-authored-by: Erez Rokah <erezrokah@users.noreply.github.com> Co-authored-by: Martin Jagodic <jagodicmartin1@gmail.com> Co-authored-by: Anze Demsar <anze.demsar@p-m.si>

floscher · 2024-06-28T16:15:54Z

packages/netlify-cms-lib-util/src/API.ts

  if (token) {
-    baseHeaders['Authorization'] = `token ${token}`
+    baseHeaders['Authorization'] = `token ${token}`;


Hi @bytrangle, do you by any chance remember, why this is using token instead of Bearer here?
This seems to be causing an issue with Gitlab logins, since this PR was merged in April: #7172 (comment)

I'm not sure about Github/Bitbucket, but it seems at least for Gitlab this line should be like this:

Suggested change

baseHeaders['Authorization'] = `token ${token}`;

baseHeaders['Authorization'] = `Bearer ${token}`;

For reference: This line was originally introduced in 3edc3ce, this comment is on a2add4a, which was later squashed to c91a70f

I don't remember. This corresponds to Gitlab docs 13.0, which is no longer available. Please feel free to make a new PR to correct it.

Thanks anyway, I now created #7242 for this change.

Nice work @floscher

I cloned the decapCMS repo locally yesterday to double check the fix.

After adding the fix and rebuilding decapCMS, I no longer got the previous error message.

However, I then still got an "API_ERROR: Branch not found" message, implying that the code this patch was meant to fix doesn't do it's job on gitlab anyway..

Only after hard-coding "branch: main" into my config.yml, was I then able to get on with things.

…Bearer" (decaporg#7242) The authorization type "Bearer" is more widely recognized than "token". E.g. Gitlab requires that you use "Bearer". This should fix decaporg#7172 where apparently login with Github was broken by this line: decaporg#7172 (comment) See also: decaporg#5844 (comment)

…Bearer" (#7242) The authorization type "Bearer" is more widely recognized than "token". E.g. Gitlab requires that you use "Bearer". This should fix #7172 where apparently login with Github was broken by this line: #7172 (comment) See also: #5844 (comment)

bytrangle added 4 commits September 27, 2021 11:46

feat: add helper for fetching default branch from Github

47ebd16

feat: add method for setting default branch

61904c8

fix: set default branch after user has authenticated successfully

c3f1fc0

fix: format code

90c9f9c

bytrangle requested a review from a team September 27, 2021 09:27

erezrokah added the type: bug code to address defects in shipped code label Sep 29, 2021

feat: add unit test for getting default branch name

43fbe2d

erezrokah suggested changes Sep 30, 2021

View reviewed changes

bytrangle added 17 commits October 11, 2021 12:02

feat: add helpers for parsing API responses

8436443

feat(lib-util): add helper for constructing request headers

3edc3ce

feat(lib-util): add helper for constructing full URL for API request

7ccbac6

feat(lib-util): store base URLs for each backend

e5407c3

feat(lib-util): add type annotation for the request config

3f0e711

This requestConfig object will be passed to a helper for making API request

feat(lib-util): add helper for handle API request error

a506b5d

feat(lib-util): add config for making api request

cc5f53a

feat(lib-util): add api request generator

cec8fbb

feat(lib-util): add helper for getting default branch name

69d124a

Include switch clause to construct API urls for different backends

feat(lib-util): export method for getting default branch name

c726ee0

feat(gh-backend): add a boolean property to check if branch is config…

bbafe77

…ured The property is needed so that we'll only set default branch when branch prop is missing in config

feat(gh-backend): set prop branch as master when it's missing in …

11e187c

…config This is needed so that this property won't be empty when authorization is revoked.

feat(gh-backend): set branch name when it's missing in config

67904b1

feat(gitlab-backend): set branch when it's not in the config

37531e1

feat(bitbucket-backend): set branch when it's not specified in config

b118aea

feat(lib-util): allow token type to be undefined

fa8e03c

Reason: Requesting information from a public repo doesn't require token

fix: format codes

a2add4a

bytrangle added 5 commits May 17, 2022 16:50

fix: remove importing of getDefaultBranchName from lib

ec0ff89

fix: removed test for getDefaultBranchName in lib packages

48e7175

fix: removed unused vars in api test for lib package

676631f

feat: retrieve default branch before creating Bitbucket AI instance

614a5b0

fix: reformat codes in Bitbucket implementation module

20caf5f

stale bot added the status: stale label Apr 26, 2023

martinjagodic removed the status: stale label Apr 28, 2023

fix: Resolve conflicts from changing cms

8db27d6

bytrangle requested a review from a team as a code owner March 18, 2024 09:55

Merge branch 'master' into fix/default-branch

ad161df

martinjagodic requested review from demshy and removed request for a team March 18, 2024 13:00

demshy added 2 commits April 2, 2024 08:43

Merge branch 'main' into fix/default-branch

d4e4fe3

fix: add missing import

c6b05f8

demshy approved these changes Apr 2, 2024

View reviewed changes

Merge branch 'main' into fix/default-branch

5827471

demshy merged commit c91a70f into decaporg:main Apr 3, 2024
9 checks passed

floscher reviewed Jun 28, 2024

View reviewed changes

floscher mentioned this pull request Jun 29, 2024

Fix login with Gitlab (change API Authorization header "token" → "Bearer") #7242

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix: Set correct branch when it's not specified in the config #5844

Fix: Set correct branch when it's not specified in the config #5844

bytrangle commented Sep 27, 2021

bytrangle commented Sep 29, 2021

erezrokah left a comment

erezrokah Sep 30, 2021

bytrangle Oct 1, 2021 •

edited

Loading

erezrokah Oct 1, 2021

bytrangle Oct 1, 2021

erezrokah Oct 1, 2021 •

edited

Loading

erezrokah Sep 30, 2021

bytrangle commented Oct 12, 2021

bytrangle commented Oct 27, 2021

erezrokah commented Oct 27, 2021

erezrokah commented Oct 27, 2021

stale bot commented Apr 26, 2023

martinjagodic commented Oct 16, 2023

paulRbr commented Feb 28, 2024

martinjagodic commented Feb 28, 2024

bytrangle commented Mar 14, 2024

martinjagodic commented Mar 14, 2024

bytrangle commented Apr 1, 2024

demshy commented Apr 2, 2024

floscher Jun 28, 2024 •

edited

Loading

bytrangle Jun 29, 2024

floscher Jun 29, 2024

b-xb Jul 1, 2024 •

edited

Loading

	this.branch = defaultBranch \|\| '';
	this.branch = defaultBranch \|\| 'master';

	baseHeaders['Authorization'] = `token ${token}`;
	baseHeaders['Authorization'] = `Bearer ${token}`;

Fix: Set correct branch when it's not specified in the config #5844

Fix: Set correct branch when it's not specified in the config #5844

Conversation

bytrangle commented Sep 27, 2021

bytrangle commented Sep 29, 2021

erezrokah left a comment

Choose a reason for hiding this comment

erezrokah Sep 30, 2021

Choose a reason for hiding this comment

bytrangle Oct 1, 2021 • edited Loading

Choose a reason for hiding this comment

erezrokah Oct 1, 2021

Choose a reason for hiding this comment

bytrangle Oct 1, 2021

Choose a reason for hiding this comment

erezrokah Oct 1, 2021 • edited Loading

Choose a reason for hiding this comment

erezrokah Sep 30, 2021

Choose a reason for hiding this comment

bytrangle commented Oct 12, 2021

bytrangle commented Oct 27, 2021

erezrokah commented Oct 27, 2021

erezrokah commented Oct 27, 2021

stale bot commented Apr 26, 2023

martinjagodic commented Oct 16, 2023

paulRbr commented Feb 28, 2024

martinjagodic commented Feb 28, 2024

bytrangle commented Mar 14, 2024

martinjagodic commented Mar 14, 2024

bytrangle commented Apr 1, 2024

demshy commented Apr 2, 2024

floscher Jun 28, 2024 • edited Loading

Choose a reason for hiding this comment

bytrangle Jun 29, 2024

Choose a reason for hiding this comment

floscher Jun 29, 2024

Choose a reason for hiding this comment

b-xb Jul 1, 2024 • edited Loading

Choose a reason for hiding this comment

bytrangle Oct 1, 2021 •

edited

Loading

erezrokah Oct 1, 2021 •

edited

Loading

floscher Jun 28, 2024 •

edited

Loading

b-xb Jul 1, 2024 •

edited

Loading