Feat: editorial workflow bitbucket gitlab #3014
Merged
+67,279
−12,343
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erezrokah
changed the title
erezrokah
changed the title
erezrokah
force-pushed
the
feat/editorial_workflow_bitbucket_gitlab
branch
3 times, most recently
from
ea39a6c
to
7fb7361
Compare
erezrokah
changed the title
erezrokah
force-pushed
the
feat/editorial_workflow_bitbucket_gitlab
branch
from
bf7cf10
to
5413d50
Compare
erezrokah
commented
Jan 9, 2020
package.json
Outdated
| @@ -24,8 +24,8 @@ | |||
| "test:e2e:ci": "run-s build:demo test:e2e:run-ci", | |||
| "test:e2e:dev": "run-p clean && start-test develop 8080 test:e2e:exec-dev", | |||
| "test:e2e:serve": "http-server dev-test", | |||
| "test:e2e:exec": "cypress run", | |||
| "test:e2e:exec-ci": "cypress run --record --parallel --ci-build-id $GITHUB_SHA --group 'GitHub CI' displayName: 'Run Cypress tests'", | |||
| "test:e2e:exec": "cypress run --browser chrome --headless", | |||
There was a problem hiding this comment.
Switch to headless chrome
erezrokah
commented
Jan 9, 2020
| }, | ||
| description: DEFAULT_PR_BODY, | ||
| // eslint-disable-next-line @typescript-eslint/camelcase | ||
| close_source_branch: true, |
There was a problem hiding this comment.
Auto delete source branch on merge
erezrokah
commented
Jan 9, 2020
erezrokah
commented
Jan 9, 2020
| @@ -16,22 +15,25 @@ export default class BitbucketAuthenticationPage extends React.Component { | |||
| base_url: PropTypes.string, | |||
| siteId: PropTypes.string, | |||
| authEndpoint: PropTypes.string, | |||
| config: ImmutablePropTypes.map, | |||
| config: PropTypes.object.isRequired, | |||
There was a problem hiding this comment.
switched to plain JavaScript object for the config
erezrokah
commented
Jan 9, 2020
| login: user.username, | ||
| token: state.token, | ||
| // eslint-disable-next-line @typescript-eslint/camelcase | ||
| avatar_url: user.links.avatar.href, |
There was a problem hiding this comment.
Fixes missing avatar image on BitBucket
erezrokah
commented
Jan 9, 2020
| @@ -113,7 +114,7 @@ export default class GitGatewayAuthenticationPage extends React.Component { | |||
| onLogin: PropTypes.func.isRequired, | |||
| inProgress: PropTypes.bool.isRequired, | |||
| error: PropTypes.node, | |||
| config: ImmutablePropTypes.map, | |||
| config: PropTypes.object.isRequired, | |||
There was a problem hiding this comment.
Switched to plain JavaScript object
erezrokah
commented
Jan 9, 2020
| ...options, | ||
| }; | ||
| this.config = config; | ||
| this.branch = config.getIn(['backend', 'branch'], 'master').trim(); | ||
| this.squash_merges = config.getIn(['backend', 'squash_merges']); | ||
| this.branch = config.backend.branch?.trim() || 'master'; |
There was a problem hiding this comment.
Switched to plain JavaScript object for config
erezrokah
commented
Jan 9, 2020
| } | ||
| entriesByFiles(collection) { | ||
| return this.backend.entriesByFiles(collection); | ||
| entriesByFolder(folder: string, extension: string, depth: number) { |
There was a problem hiding this comment.
Each backend had the same code to extract the folder and depth from the collection so I moved that to the core
erezrokah
commented
Jan 9, 2020
| } | ||
| fetchFiles(files) { | ||
| return this.backend.fetchFiles(files); | ||
| entriesByFiles(files: ImplementationFile[]) { |
There was a problem hiding this comment.
Each backend had the same code to extract the files from the collection so I moved that to the core
erezrokah
commented
Jan 9, 2020
| } | ||
| getEntry(collection, slug, path) { | ||
| return this.backend.getEntry(collection, slug, path); | ||
| getEntry(path: string) { |
There was a problem hiding this comment.
collection, slug were not used
erezrokah
commented
Jan 9, 2020
| } | ||
|
|
||
| const mediaFiles = await Promise.all( | ||
| files.map(async file => { | ||
| if (client.matchPath(file.path)) { | ||
| const { sha: id, path } = file; | ||
| const { id, path } = file; |
There was a problem hiding this comment.
All backends return the same file structure using id and not sha
erezrokah
commented
Jan 9, 2020
| } else { | ||
| return this.retrieveContent({ path, branch, repoURL, parseText }); | ||
| if (!sha) { | ||
| sha = await this.getFileSha(path, { repoURL, branch }); |
There was a problem hiding this comment.
Always use the sha to get the file to support caching.
erezrokah
commented
Jan 9, 2020
| @@ -479,38 +463,10 @@ export default class API { | |||
| } | |||
| } | |||
|
|
|||
| async getMediaAsBlob(sha: string | null, path: string) { | |||
There was a problem hiding this comment.
Moved to a shared file in lib-util
erezrokah
commented
Jan 9, 2020
| .join('/')}` | ||
| : this.repoURL; | ||
|
|
||
| const [fileData, isModification] = await Promise.all([ |
There was a problem hiding this comment.
I find this easier to read instead of the nesting when using resolvePromiseProperties
erezrokah
commented
Jan 9, 2020
| } | ||
|
|
||
| isUnpublishedEntryModification(path: string, branch: string) { | ||
| isUnpublishedEntryModification(path: string) { |
There was a problem hiding this comment.
Was always called with the default branch
erezrokah
commented
Jan 9, 2020
| @@ -28,7 +27,7 @@ export default class GitHubAuthenticationPage extends React.Component { | |||
| base_url: PropTypes.string, | |||
| siteId: PropTypes.string, | |||
| authEndpoint: PropTypes.string, | |||
| config: ImmutablePropTypes.map, | |||
| config: PropTypes.object.isRequired, | |||
There was a problem hiding this comment.
As before, switch to plain object for config
erezrokah
commented
Jan 9, 2020
| return { ...pullRequest, head: { sha: pullRequest.headRefOid } }; | ||
| } | ||
|
|
||
| async getPullRequestCommits(number) { |
There was a problem hiding this comment.
Wasn't required since we switched to compare API
erezrokah
commented
Jan 9, 2020
| const mergeRequest = await this.getBranchMergeRequest(branch); | ||
| const diff = await this.getDifferences(mergeRequest.sha); | ||
| const path = diff.find(d => d.old_path.includes(slug))?.old_path as string; | ||
| // TODO: get real file id |
There was a problem hiding this comment.
The media file id is used with Large Media support in git-gateway.
We should be able to get it using aHEAD request once https://gitlab.com/gitlab-org/gitlab/issues/194897 is resolved
erezrokah
commented
Jan 9, 2020
| } | ||
|
|
||
| async rebaseMergeRequest(mergeRequest: GitLabMergeRequest) { | ||
| let rebase: GitLabMergeRebase = await this.requestJSON({ |
There was a problem hiding this comment.
Very useful asynchronous API
erezrokah
commented
Jan 9, 2020
| @@ -200,7 +202,7 @@ export function persistMedia(file: File, opts: MediaOptions = {}) { | |||
| const fileName = sanitizeSlug(file.name.toLowerCase(), state.config.get('slug')); | |||
| const existingFile = files.find(existingFile => existingFile.name.toLowerCase() === fileName); | |||
|
|
|||
| const editingDraft = selectEditingWorkflowDraft(state); | |||
| const editingDraft = selectEditingDraft(state.entryDraft); | |||
There was a problem hiding this comment.
Media files will be persisted in a single commit with the entry regardless if you're using editorial workflow.
erezrokah
commented
Jan 9, 2020
| @@ -349,7 +275,7 @@ export class Backend { | |||
| async entryExist(collection: Collection, path: string, slug: string) { | |||
| const unpublishedEntry = | |||
| this.implementation.unpublishedEntry && | |||
| (await this.implementation.unpublishedEntry(collection, slug).catch(error => { | |||
| (await this.implementation.unpublishedEntry(collection.get('name'), slug).catch(error => { | |||
There was a problem hiding this comment.
The backends only need the collection name. I've aligned all the backends to just receive the relevant arguments instead of the collection immutable object (we have some places we were passing a collection object and other places we were passing a collection string which was confusing).
gitlab git-gateway doesn't support editorial workflow APIs yet. This change makes sure not to call them in simple workflow
erquhart
force-pushed
the
feat/editorial_workflow_bitbucket_gitlab
branch
from
49923e6
to
d83f006
Compare
|
Rebased, fixed merge conflicts. |
This was referenced Jan 16, 2020
nathankitchen
pushed a commit
to nathankitchen/netlify-cms-backend-azure
that referenced
this pull request
Feb 24, 2020
* refactor: typescript the backends * feat: support multiple files upload for GitLab and BitBucket * fix: load entry media files from media folder or UI state * chore: cleanup log message * chore: code cleanup * refactor: typescript the test backend * refactor: cleanup getEntry unsued variables * refactor: moved shared backend code to lib util * chore: rename files to preserve history * fix: bind readFile method to API classes * test(e2e): switch to chrome in cypress tests * refactor: extract common api methods * refactor: remove most of immutable js usage from backends * feat(backend-gitlab): initial editorial workflow support * feat(backend-gitlab): implement missing workflow methods * chore: fix lint error * feat(backend-gitlab): support files deletion * test(e2e): add gitlab cypress tests * feat(backend-bitbucket): implement missing editorial workflow methods * test(e2e): add BitBucket backend e2e tests * build: update node version to 12 on netlify builds * fix(backend-bitbucket): extract BitBucket avatar url * test: fix git-gateway AuthenticationPage test * test(e2e): fix some backend tests * test(e2e): fix tests * test(e2e): add git-gateway editorial workflow test * chore: code cleanup * test(e2e): revert back to electron * test(e2e): add non editorial workflow tests * fix(git-gateway-gitlab): don't call unpublishedEntry in simple workflow gitlab git-gateway doesn't support editorial workflow APIs yet. This change makes sure not to call them in simple workflow * refactor(backend-bitbucket): switch to diffstat API instead of raw diff * chore: fix test * test(e2e): add more git-gateway tests * fix: post rebase typescript fixes * test(e2e): fix tests * fix: fix parsing of content key and add tests * refactor: rename test file * test(unit): add getStatues unit tests * chore: update cypress * docs: update beta docs
vladdu
pushed a commit
to vladdu/netlify-cms
that referenced
this pull request
Jan 26, 2021
* refactor: typescript the backends * feat: support multiple files upload for GitLab and BitBucket * fix: load entry media files from media folder or UI state * chore: cleanup log message * chore: code cleanup * refactor: typescript the test backend * refactor: cleanup getEntry unsued variables * refactor: moved shared backend code to lib util * chore: rename files to preserve history * fix: bind readFile method to API classes * test(e2e): switch to chrome in cypress tests * refactor: extract common api methods * refactor: remove most of immutable js usage from backends * feat(backend-gitlab): initial editorial workflow support * feat(backend-gitlab): implement missing workflow methods * chore: fix lint error * feat(backend-gitlab): support files deletion * test(e2e): add gitlab cypress tests * feat(backend-bitbucket): implement missing editorial workflow methods * test(e2e): add BitBucket backend e2e tests * build: update node version to 12 on netlify builds * fix(backend-bitbucket): extract BitBucket avatar url * test: fix git-gateway AuthenticationPage test * test(e2e): fix some backend tests * test(e2e): fix tests * test(e2e): add git-gateway editorial workflow test * chore: code cleanup * test(e2e): revert back to electron * test(e2e): add non editorial workflow tests * fix(git-gateway-gitlab): don't call unpublishedEntry in simple workflow gitlab git-gateway doesn't support editorial workflow APIs yet. This change makes sure not to call them in simple workflow * refactor(backend-bitbucket): switch to diffstat API instead of raw diff * chore: fix test * test(e2e): add more git-gateway tests * fix: post rebase typescript fixes * test(e2e): fix tests * fix: fix parsing of content key and add tests * refactor: rename test file * test(unit): add getStatues unit tests * chore: update cypress * docs: update beta docs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replaces #2409
Fixes #568
Initial GitLab editorial workflow support (BitBucket coming soon).Still work in progress and needs code cleanup, testing, performance improvements, bug fixes and a rebase with master.Uses merge requests labels for entry status.Wanted to get this out there for people to experiment and give feedback (also on the code) while I work out the details.I'll add comments to the PR next week.BitBucket and GitLab editorial workflow support.
I TypeScripted all the backends, then moved some of the shared code to the
core/lib-util. Probably we can do more after we align GitHub metadata handling with the other backends.Metadata handling
Limitations:
TODO