Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fork repo for daily updates in Java #25677

Merged
merged 14 commits into from
Dec 2, 2021
Merged

Use fork repo for daily updates in Java #25677

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
ca95682
use fork
sima-zhu Nov 24, 2021
0c81929
checkin from fork repo to original repo
sima-zhu Nov 24, 2021
e045403
fix doxindex
sima-zhu Nov 24, 2021
b951311
remove exist
sima-zhu Nov 24, 2021
88e4aa6
another way
sima-zhu Nov 24, 2021
1f0ad01
morefix
sima-zhu Nov 24, 2021
8ac0514
create new folder
sima-zhu Nov 24, 2021
eee1649
Update docindex.yml
sima-zhu Nov 29, 2021
d770563
Added variables
sima-zhu Nov 29, 2021
d64e8a4
Added variables
sima-zhu Nov 30, 2021
087e8c4
Remove doc repo owner
sima-zhu Nov 30, 2021
20eb245
create remote to azure-sdk instead of clone
sima-zhu Dec 1, 2021
7871736
Fix errors
sima-zhu Dec 1, 2021
8468d1d
tweak more comands
sima-zhu Dec 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions eng/pipelines/docindex.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,30 @@ jobs:
- template: /eng/common/pipelines/templates/steps/set-daily-docs-branch-name.yml
parameters:
DailyBranchVariableName: DailyDocsBranchName
# Sync docs fork repo onboarding files/folders
weshaggard marked this conversation as resolved.
Show resolved Hide resolved
- pwsh: |
$dailyDocsRepoLocation = "$(DocRepoLocation)/dailyDocs"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to set this dynamically or can we set it statically where we define DocRepoLocation?

Copy link
Contributor Author

@sima-zhu sima-zhu Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In DocRepoLocation ,

Pull remote Azure/main -> local Azure/main 
Push local Azure/main -> remote Azure/main

In dailyDocsRepoLocation ,

Pull remote azure-sdk/daily-> local azure-sdk/daily
Push local azure-sdk/daily-> remote Azure/daily

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two steps both happen in schedule run

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can eliminate this pwsh task by setting the variable above:

    variables:
      DocRepoLocation: $(Pipeline.Workspace)/docs
      DailyDocRepoLocation: $(Pipeline.Workspace)/dailyDocsFork  # Add this line
      DocRepoOwner: Azure
      DocRepoName: azure-docs-sdk-java

That would eliminate the need to create this variable at runtime and the sparse checkout process handles creating the directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that is wat I was wondering about. However see my other comment as well as we should consider just adding a remote to the fork instead of a separate clone.

echo "##vso[task.setvariable variable=DailyDocsRepoLocation]$dailyDocsRepoLocation"
if (!(Test-Path $dailyDocsRepoLocation)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to create this? Won't the clone create it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The working directory contains two clones, 1 main branch from Azure repo, 2 daily branch from azure-sdk repo.
1 used in schedule run updating the Azure main repo. (For release ones)
2 used in schedule run updating the Azure daily repo. (For daily docs)

Both branches get updated in schedule run.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can consider using the same clone but just adding a new remote that would save on some of the forking going on and the duplicate cloning.

New-Item -Path $dailyDocsRepoLocation -ItemType "directory"
}
displayName: Create daily docs repo
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
sima-zhu marked this conversation as resolved.
Show resolved Hide resolved
parameters:
SkipDefaultCheckout: true
Paths:
- package.json
- metadata/
Repositories:
- Name: "azure-sdk/$(DocRepoName)"
WorkingDirectory: $(DailyDocsRepoLocation)
- pwsh: |
$ErrorActionPreference = "Continue"
git checkout "origin/$(DailyDocsBranchName)" 2>&1 | Out-Null
$LASTEXITCODE = 0 # This ignores any error from git checkout
git status
displayName: Checkout daily branch if it exists
workingDirectory: $(DocRepoLocation)
workingDirectory: $(DailyDocsRepoLocation)

# Docs daily updates is supposed to download packages from public feed repository, so we have to specify additional repositories in a POM or the profile.
# Here is maven documentation: https://maven.apache.org/guides/mini/guide-multiple-repositories.html
Expand All @@ -76,7 +93,7 @@ jobs:
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackages.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
arguments: -DocRepoLocation "$(DailyDocsRepoLocation)"
displayName: Update Docs Onboarding for Daily branch
- template: /eng/common/pipelines/templates/steps/git-push-changes.yml
parameters:
Expand All @@ -85,7 +102,7 @@ jobs:
CommitMsg: "Update targeting packages based on release metadata. (Daily docs)"
TargetRepoName: $(DocRepoName)
TargetRepoOwner: $(DocRepoOwner)
WorkingDirectory: $(DocRepoLocation)
WorkingDirectory: $(DailyDocsRepoLocation)
ScriptDirectory: $(Build.SourcesDirectory)/eng/common/scripts

- task: PowerShell@2
Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/templates/stages/archetype-java-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ stages:
- ${{if ne(artifact.skipPublishDocMs, 'true')}}:
- $(Pipeline.Workspace)/${{parameters.ArtifactName}}/PackageInfo/${{artifact.name}}.json
WorkingDirectory: $(System.DefaultWorkingDirectory)
TargetDocRepoOwner: ${{parameters.TargetDocRepoOwner}}
TargetDocRepoOwner: "azure-sdk"
sima-zhu marked this conversation as resolved.
Show resolved Hide resolved
TargetDocRepoName: ${{parameters.TargetDocRepoName}}
Language: 'java'
DailyDocsBuild: true
Expand Down