Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add refdata repo selection in template #1658

Merged
merged 10 commits into from
Jul 1, 2021

Conversation

epassaro
Copy link
Member

@epassaro epassaro commented Jun 17, 2021

Description

  • Change default pipeline template to allow selection of tardis-refdata repo source between azure or github.
  • Change scheduled trigger in compare-refdata pipeline. Instead run when a new tag is pushed (new release).
  • Simplify compare-refdata and update-refdata pipelines by using github source.
  • Rename skipInstall step to tardisEnv.
  • Add more subfolders to OpenSupernova result page (for manual builds and releases).
  • Update documentation.

Motivation and context

Trying to test new atomic data (#1668) realized the reference data mirror does not suit the use case of comparing PRs made to tardis-refdata (for example upstream/pr/46). That happens because the mirror does not contain information about the PRs.

Then I added a new parameter to the default template to select the source of reference data. The compare-refdata and update-refdata pipelines use github source by default (thanks to new GH sponsorship we have free LFS quota). This solves the above described scenario, and simplifies the pipeline in many ways.

How has this been tested?

  • Testing pipeline.
  • Other. compare-refdata pipeline.

Examples

Type of change

  • Bug fix.
  • New feature.
  • Breaking change.
  • None of the above.

Checklist

  • My change requires a change to the documentation.
    • I have updated the documentation accordingly.
    • (optional) I have built the documentation on my fork following the instructions.
  • I have assigned and requested two reviewers for this pull request.

@codecov
Copy link

codecov bot commented Jun 17, 2021

Codecov Report

Merging #1658 (a0c429f) into master (eaab7a1) will not change coverage.
The diff coverage is n/a.

❗ Current head a0c429f differs from pull request most recent head e1f79be. Consider uploading reports for the commit e1f79be to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1658   +/-   ##
=======================================
  Coverage   61.89%   61.89%           
=======================================
  Files          62       62           
  Lines        5732     5732           
=======================================
  Hits         3548     3548           
  Misses       2184     2184           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eaab7a1...e1f79be. Read the comment docs.

@epassaro
Copy link
Member Author

/AzurePipelines run compare-refdata

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link

Build succeeded 96b9375

Click here to see results.

@epassaro epassaro force-pushed the carsus-x-test branch 3 times, most recently from edf8058 to 713866a Compare June 17, 2021 21:34
@epassaro epassaro marked this pull request as draft June 17, 2021 21:55
@epassaro
Copy link
Member Author

/AzurePipelines run compare-refdata

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link

Build succeeded 52b3d7d

Click here to see results.

@epassaro
Copy link
Member Author

/azp run compare-refdata

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@azure-pipelines
Copy link

Build succeeded 42c698f

Click here to see results.

@epassaro epassaro changed the title test carsus atomic files through compare-refdata pipeline add refdata repo selection in template Jun 27, 2021
@epassaro epassaro marked this pull request as ready for review June 27, 2021 20:08
trigger:
tags:
include:
- '*'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compare reference data every time a new TARDIS release is made.

branches:
include:
- master

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not run monthly tests anymore.

@@ -38,26 +34,18 @@ jobs:
steps:
- template: templates/default.yml
parameters:
fetchRefdata: false # See the comment below.
fetchRefdata: true
refdataRepo: 'github'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compare-refdata pipeline is simplified by using GitHub source instead of Azure (thanks to our new sponsor GitHub we have free LFS).

- bash: |
cd $(refdata.dir)
git remote add upstream https://tardis-sn@dev.azure.com/tardis-sn/TARDIS/_git/tardis-refdata
git remote add upstream $(git remote get-url origin)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, upstream is the same as origin.

- bash: |
ssh azuredevops@opensupernova.org "mkdir -p /home/azuredevops/public_html/files/refdata-results/$(pr.number)"
ssh azuredevops@opensupernova.org "mkdir -p /home/azuredevops/public_html/files/refdata-results/$(subfolder)"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put everything under a subfolder: PR (by pr number), manual triggered builds and release builds.

@tardis-bot
Copy link
Contributor

Before a pull request is accepted, it must meet the following criteria:

  • Is the necessary information provided?
  • Is this a duplicate PR?
    • If a new PR is clearly a duplicate, ask how this PR is different from the original PR?
    • If this PR is about to be merged, close the original PR with a link to this new PR that solved the issue.
  • Does it pass existing tests and are new tests provided if required?
    • The test coverage should not decrease, and for new features should be close to 100%.
  • Is the code tidy?
    • No unnecessary print lines or code comments.

@wkerzendorf wkerzendorf merged commit 9ee2c9c into tardis-sn:master Jul 1, 2021
atharva-2001 pushed a commit to atharva-2001/tardis that referenced this pull request Oct 1, 2021
* changes to pipeline template

* changes to refdata pipeline

* remove scheduled trigger, use tags instead

* revert some changes to template

* change param name

* update update-refdata pipeline

* specify refdata repo

* simplify compare-refdata pipeline, use gh repo by default

* remove debug prints

* change docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants