Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ORCA-261] Adds HDF5Suite #46

Merged
merged 4 commits into from
Jul 27, 2023
Merged

[ORCA-261] Adds HDF5Suite #46

merged 4 commits into from
Jul 27, 2023

Conversation

BWMac
Copy link
Contributor

@BWMac BWMac commented Jul 26, 2023

This PR adds support for tests specific to HDF5 files by introducing the HDF5Suite.

The file extensions included are .hdf, .hdf5, .h5, and .he5.

This change has been tested with nf-dcqc both locally and on Tower. Example output using the qc-file command:

dcqc qc-file -t HDF5 -m '{"md5_checksum": "93d78663da24b6a708f6c81207e4410a"}' tests/data/test.hdf
{
  "type": "HDF5Suite",
  "target": {
    "id": null,
    "files": [
      {
        "url": "tests/data/test.hdf",
        "metadata": {
          "md5_checksum": "93d78663da24b6a708f6c81207e4410a"
        },
        "type": "HDF5",
        "name": "test.hdf",
        "local_path": "/var/folders/sr/3g4hnkfd4ld306tty7kqf1rr0000gr/T/dcqc-staged-b_zylzop/test.hdf"
      }
    ],
    "type": "SingleTarget"
  },
  "suite_status": {
    "required_tests": [
      "FileExtensionTest",
      "Md5ChecksumTest"
    ],
    "skipped_tests": [],
    "status": "GREEN"
  },
  "tests": [
    {
      "type": "FileExtensionTest",
      "tier": 1,
      "is_external_test": false,
      "status": "passed"
    },
    {
      "type": "Md5ChecksumTest",
      "tier": 1,
      "is_external_test": false,
      "status": "passed"
    }
  ]
}

It also adds an example test.hdf file for testing purposes.

@swarmia
Copy link

swarmia bot commented Jul 26, 2023

✅  Linked to Story ORCA-261 · DCQC: Add support for hdf5 files
➡️  Part of Epic ORCA-182 · DCQC tests
➡️  Part of IBCDPE-502 · DCQC

@codecov
Copy link

codecov bot commented Jul 26, 2023

Codecov Report

Merging #46 (3e46823) into main (175d3fa) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #46   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           23        23           
  Lines         1169      1172    +3     
  Branches       192       192           
=========================================
+ Hits          1169      1172    +3     
Files Changed Coverage Δ
src/dcqc/file.py 100.00% <100.00%> (ø)
src/dcqc/suites/suites.py 100.00% <100.00%> (ø)

@BWMac BWMac requested a review from thomasyu888 July 26, 2023 22:07
@BWMac BWMac marked this pull request as ready for review July 26, 2023 22:07
Copy link
Contributor

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪐 Nice! This is great!

With these, we can work together with Adam to add tests for some of the file formats we just added.

@BWMac BWMac merged commit 2effca6 into main Jul 27, 2023
12 checks passed
@BWMac BWMac deleted the bwmac/orca-261/hdf5_support branch July 27, 2023 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants