Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: general purpose data distribution testing to support non-uniform data #129

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Commits on Aug 20, 2024

  1. Start data distribution out as varient of normal distribution

    Copy h5bench_write_normal_dist.c as baseline.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    8ead940 View commit details
    Browse the repository at this point in the history
  2. Add write_var_data_dist benchmark support to h5bench.py

    Created a new benchmark test name and added selection
    support into h5bench.py test wrapper.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    bddc4c9 View commit details
    Browse the repository at this point in the history
  3. Add DATA_DIST_PATH support to params struct and config file

    Added parsing of DATA_DIST_PATH into params struct to record
    the data distrbution based on an input file.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    776cc51 View commit details
    Browse the repository at this point in the history
  4. Add data dist file parsing to populate memory allocation pattern

    Takes the provided DATA_DIST_PATH file and reads the data sizes
    per rank from the file feeding into holder array.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    2088710 View commit details
    Browse the repository at this point in the history
  5. Add build and install targets for new data distribution test

    Create make targets and add file to install list.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    d899116 View commit details
    Browse the repository at this point in the history
  6. Update per process accounting

    Remove rank 0 output summary limit so all processes report
    performance rather than just accepting rank 0 results.  This
    is especially important for non-uniform data distribution but
    should be considered for uniform since no summary stats are computed.
    Recommend --tag-output to track per-rank stdout.
    
    Remove naive total size compute and limit it to a per rank size value.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    e051203 View commit details
    Browse the repository at this point in the history
  7. Add scaling factor for data distribution particle counts

    Add scaling paramter to tests to scale particle count in order
    to create memory footprints that more accurately reflect a data
    distribution.
    
    Data distribution inputs are in particle counts.
    Particles are 32-byte structures so a data distribution measured in
    bytes needs to be scaled down so the particals instiated match
    the actual data footprint, in multiples of 32-byte particles.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    9cf8232 View commit details
    Browse the repository at this point in the history
  8. Fix the data dist benchmark binary name to use "var" naming convention

    Change the deployed binary to "write_var_data_dist" so it matches
    the configured test reference in h5bench.
    The binary name needs to match the test name so that it can be called
    by the hbench wrapper.
    This follows convention of "write_var_normal_dist".
    Update code to log the correct benchmark name.
    jprorama committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    de1d787 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Update the github action/artifacts-upload version

    Change action/artifacts-upload from v2 to v4 to remove
    dependence on deprecated v2.
    The v4 syntax for the artifact-upload remains the same
    so a simple update of the version number should be sufficient.
    
    See blog post for details:
    
    https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/
    jprorama committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    241fee7 View commit details
    Browse the repository at this point in the history
  2. Update clang-format container version to 0.18.2

    Update to latest version of container to see if it avoids the
    missing distutils dependency reported with the @0.11 version.
    jprorama committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    7610377 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'fix-github-upload-artifact-version' of github.com:jpror…

    …ama/h5bench into feat-data-dist
    
    Grab the github actions fixes to ensure current feature branch validation tests
    run successfully.
    jprorama committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    dff8efc View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    fd64d7a View commit details
    Browse the repository at this point in the history