Add page buffer cache command line option to tools #4562

byrnHDF · 2024-06-12T14:26:34Z

No description provided.

…evelop-page-cache

tools/lib/h5diff.c

tools/src/h5dump/h5dump.c

tools/src/h5ls/h5ls.c

tools/src/h5repack/h5repack_main.c

tools/src/h5stat/h5stat.c

byrnHDF · 2024-06-13T18:30:11Z

Closes #4508

…evelop-page-cache

jhendersonHDF · 2024-06-13T20:02:17Z

tools/src/misc/h5mkgrp.c

@@ -252,18 +252,24 @@ parse_command_line(int argc, const char *const *argv, mkgrp_opt_t *options)

    /* Setup a custom fapl for file accesses */
    if (custom_vol || custom_vfd) {
-        if ((tmp_fapl_id = h5tools_get_fapl(options->fapl_id, custom_vol ? &vol_info : NULL,


I can't comment on the h5tools_fopen call below since it's outside the diff context, but another case where (params_g.fapl_id != H5P_DEFAULT) should probably be switched to custom_vol || custom_vfd

jhendersonHDF · 2024-06-13T20:09:31Z

tools/src/h5ls/h5ls.c

@@ -2816,6 +2820,9 @@ main(int argc, char *argv[])
                leave(EXIT_FAILURE);
            }
        }
+        else if (!strncmp(argv[argno], "--page-buffer-size=", (size_t)19)) {
+            page_cache = strtoul(argv[argno] + 8, &rest, 0);


Need to set this to 19 as well

jhendersonHDF · 2024-06-13T20:15:21Z

Just a couple things left, but looking good. After thinking on it for a bit, I think making set_fapl_vol and set_fapl_vfd separate from get_fapl is a better design anyway based on how the functions were being used in the tools.

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fix typos in context/property documentation (#4550) * Fix CI markdown link check http 500 errors (#4556) Sites like GitLab can have internal problems that return http 500 errors while they fix their problems. Some sites also return http 200 OK, which is fine. This PR adds a config file to the markdown link check so those are considered "passing" and don't break the CI. * Simplify property copying between lists internally (#4551) * Add Python examples (#4546) These examples are referred to from the replacement page of https://portal.hdfgroup.org/display/HDF5/Other+Examples. * Correct property cb signatures in docs (#4554) * Correct property cb signatures in docs * Correct delete callback type name in docs * add missing word to H5P__free_prop doc * Move C++ and Fortran and examples to HDF5Examples folder (#4552) * Document 'return-and-read' field in API context (#4560) * Add compression includes to tests needing zlib support (#4561) * Allow usage of page buffering for serial file access from parallel HDF5 builds (#4568) * Remove old version of libaec (#4567) * Add property names to context field docs (#4563) * Document property shared name behavior (#4565) * Clarify H5CX macro documentation (#4569) * Document H5Punregister modifying default properties (#4570) * Update NVHPC to 24.5 (#4171) We don't test parallel in other GitHub actions, so this also converts the NVHPC check to configure and build only while we discuss how we'll test parallel HDF5 in GitHub. There is a blocking GitHub issue to address the test failures for HDF5 1.14.5 (#4571). * Clean up comments in H5FDros3.c (#4572) * Rename INSTALL_Auto.txt to INSTALL_Autotools.txt (#4575) * Clean up ros3 VFD stats code (#4579) * Removes printf debugging * Simplifies and centralizes stats code * Use #ifdef ROS3_STATS instead of #if * Other misc tidying * Turn off ros3 VFD stat collection by default (#4581) Not a new change - an artifact from a previous check-in. * Pause recording errors instead of clearing the error stack (#4475) An internal capability that's similar to the H5E_BEGIN_TRY / H5E_END_TRY macros in H5Epublic.h, but more efficient since we can avoid pushing errors on the stack entirely (and those macros use public API routines). This capability (and other techniques) can be used to remove use of H5E_clear_stack() and H5E_BEGIN_TRY / H5E_END_TRY within library routines. We want to remove H5E_clear_stack() because it can trigger calls to the H5I interface from within the H5E code, which creates a great deal of complexity for threadsafe code. And we want to remove H5E_BEGIN_TRY / H5E_END_TRY's because they make public API calls from within the library code. Also some other minor tidying in routines related to removing the use of H5E_clear_stack() and H5E_BEGIN_TRY / H5E_END_TRY from H5Fint.c * Add page buffer cache command line option to tools (#4562) Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Clarify documentation for H5CX_get_data_transform (#4580) * Correct comment for H5CX_get_data_transform * Document why data transform ctx field doesnt use macro * Remove public API call from ros3 VFD (#4583) * Remove printf debugging from H5FDs3comms.c (#4584) * Cleanup of ros3 test (#4587) * Removed JS* macro scheme (replaced w/ h5test.h macros) * Moved curl setup/teardown to main() * A lot of cleanup and simplification * Removed unused code from H5FDs3comms.c (#4588) * H5FD_s3comms_nlowercase() * H5FD_s3comms_trim() * H5FD_s3comms_uriencode() * Remove magic fields from s3comms structs (#4589) * Remove dead H5FD_s3comms_percent_encode_char() (#4591) * Rework the TestExpress usage and refactor dead code (#4590) * Skip examples if running sanitizers (#4592) * Clean up s3comms test code (#4594) * Remove JS* macros * Remove dead code * Bring in line with other test code * Add publish to bucket workflow (#4566) * Update abi report CI workflow for last release (#4596) * Update abi report workflow to handle 1.14.4.3 release * Update name of java report * Document that ctx VOL property isn't drawn from the FAPL (#4597) * Update macos workflow to 14 (keep 13 as alternate) (#4603) * Removed unnecessary call to H5E_clear_stack (#4607) H5FO_opened and H5SL_search don't push errors on the stack * Bring subfiling VFD code closer to typical library code (#4595) Remove API calls, use FUNC_ENTER/LEAVE macros, use the library's error macros, rename functions to have more standardized names, etc. * Correct documentation for return-and-read fields (#4598) * These two generators create strings without NUL for testing (#4608) * Fix Fortran pkconfig to indicate full path of modules (#4593) * Updated release schedule (#4615) 1.16 and 2.0 information * Document VOL object wrapping context (#4611) * Earray.c and farray.c in hdf5_1_14 still need time_t curr_time for HDsrandom. * Remove line to use future 116_API from CMakeListat.txt files in HDF5 examples directories

byrnHDF and others added 6 commits June 11, 2024 13:33

Add page buffer cache size option to tools

9b4d818

Committing clang-format changes

9c988ee

Adjust settings

2631633

Add test for page cache option

1d96b9b

Merge branch 'develop' of https://github.com/HDFGroup/hdf5.git into d…

dcb2cc5

…evelop-page-cache

Add release note

4d0df6c

byrnHDF self-assigned this Jun 12, 2024

byrnHDF requested review from lrknox, derobins, fortnern, jhendersonHDF, qkoziol, vchoi-hdfgroup, bmribler, glennsong09, mattjala and brtnfld as code owners June 12, 2024 14:26

Correct h5ls option parse

ccab831