Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow page buffer cache setting for non-PAGE HDF5 file open #3382

Closed
ajelenak opened this issue Aug 14, 2023 · 0 comments
Closed

Allow page buffer cache setting for non-PAGE HDF5 file open #3382

ajelenak opened this issue Aug 14, 2023 · 0 comments
Assignees
@derobins @fortnern @jhendersonHDF
Labels
Component - C Library Core C library issues (usually in the src directory) Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Improvement Improvements that don't add a new feature or functionality
Milestone
1.14.4

Comments

@ajelenak
Copy link
Contributor

I have not found a way to find out if an HDF5 file has been created with the PAGE file space strategy without opening it first, checking the file space strategy, and then closing and re-opening if I want to enable the page buffer cache. It would be easier if the library does not throw an error when opening a file with the page buffer cache set even if the file is not created with the PAGE space strategy.

Below is the error reported by h5py when opening a non-PAGE HDF5 file with the page buffer cache set:

OSError: Unable to synchronously open file (Enabling Page Buffering requires PAGE file space strategy)

Ideally, the library should open the file without error and the user can later check whether the page buffer cache is available.
@jhendersonHDF jhendersonHDF added Priority - 2. Medium ⏹ It would be nice to have this in the next release Component - C Library Core C library issues (usually in the src directory) Type - Improvement Improvements that don't add a new feature or functionality labels Aug 21, 2023
@derobins derobins self-assigned this Aug 24, 2023
@derobins derobins added this to the 1.14.3 milestone Aug 31, 2023
@derobins derobins modified the milestones: 1.14.3, 1.14.4 Oct 24, 2023
derobins added a commit to derobins/hdf5 that referenced this issue Mar 29, 2024
@derobins
Relaxed behavior of H5Pset_page_buffer_size() when opening files
b8f50f1 
This API call sets the size of a file's page buffer cache. This call
was extremely strict about matching its parameters to the file strategy
and page size used to create the file, requiring a separate open of the
file to obtain these parameters.

These requirements have been relaxed when using the fapl to open
a previously-created file:

* When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE
  strategy, the setting is ignored and the file will be opened, but
  without a page buffer cache. This was previously an error.

* When opening a file that has a page size larger than the desired
  page buffer cache size, the page buffer cache size will be increased
  to the file's page size. This was previously an error.

The behavior when creating a file using H5Pset_page_buffer_size() is
unchanged.

Fixes GitHub issue HDFGroup#3382
@derobins derobins mentioned this issue Mar 29, 2024
Merged
derobins added a commit that referenced this issue Mar 29, 2024
@derobins
Relaxed behavior of H5Pset_page_buffer_size() when opening files (#4280)
855b95b 
This API call sets the size of a file's page buffer cache. This call
was extremely strict about matching its parameters to the file strategy
and page size used to create the file, requiring a separate open of the
file to obtain these parameters.

These requirements have been relaxed when using the fapl to open
a previously-created file:

* When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE
  strategy, the setting is ignored and the file will be opened, but
  without a page buffer cache. This was previously an error.

* When opening a file that has a page size larger than the desired
  page buffer cache size, the page buffer cache size will be increased
  to the file's page size. This was previously an error.

The behavior when creating a file using H5Pset_page_buffer_size() is
unchanged.

Fixes GitHub issue #3382
lrknox pushed a commit to lrknox/hdf5 that referenced this issue Apr 1, 2024
@derobins @lrknox
Relaxed behavior of H5Pset_page_buffer_size() when opening files (HDF…
9ae4b3f 
…Group#4280)

This API call sets the size of a file's page buffer cache. This call
was extremely strict about matching its parameters to the file strategy
and page size used to create the file, requiring a separate open of the
file to obtain these parameters.

These requirements have been relaxed when using the fapl to open
a previously-created file:

* When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE
  strategy, the setting is ignored and the file will be opened, but
  without a page buffer cache. This was previously an error.

* When opening a file that has a page size larger than the desired
  page buffer cache size, the page buffer cache size will be increased
  to the file's page size. This was previously an error.

The behavior when creating a file using H5Pset_page_buffer_size() is
unchanged.

Fixes GitHub issue HDFGroup#3382
derobins added a commit that referenced this issue Apr 1, 2024
@lrknox @derobins
@hyoklee @brtnfld @byrnHDF @jhendersonHDF @bmribler @fortnern
Sync develop changes March 28 - April 1 to 1 14 (#4297)
9f7b987 
* Relaxed behavior of H5Pset_page_buffer_size() when opening files (#4280)

This API call sets the size of a file's page buffer cache. This call
was extremely strict about matching its parameters to the file strategy
and page size used to create the file, requiring a separate open of the
file to obtain these parameters.

These requirements have been relaxed when using the fapl to open
a previously-created file:

* When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE
  strategy, the setting is ignored and the file will be opened, but
  without a page buffer cache. This was previously an error.

* When opening a file that has a page size larger than the desired
  page buffer cache size, the page buffer cache size will be increased
  to the file's page size. This was previously an error.

The behavior when creating a file using H5Pset_page_buffer_size() is
unchanged.

Fixes GitHub issue #3382

* chore: improve error message (#4287)

close #192

* changed to if string contains instead (#4286)

* Fix off_t straggers (#4291)

Convert off_t to HDoff_t (mainly for Windows):

* h5jam
* h5unjam
* chunk_info test

* Add missing foreach for VFD and Pasthrough runs (#4292)

* Fixes for file format security issues (#4283)

* Add configure options for disabling extension features (#4277)

Add configure option to enable or disable extension features in general

Add configure option to enable or disable _Float16 support

Add new config options to various settings files

* Add documentation (H5FD) (#4269)

* Use AOCC 4.2 and OpenMPI 4.1.6 (#4290)

* Fix problems with background buffers and array datatypes (#4218)

* Fix bug in array conversion with strided background buffer. Convert some
memmove calls to non-overlapping buffers to memcpy.

* Revert inappropriate use of mempy to memmove in H5T__conv_array

* Add testing

* Add RELEASE.txt note and overwrite test case.

* Synch workflows and require apt-get update (#4294)

* Fix Figure 9. table format in HDF5 Groups User Guide (#4295)

---------

Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: jhendersonHDF <jhenderson@hdfgroup.org>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
Co-authored-by: Neil Fortner <fortnern@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@derobins derobins

@fortnern fortnern

@jhendersonHDF jhendersonHDF

Labels
Component - C Library Core C library issues (usually in the src directory) Priority - 2. Medium ⏹ It would be nice to have this in the next release Type - Improvement Improvements that don't add a new feature or functionality
Projects
None yet
Milestone
1.14.4
Development

No branches or pull requests
4 participants
@derobins @ajelenak @fortnern @jhendersonHDF