Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi-thread fix and request merging #205

Merged
merged 45 commits into from
Sep 5, 2024
Merged

Multi-thread fix and request merging #205

merged 45 commits into from
Sep 5, 2024

Conversation

houjun
Copy link
Member

@houjun houjun commented Jul 1, 2024 •
edited
Loading

Related Issues / Pull Requests

#190

Description

  • Add client-side region transfer requests merging optimization, currently limited to 1D, and when there are more than 50 requests (PDC_MERGE_TRANSFER_MIN_COUNT).
  • Add new PDCregion_transfer_start(_all)_mpi APIs that internally call MPI_Barrier for more coordinated metadata and data operations, avoiding forced collective operations with the original PDCregion_transfer_start(_all) APIs (Performace improvement for vpicio #211).
  • Fix compile issues when multi-threading is enabled.

What changes are proposed in this pull request?

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality not to work as expected; for instance, examples in this repository must be updated too)
  • This change requires a documentation update

Checklist:

  • My code modifies existing public API, or introduces new public API, and I updated or wrote docstrings
  • I have commented my code
  • My code requires documentation updates, and I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

houjun and others added 25 commits February 23, 2024 14:51
@houjun
Update getting_started.rst (#184)
f8d5f8d
@hyunjuooh @jeanbez
Removing gres option for ctest (#182)
38d5f77 
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>
@houjun
Fix an issue with region transfer request
ad20fab
@github-actions
Committing clang-format changes
94ec00b
@houjun
Merge small requests when they are contiguous and 1D, change srun com…
7c6e03c 
…mands in run scripts to detect Perlmutter compute nodes
@houjun
Merge only for REGION_LOCAL partition
a05d65c
@github-actions
Committing clang-format changes
8d8e298
@houjun
Fix a bug that causes some tests to fail
6a9fbd9
@houjun
Fix a couple of issues with start/wait all
9becfe4
@github-actions
Committing clang-format changes
88bfafe
@houjun
Add aggregation support for contiguous read operations
2aa41af
@github-actions
Committing clang-format changes
6687368
@houjun
Fix compile issue when multithread is enabled
ff58cef
@github-actions
Committing clang-format changes
fce422f
@houjun
minor change with test code
7cdb866
@github-actions
Committing clang-format changes
1b3b1da
@houjun
Remove metadata mutex for multi threading
67cc3b6
@github-actions
Committing clang-format changes
d40ee99
@houjun
Fix mutex
d1c5788
@github-actions
Committing clang-format changes
4559692
@houjun
Fix an issue when closing an obj
a094446
@jeanbez @houjun
@hyunjuooh @github-actions
Sync develop to stable (v.0.5) (#201)
bdeef59 
* Update getting_started.rst (#184)
* Removing gres option for ctest (#182)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* enable cache by default (#187)
* Removing PDC macro (#189)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core
* Remove PDC macro
* Committing clang-format changes

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* BDCATS fix (#193)
* Fix issues with bdcats_batch
* Committing clang-format changes

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* Update mpi_test.sh (#197)
* Update .gitlab-ci.yml (#195)
* Updates for latest integration with Jacamar and Gitlab tokens in CI
* VPICIO bugfix (#196)
* Fix VPICIO bug
* Add more checks and error out when no server is selected
* Committing clang-format changes
* Add VPICIO and BDCATS to MPI test

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* Fix vpicio_mts (#199)

---------

Co-authored-by: Houjun Tang <htang4@lbl.gov>
Co-authored-by: Hyunju Oh <oh.693@osu.edu>
Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
@github-actions @houjun
Committing clang-format changes
64fdda4
@houjun
Fix rebase issue
958ff80
@houjun
Fix rebase issue
2510a57
@houjun houjun marked this pull request as ready for review July 1, 2024 21:37
@houjun houjun marked this pull request as draft July 1, 2024 21:38
@houjun houjun marked this pull request as ready for review August 30, 2024 22:42
@jeanbez jeanbez merged commit 275919a into develop Sep 5, 2024
8 checks passed
@houjun houjun deleted the wait_all_fix branch September 10, 2024 18:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhangwei217245 zhangwei217245 zhangwei217245 approved these changes

@jeanbez jeanbez jeanbez approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v.0.6
Development

Successfully merging this pull request may close these issues.
4 participants
@houjun @zhangwei217245 @jeanbez @hyunjuooh