Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VPICIO bugfix #196

Merged
merged 6 commits into from
May 24, 2024
Merged

VPICIO bugfix #196

merged 6 commits into from
May 24, 2024

Conversation

houjun
Copy link
Member

@houjun houjun commented Apr 26, 2024

Related Issues / Pull Requests

#192

Description

The default "static partitioning" method relies on the object dimension size for partitioning data regions and send the data to the corresponding server, previous vpicio code sets a smaller value than what it should, resulting in no server selected for rank > 0, and thus the flush data size is less than expected.
I also added more checks in the client code to report and print error messages.

What changes are proposed in this pull request?

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality not to work as expected; for instance, examples in this repository must be updated too)
  • This change requires a documentation update

Checklist:

  • My code modifies existing public API, or introduces new public API, and I updated or wrote docstrings
  • I have commented my code
  • My code requires documentation updates, and I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@jeanbez jeanbez changed the title Vpicio bugfix VPICIO bugfix Apr 26, 2024
@jeanbez jeanbez added type: bug Something isn't working priority: medium Medium priority labels Apr 26, 2024
@jeanbez jeanbez added this to the v.0.5 milestone Apr 26, 2024
@jeanbez
Copy link
Member

jeanbez commented Apr 26, 2024

@houjun, would it be interesting to have one test with the local and static partitions? so we can catch this if it happens again

@houjun
Copy link
Member Author

houjun commented Apr 26, 2024

We have tests in the CI for local and static partitions, and since their object dimensions are set correctly, the tests pass without problem. We only tested vpicio and bdcatsio with the serial tests, I will add a test to run them in parallel.

zhangwei217245
zhangwei217245 approved these changes May 3, 2024
View reviewed changes
src/tests/vpicio.c Show resolved Hide resolved
@jeanbez jeanbez merged commit 3a52620 into develop May 24, 2024
8 checks passed
@jeanbez jeanbez assigned jeanbez and houjun and unassigned jeanbez May 24, 2024
@houjun houjun mentioned this pull request Jun 4, 2024
Merged
9 tasks
jeanbez added a commit that referenced this pull request Jun 11, 2024
@jeanbez @houjun
@hyunjuooh @github-actions
Sync develop to stable (v.0.5) (#201)
8d5d097 
* Update getting_started.rst (#184)
* Removing gres option for ctest (#182)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* enable cache by default (#187)
* Removing PDC macro (#189)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core
* Remove PDC macro
* Committing clang-format changes

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* BDCATS fix (#193)
* Fix issues with bdcats_batch
* Committing clang-format changes

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* Update mpi_test.sh (#197)
* Update .gitlab-ci.yml (#195)
* Updates for latest integration with Jacamar and Gitlab tokens in CI
* VPICIO bugfix (#196)
* Fix VPICIO bug
* Add more checks and error out when no server is selected
* Committing clang-format changes
* Add VPICIO and BDCATS to MPI test

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* Fix vpicio_mts (#199)

---------

Co-authored-by: Houjun Tang <htang4@lbl.gov>
Co-authored-by: Hyunju Oh <oh.693@osu.edu>
Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
houjun added a commit that referenced this pull request Jul 1, 2024
@jeanbez @houjun
@hyunjuooh @github-actions
Sync develop to stable (v.0.5) (#201)
bdeef59 
* Update getting_started.rst (#184)
* Removing gres option for ctest (#182)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* enable cache by default (#187)
* Removing PDC macro (#189)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core
* Remove PDC macro
* Committing clang-format changes

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* BDCATS fix (#193)
* Fix issues with bdcats_batch
* Committing clang-format changes

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* Update mpi_test.sh (#197)
* Update .gitlab-ci.yml (#195)
* Updates for latest integration with Jacamar and Gitlab tokens in CI
* VPICIO bugfix (#196)
* Fix VPICIO bug
* Add more checks and error out when no server is selected
* Committing clang-format changes
* Add VPICIO and BDCATS to MPI test

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* Fix vpicio_mts (#199)

---------

Co-authored-by: Houjun Tang <htang4@lbl.gov>
Co-authored-by: Hyunju Oh <oh.693@osu.edu>
Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
jeanbez added a commit that referenced this pull request Sep 5, 2024
@houjun @hyunjuooh
@jeanbez @github-actions
Multi-thread fix and request merging (#205)
275919a 
* Update getting_started.rst (#184)
* Removing gres option for ctest (#182)
* Removing gres option from scripts
* Update check for core

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* Fix an issue with region transfer request
* Committing clang-format changes
* Merge small requests when they are contiguous and 1D, change srun commands in run scripts to detect Perlmutter compute nodes
* Merge only for REGION_LOCAL partition
* Committing clang-format changes
* Fix a bug that causes some tests to fail
* Fix a couple of issues with start/wait all
* Committing clang-format changes
* Add aggregation support for contiguous read operations
* Committing clang-format changes
* Fix compile issue when multithread is enabled
* Committing clang-format changes
* minor change with test code
* Committing clang-format changes
* Remove metadata mutex for multi threading
* Committing clang-format changes
* Fix mutex
* Committing clang-format changes
* Fix an issue when closing an obj
* Sync develop to stable (v.0.5) (#201)
* Update getting_started.rst (#184)
* Removing gres option for ctest (#182)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* enable cache by default (#187)
* Removing PDC macro (#189)
* Removing gres option for ctest
* Removing gres option from scripts
* Update check for core
* Remove PDC macro
* Committing clang-format changes

---------

Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* BDCATS fix (#193)
* Fix issues with bdcats_batch
* Committing clang-format changes

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* Update mpi_test.sh (#197)
* Update .gitlab-ci.yml (#195)
* Updates for latest integration with Jacamar and Gitlab tokens in CI
* VPICIO bugfix (#196)
* Fix VPICIO bug
* Add more checks and error out when no server is selected
* Committing clang-format changes
* Add VPICIO and BDCATS to MPI test

---------

Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>

* Fix vpicio_mts (#199)

---------

Co-authored-by: Houjun Tang <htang4@lbl.gov>
Co-authored-by: Hyunju Oh <oh.693@osu.edu>
Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>

* Committing clang-format changes
* Fix rebase issue
* Add timers
* Committing clang-format changes
* Add explict transfer start (all) with MPI communicator
* Committing clang-format changes
* MPI fix
* remove debug msg
* Committing clang-format changes
* Add function comment for doc
* Revert script changes
* Committing clang-format changes
* Revert script changes
* Committing clang-format changes
* Revert script setting

---------

Co-authored-by: Hyunju Oh <oh.693@osu.edu>
Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov>
Co-authored-by: Jean Luca Bez <jlbez@lbl.gov>
Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
@houjun houjun deleted the vpicio_bugfix branch September 10, 2024 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jeanbez jeanbez jeanbez approved these changes

@zhangwei217245 zhangwei217245 zhangwei217245 approved these changes

Assignees

@houjun houjun

Labels
priority: medium Medium priority type: bug Something isn't working
Projects
None yet
Milestone
v.0.5
Development

Successfully merging this pull request may close these issues.
3 participants
@houjun @jeanbez @zhangwei217245