Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix dataloader for toolkit #61867

Merged
merged 1 commit into from
Feb 22, 2024
Merged

Fix dataloader for toolkit #61867

merged 1 commit into from
Feb 22, 2024

Conversation

KB-Ding
Copy link
Contributor

@KB-Ding KB-Ding commented Feb 20, 2024 •
edited
Loading

PR types

Bug fixes

PR changes

Others

Description

fix dataloader error in toolkit when num_worker > 0.
1708429883

@paddle-bot Paddle-bot
Copy link

paddle-bot bot commented Feb 20, 2024

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@ZeyuChen ZeyuChen merged commit ba94e24 into PaddlePaddle:develop Feb 22, 2024
29 of 30 checks passed
KB-Ding added a commit to KB-Ding/Paddle that referenced this pull request Feb 22, 2024
@KB-Ding
fix dataloaer for toolkit (PaddlePaddle#61867)
80428c5
KB-Ding added a commit to KB-Ding/Paddle that referenced this pull request Feb 22, 2024
@KB-Ding
fix dataloaer for toolkit (PaddlePaddle#61867)
0c6869c
@KB-Ding KB-Ding mentioned this pull request Feb 22, 2024
Merged
ZeyuChen pushed a commit that referenced this pull request Feb 23, 2024
@KB-Ding
fix dataloaer for toolkit (#61867) (#61994)
b50e906
hanhaowen-mt added a commit to hanhaowen-mt/Paddle that referenced this pull request May 13, 2024
@hanhaowen-mt
Revert "fix dataloaer for toolkit (PaddlePaddle#61867) (PaddlePaddle#…
e627c5b 
…61994)"

This reverts commit b50e906.
qili93 pushed a commit that referenced this pull request May 13, 2024
@hanhaowen-mt
paddle_musa v2.6.0 release initialization (#64265)
6caf5d5 
* Revert "fix rpc_sync and rpc_async doc;test=develop (#64107)"

This reverts commit 1319992.

* Revert "[Dy2St][2.6] Disable `test_sentiment` on release/2.6 (#63197)"

This reverts commit 9013831.

* Revert "Revert "fix security (#62626) (#62683)" (#62890)"

This reverts commit 89a60d7.

* Revert "Enhance several unit tests (#62477) (#62776)"

This reverts commit 0348f3f.

* Revert "[Fix_ci] set PLUGIN_TAG release/2.6 (#62731)"

This reverts commit 97ffa07.

* Revert "fix security (#62626) (#62683)"

This reverts commit 6a73547.

* Revert "add more capi to support stride (#62716)"

This reverts commit 683a141.

* Revert "[XPU] default no autotune (#62636)"

This reverts commit fde63d1.

* Revert "[DCU] fix dcu compile failure (#62573)"

This reverts commit d527fb5.

* Revert "[AutoParallel] Adjust time restriction for test_semi_auto_parallel_hybrid_strategy.py (#62278)"

This reverts commit fbf852d.

* Revert "disable llm_int8 ut (#62282)"

This reverts commit e816529.

* Revert "fix openssl-cpu compile bug (#62079) (#62224)"

This reverts commit 59c61db.

* Revert "[CINN] Add IntrinsicOps into ir_codes_collector (#60556) (#62245)"

This reverts commit 773ea41.

* Revert "rm graph_reindex_test (#62057)"

This reverts commit 521dc70.

* Revert "fix (#61923) (#62186)"

This reverts commit d077553.

* Revert "fix cpups training bug:executor trainer use_ps_gpu value;test=develop (#62111)"

This reverts commit d804975.

* Revert "[cherry-pick 2.6] Fix bug of put_along_axis/take_along_axis (#62065)"

This reverts commit 3a083c3.

* Revert "[Cherry-pick] Fix indexing shape bug and Optimize (#62117)"

This reverts commit 609f55e.

* Revert "cherry pick: reduce log for type promotion. (#62116)"

This reverts commit f4d9adf.

* Revert "fix test_communicator_half_async random core;test=develop (#62092)"

This reverts commit dba9992.

* Revert "fix the unqiue op that generate the wrong the inreverse result (#62104)"

This reverts commit b89066a.

* Revert "[Cherry-pick] Fix Paddle-TRT UT fails (#61605)"

This reverts commit 867ab0d.

* Revert "fix se (#61640) (#61702)"

This reverts commit c0f4a49.

* Revert "fix dataloaer for toolkit (#61867) (#61994)"

This reverts commit b50e906.

* Revert "[Cherry-Pick] Fix CacheKV Quant Bug (#61966)"

This reverts commit 04ac1c0.

* Revert "[Paddle-TRT] fix solve (#61806)"

This reverts commit df0155f.

* Revert "fix launch when elastic run (#61847) (#61878)"

This reverts commit f09d9d8.

* Revert "Support Fake GroupWise Quant (#61900)"

This reverts commit 2175de0.

* Revert "repeat_interleave support bf16 dtype (#61854) (#61899)"

This reverts commit 96c2aaf.

* Revert "[security] refine _get_program_cache_key (#61827) (#61896)"

This reverts commit b6a38d0.

* Revert "merge (#61866)"

This reverts commit 39010bf.

* Revert "fix doc style (#61688)"

This reverts commit 12e5c97.

* Revert "fix layer_norm decompose dtyte bugs, polish codes (#61631)"

This reverts commit e5a85b6.

* Revert "remove _wget (#61356) (#61569)"

This reverts commit 9250f66.

* Revert "cinn(py-dsl): skip eval string in python-dsl (#61380) (#61586)"

This reverts commit a37f6fb.

* Revert "Fix unique (#60840) (#61044)"

This reverts commit 3452e61.

* Revert "[CherryPick] Fix issue 60092 (#61427)"

This reverts commit f025385.

* Revert "[cherry-pick] adapt c_embedding to phi namespace for custom devices (#60774) (#61045)"

This reverts commit 0ccb9cb.

* Revert "check eval for security (#61389)"

This reverts commit 60325a1.

* Revert "[Security] fix download security problem (#61162) (#61388)"

This reverts commit 5f3bbeb.

* Revert "[Security] fix security problem for run_cmd (#61285) (#61398)"

This reverts commit 9cd0c91.

* Revert "[Security] fix security problem for prune_by_memory_estimation (#61382)"

This reverts commit af9b8c5.

* Revert "Fix CVE-2024-0521 (#61032) (#61287)"

This reverts commit f99d4f2.

* Revert "fix _decompress security problem (#61294) (#61337)"

This reverts commit 0227a0d.

* Revert "[Security] fix draw security problem (#61161) (#61338)"

This reverts commit aeaa0ca.

* Revert "fix qat tests (#61211) (#61284)"

This reverts commit ff119d0.

* Revert "fix core dump when fallback gather_nd_grad and MemoryAllocateHost (#61067)"

This reverts commit ac1702b.

* Revert "[cherry-pick] This PR enable offset of generator for custom device. (#60616) (#60772)"

This reverts commit 0f732a5.

* Revert "[Cherry-pick] fix set_value with scalar grad (#60930)"

This reverts commit 1aa5f4b.

* Revert "[Dy2St][2.6] Increase `test_transformer` and `test_mobile_net` ut time (#60829) (#60875)"

This reverts commit d788e9b.

* Revert "[Dy2St][2.6] Disable `test_transformer` on `release/2.6` and update README (#60786)"

This reverts commit e738f49.

* Revert "fix bug of ci (#59926) (#60785)"

This reverts commit 7b0d2e9.

* Revert "[Dy2St][2.6] Disable `test_grad` on release/2.6 (#60662)"

This reverts commit e50f43e.

* Revert "[cherry-pick]update pdsa-2023-019 (#60649)"

This reverts commit ccdf528.

* Revert "[cherry-pick]fix fleetutil get_online_pass_interval bug3 (#60620)"

This reverts commit bbc13eb.

* Revert "fix fused_rope diff (#60217) (#60593)"

This reverts commit 97b65c7.

* Revert "fix fleetutil get_online_pass_interval bug2; test=develop (#60545)"

This reverts commit ae2e588.

* Revert "update 2023 security advisory, test=document_fix (#60532)"

This reverts commit 83ce809.

* Revert "add chunk allocator posix_memalign return value check (#60208) (#60495)"

This reverts commit b065877.

* Revert "tile (#60261)"

This reverts commit 203754e.

* Revert "[Cherry-pick] fix weight quant kernel bug when n div 64 != 0 (#60184)"

This reverts commit 20d3558.

* Revert "[Dy2St] Disable `test_bert` on CPU (#60173) (#60324)"

This reverts commit a4cd847.

* Revert "fix windows bug for common lib (#60308)"

This reverts commit 1b696a1.

* update to v2.6.0

* enable WITH_DISTRIBUTED in CMakeLists.txt and port related source file from cuda to musa

* fix some bugs when WITH_DISTRIBUTED is enabled

* delete useless cout in ../paddle/phi/backends/gpu/musa/musa_info.cc and set compute capacity to 9.9 for UT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@heavengate heavengate heavengate approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@KB-Ding @heavengate @ZeyuChen