Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Optimize]SparseKV speedup and memory save #32048

Merged
merged 29 commits into from
Apr 20, 2021
Merged

[Optimize]SparseKV speedup and memory save #32048

merged 29 commits into from
Apr 20, 2021

Conversation

seiriosPlus
Copy link
Collaborator

@seiriosPlus seiriosPlus commented Apr 2, 2021 •
edited
Loading

PR types

Performance optimization

PR changes

Others

Describe

  1. 移除稀疏存储时的shared_ptr，在参数维度较小时(维度小于64)，内存消耗降低明显.
  2. 采用roundrobin map 替换 unordered_map， 内存在参数维度较小时(维度小于64)亦有微小降幅，初始化value速度提升。

结论：

  1. 在稀疏参数维度为8/16/32/64 的情况下， 相比于develop， 内存消耗降低 42%, 36%, 28%, 19%.
  2. 在稀疏参数维度为8/16/32/64 的情况下， 相比于develop，初始化速度提升 18%, 15%, 16%, 17%.
  3. merged add PullSparseValue for pull sparse #31661

@seiriosPlus
add PullSparseValue for pull sparse
00738e2 
Change-Id: I516fe974422decaa29c8f1cc70d7ad71f110e732
@seiriosPlus
fix bug for PullSparseValue
2180218 
Change-Id: I4824a4036e1ea5e6da05d66883afe428fe9a339d
@seiriosPlus
fix bug for PullSparseValue
4603446 
Change-Id: I93eedd86d521cc26b05223c3f44c575063e366b3
@seiriosPlus
fix bug for PullSparseValue
67fe1ef 
Change-Id: Idcf48fc9356645d2034e78855dc1a18bb1202cbc
@seiriosPlus
add test mode in lookuptable
67861ed 
Change-Id: Iacfa0e417ba12293c23add5c17aab9fa00be994f
@seiriosPlus
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
15a2691 
… fix/sparse_value

Change-Id: I57f1d38ebb78d3c4363945f848d15b5f6cbe3b0c
@seiriosPlus
fix bug
7e41ea3 
Change-Id: I454921269d44892ec174a2fa7b89847deec661bd
@seiriosPlus
revert API change
b3ac492 
Change-Id: Iac18e5dcc6fb363f844b79581f9d09077f2d9288
@seiriosPlus
merge develop
f2d233b 
Change-Id: Ie858cdec6dae2bbe9e907ca3a57d286b704eb450
@seiriosPlus
merge develop
00ffb70 
Change-Id: Ie35a09772e46f7d90cb68ca82c1d18b9201d1abe
@seiriosPlus
large scale kv store optimize
5f5a4e1 
Change-Id: I582cc661afdaa20749ec7493eae1b88c32b967f7
@seiriosPlus
replace std::unorded_map with roundrobin map
3af8fbc 
Change-Id: I9a821ef8c573b47175f1174700704ae283f6d4f8
@paddle-bot-old
Copy link

paddle-bot-old bot commented Apr 2, 2021

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@seiriosPlus
replace std::unorded_map with roundrobin map
ea42db7 
Change-Id: Ic2334b334e235d6d6ec5a0c0118596a4cbd74497
@seiriosPlus
merge sparse struct
a901f9f 
Change-Id: I93543eb95cece9a68796a290a8dd7ed4bc5150c2
@seiriosPlus
merge sparse struct
f8bfe1c 
Change-Id: I7dc710e6ce4ac028b0bba6ca04d556ae8acdce8f
@seiriosPlus
merge develop
1d798a0 
Change-Id: I80b2327ee7f7bd904d0449c3323eb45b4e4915cc
@seiriosPlus
Merge branch 'develop' into fix/sparse_embedding_test
b8e9e3c
MrChengmo
MrChengmo previously approved these changes Apr 12, 2021
View reviewed changes
Copy link
Contributor

@MrChengmo MrChengmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@seiriosPlus
remove cpplint
89eef16 
Change-Id: Ib131082ae4efc70b40574730d029711d47f6d954
@seiriosPlus
remove cpplint
454b9a8 
Change-Id: I48ee0efef38853876c92d982cdfcac6603c52c88
@seiriosPlus
fix bug
c3c11f6 
Change-Id: I8192a9d5879bd1ef6fb0082c68f2f34a342e7c46
@seiriosPlus
fix bug
be27f4b 
Change-Id: Iaa2785c47294fb2bad5fa09ca984fc2bb6b23ce1
@PaddlePaddle PaddlePaddle locked and limited conversation to collaborators Apr 14, 2021
@PaddlePaddle PaddlePaddle unlocked this conversation Apr 14, 2021
@seiriosPlus
remove license
8065820 
Change-Id: I9752fb5940d5a5fcb5ca6790cb6c7b1135277ccd
chenwhql
chenwhql previously approved these changes Apr 14, 2021
View reviewed changes
Copy link
Contributor

@chenwhql chenwhql left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for const_cast

@seiriosPlus
remove license
b371177 
Change-Id: I2e99709597d6d80bb6f2f9517ce4a426cafc7480
chalsliu
chalsliu previously approved these changes Apr 14, 2021
View reviewed changes
@seiriosPlus
merge develop
05c0838 
Change-Id: I483d79dc37052d2ccca62e81a1a567a0b315546e
@seiriosPlus
merge develop
d5305af 
Change-Id: I5b2e6805d69f2049b11328b44d7fd0b9460eedb0
@seiriosPlus
merge develop
ac024bb 
Change-Id: Iaaa92afc0ecfe0e7fdd9d524e1eb0b7f7d213cec
@seiriosPlus
fix cpp lint
555e13a 
Change-Id: I6ac0187119b025010c99adb7146e5e408574554e
@seiriosPlus
fix cpp lint
427c272 
Change-Id: Ia21fafa65adc09bb9094f7dbc987e31d5af2686e
@PaddlePaddle PaddlePaddle locked and limited conversation to collaborators Apr 16, 2021
@PaddlePaddle PaddlePaddle unlocked this conversation Apr 16, 2021
@seiriosPlus seiriosPlus merged commit 5e7e7c9 into PaddlePaddle:develop Apr 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@luotao1 luotao1 luotao1 approved these changes

@raindrops2sea raindrops2sea raindrops2sea approved these changes

@chalsliu chalsliu chalsliu approved these changes

@chenwhql chenwhql chenwhql left review comments

@MrChengmo MrChengmo MrChengmo left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@seiriosPlus @luotao1 @chenwhql @MrChengmo @raindrops2sea @chalsliu