fix entry #31079

seiriosPlus · 2021-02-20T07:35:49Z

PR types

Bug fixes

PR changes

Others

Describe

飞桨分布式准入策略新增API:

paddle.distributed.ProbabilityEntry：概率准入
paddle.distributed.CountFilterEntry：频次准入

用法示例：

import paddle
paddle.enable_static()

embedding_dim = 128
# entry = paddle.distributed.ProbabilityEntry(10)
entry = paddle.distributed.CountFilterEntry(10)

x = paddle.static.data(name='X', shape=[1], dtype='int64')

emb = paddle.static.nn.sparse_embedding(
    input=x,
    size=[1024, embedding_dim],
    is_test=False,
    entry=CountFilterEntry(10),
    param_attr=paddle.ParamAttr(name="SparseFeatFactors"))

修复准入配置未正确传递给Server端的bug。
修复组网中多个embedding的输入输出存在冲突导致fuse失败的bug。
将准入策略 ProbabilityEntry 和 CountFilterEntry 迁移至 paddle.distributed 下。

Change-Id: Icb5976151444720e9f186d0a33ac5fb0b74c2101

Change-Id: Ifb721421d29b6d22398499084062c2b2bd3031fd

Change-Id: If63c78702938677ca03de151aaf88b75794b9f78

… save_load_shrink Change-Id: I954aed20aa24003aa2e16fe29006af5ca8dbfdec

Change-Id: I14edfa162ea8bf8ca5647b30d90be7085ca70552

Change-Id: I212801608ebdcb773336b24b7d65c338ffe5eda0

… save_load_shrink Change-Id: I8efcd47f53b56edd719409910dce761068b99473

Change-Id: I599e04422f19f333243aaafd040a4c5dcfa033b5

… save_load_shrink Change-Id: Icb60dcf7b86bd88aaceb48ad1fad16253f4f93f6

paddle-bot-old · 2021-02-20T07:36:05Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Change-Id: I98d390ada7e875411015c39f35c2d1d0e130b0b8

Change-Id: Ifbcf29d1a66a7875f8e509892ebbd94c7feb03cb

Change-Id: Ie5f31230df3d162fee7087a7deb55ebdfb21498f

Change-Id: If86c71b17f18c88561c777766a5762818ea682d9

Change-Id: Ib8229c92a68aa28dd00200ba677ad81e8400eaf9

Change-Id: Ibac5367961723cca3f45205a85054e77b126a4c1

Change-Id: I0ed1c8bf21109c3016c836d7454ce90ee7f1d8be

Change-Id: Ief21b91f0859da1997b3cd1683be6691edf96c5d

Change-Id: Ib2149d68e948bc8d72041cd7d3904d9b2c7f9036

Change-Id: I482f7a21fdfaedfb6e28e56ff504ecc9d7790e14

Change-Id: Iba7b87922bdd6503782c2e99413e5280acd33338

Change-Id: Iaa9bb2b2b59ffa23394f9b207957d36e30c6823f

Change-Id: Ia907a88aa5e1351312d3441e931f43f43389ff83

Change-Id: Iabbdc8c6ad438262e345f93ef20e4bb89c98cf9a

Change-Id: I8232e801ad0ba9282eb1f5ff4ec543eea63b8161

TCChenlong · 2021-02-23T14:25:29Z

python/paddle/distributed/entry_attr.py

+
+            input = paddle.static.data(name='ins', shape=[1], dtype='int64')
+
+            emb = fluid.contrib.layers.sparse_embedding(


fluid.contrib.layers.sparse_embedding -> paddle.static.nn.sparse_embedding

TCChenlong · 2021-02-23T14:25:45Z

python/paddle/distributed/entry_attr.py

+                size=[sparse_feature_dim, embedding_size],
+                is_test=False,
+                entry=entry,
+                param_attr=fluid.ParamAttr(name="SparseFeatFactors",


fluid.ParamAttr -> paddle.ParamAttr

TCChenlong · 2021-02-23T14:26:11Z

python/paddle/distributed/entry_attr.py

+                is_test=False,
+                entry=entry,
+                param_attr=fluid.ParamAttr(name="SparseFeatFactors",
+                                           initializer=fluid.initializer.Uniform()))


fluid.initializer.Uniform() -> paddle.nn.initializer.Uniform

Change-Id: Ib5c3c34c7f97b68c5609b5514aeb2dcb008a9dc2

TCChenlong

LGTM

zhiqiu

LGTM for contrib API sparse_embedding without core.ops

* fix entry * fix distributed lookup table fuse case * fix entry bug at first time * move entry from paddle.fluid -> paddle.distributed * fix ut with paddle.enable_static() Co-authored-by: malin10 <malin10@baidu.com>

* fix entry * fix distributed lookup table fuse case * fix entry bug at first time * move entry from paddle.fluid -> paddle.distributed * fix ut with paddle.enable_static() Co-authored-by: malin10 <malin10@baidu.com> Co-authored-by: malin10 <malin10@baidu.com>

123malin and others added 11 commits January 21, 2021 11:30

test=develop, save/load, shrink

5e915fe

test=develop, bug fix

d8366e4

fix code format

60eae81

Change-Id: Icb5976151444720e9f186d0a33ac5fb0b74c2101

fix code format

49f0c58

Change-Id: Ifb721421d29b6d22398499084062c2b2bd3031fd

fix code format

f50f254

Change-Id: If63c78702938677ca03de151aaf88b75794b9f78

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ff2b388

… save_load_shrink Change-Id: I954aed20aa24003aa2e16fe29006af5ca8dbfdec

fix entry

c125a92

Change-Id: I14edfa162ea8bf8ca5647b30d90be7085ca70552

fix entry

3cf6214

Change-Id: I212801608ebdcb773336b24b7d65c338ffe5eda0

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

6f162c1

… save_load_shrink Change-Id: I8efcd47f53b56edd719409910dce761068b99473

fix entry

9e112c6

Change-Id: I599e04422f19f333243aaafd040a4c5dcfa033b5

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

cefc2ae

… save_load_shrink Change-Id: Icb60dcf7b86bd88aaceb48ad1fad16253f4f93f6

merge develop

4330ff7

Change-Id: I98d390ada7e875411015c39f35c2d1d0e130b0b8

seiriosPlus changed the title ~~Feature/save load shrink~~ fix entry Feb 20, 2021

seiriosPlus added 4 commits February 20, 2021 17:18

fix entry

693b8a5

Change-Id: Ifbcf29d1a66a7875f8e509892ebbd94c7feb03cb

fix distributed lookup table fuse case

2483171

Change-Id: Ie5f31230df3d162fee7087a7deb55ebdfb21498f

fix entry ut case

fd6d6a5

Change-Id: If86c71b17f18c88561c777766a5762818ea682d9

fix entry display

1622885

Change-Id: Ib8229c92a68aa28dd00200ba677ad81e8400eaf9

PaddlePaddle locked as resolved and limited conversation to collaborators Feb 22, 2021

PaddlePaddle unlocked this conversation Feb 22, 2021

fix entry display

989394d

Change-Id: Ibac5367961723cca3f45205a85054e77b126a4c1

seiriosPlus mentioned this pull request Feb 22, 2021

[Chery-Pick]online learning fix #31107

Merged

seiriosPlus added 8 commits February 22, 2021 15:28

fix entry bug at first time

c9cfda5

Change-Id: I0ed1c8bf21109c3016c836d7454ce90ee7f1d8be

move entry from paddle.fluid -> paddle.distributed

aca0207

Change-Id: Ief21b91f0859da1997b3cd1683be6691edf96c5d

move entry from paddle.fluid -> paddle.distributed

5502ed1

Change-Id: Ib2149d68e948bc8d72041cd7d3904d9b2c7f9036

move entry from paddle.fluid -> paddle.distributed

b336bb1

Change-Id: I482f7a21fdfaedfb6e28e56ff504ecc9d7790e14

move entry from paddle.fluid -> paddle.distributed

bdee7c7

Change-Id: Iba7b87922bdd6503782c2e99413e5280acd33338

fix ut with paddle.enable_static()

b089b4f

Change-Id: Iaa9bb2b2b59ffa23394f9b207957d36e30c6823f

fix ut with paddle.enable_static()

70a5949

Change-Id: Ia907a88aa5e1351312d3441e931f43f43389ff83

fix ut with paddle.enable_static()

3e6f7b5

Change-Id: Iabbdc8c6ad438262e345f93ef20e4bb89c98cf9a

fix embedding doc with paddle.static.nn

4bfc3f1

Change-Id: I8232e801ad0ba9282eb1f5ff4ec543eea63b8161

TCChenlong reviewed Feb 23, 2021

View reviewed changes

move entry from paddle.fluid -> paddle.distributed

1856512

Change-Id: Ib5c3c34c7f97b68c5609b5514aeb2dcb008a9dc2

TCChenlong approved these changes Feb 23, 2021

View reviewed changes

zhiqiu self-requested a review February 24, 2021 02:49

zhiqiu approved these changes Feb 24, 2021

View reviewed changes

fuyinno4 approved these changes Feb 24, 2021

View reviewed changes

lanxianghit approved these changes Feb 24, 2021

View reviewed changes

luotao1 approved these changes Feb 24, 2021

View reviewed changes

seiriosPlus merged commit ebbdf52 into PaddlePaddle:develop Feb 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix entry #31079

fix entry #31079

seiriosPlus commented Feb 20, 2021 •

edited

Loading

paddle-bot-old bot commented Feb 20, 2021

TCChenlong Feb 23, 2021

TCChenlong Feb 23, 2021

TCChenlong Feb 23, 2021

TCChenlong left a comment

zhiqiu left a comment


		input = paddle.static.data(name='ins', shape=[1], dtype='int64')

		emb = fluid.contrib.layers.sparse_embedding(

fix entry #31079

fix entry #31079

Conversation

seiriosPlus commented Feb 20, 2021 • edited Loading

PR types

PR changes

Describe

paddle-bot-old bot commented Feb 20, 2021

TCChenlong Feb 23, 2021

Choose a reason for hiding this comment

TCChenlong Feb 23, 2021

Choose a reason for hiding this comment

TCChenlong Feb 23, 2021

Choose a reason for hiding this comment

TCChenlong left a comment

Choose a reason for hiding this comment

zhiqiu left a comment

Choose a reason for hiding this comment

seiriosPlus commented Feb 20, 2021 •

edited

Loading