Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking β€œSign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce SOKEmbedding using Sparse Operation Kit #863

Merged
merged 52 commits into from
Mar 21, 2023
Merged
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
0141ad4
new sok class
Nov 4, 2022
d7ad9ba
new sok class
Nov 4, 2022
7825963
test sok dynamic variable
Nov 7, 2022
475145a
test sok dynamic variable
Nov 7, 2022
4ef7a9e
bug fix comma
Nov 7, 2022
720eacf
add some comments and test distributed var
Nov 16, 2022
7a3a177
format the comments
Nov 17, 2022
54f795b
assert condition in sok lookup sparse
Dec 6, 2022
f3136b7
Merge branch 'main' into fea-sok-integration-wj
edknv Dec 14, 2022
7555c6f
Move SOKEmbedding to a separate file
edknv Dec 14, 2022
d0111b1
Clean up
edknv Dec 14, 2022
debb6e2
Clean up
edknv Dec 14, 2022
a4e3ffc
fix some import and param bug
Dec 27, 2022
97d51f5
remove some unused variable
Dec 28, 2022
b978afa
remove intial vals
Dec 28, 2022
4f22c0f
fix import
Dec 28, 2022
16fb414
reorder the import
Dec 28, 2022
9b37d4a
Merge branch 'main' into fea-sok-integration-wj
marcromeyn Jan 9, 2023
557af98
Merge branch 'main' into fea-sok-integration-wj
edknv Jan 9, 2023
f41f52f
fix import error in test embedding
Jan 12, 2023
f03e4ef
Merge branch 'fea-sok-integration-wj' of https://github.com/NVIDIA-Me…
Jan 12, 2023
fe34f9d
format the code
Jan 13, 2023
98bb17b
change the way of import
Jan 13, 2023
b0de517
Merge branch 'main' into fea-sok-integration-wj
WonderingWJ Jan 15, 2023
3e3c3b7
Merge branch 'main' into fea-sok-integration-wj
Jan 15, 2023
f7ff20f
Merge branch 'fea-sok-integration-wj' of https://github.com/NVIDIA-Me…
Jan 15, 2023
f3743e0
Merge branch 'main' into fea-sok-integration-wj
edknv Jan 27, 2023
4e8116c
Merge branch 'main' into fea-sok-integration-wj
edknv Feb 8, 2023
c266cd2
Merge branch 'main' into fea-sok-integration-wj
rnyak Feb 13, 2023
c3373e8
support sp_weights in lookup
Feb 15, 2023
d7531f6
Add unit tests for SOKEmbedding (#980)
edknv Feb 15, 2023
1e4628d
remove sok from ci since no gpu
edknv Feb 15, 2023
25a789d
Merge branch 'main' into fea-sok-integration-wj
edknv Feb 15, 2023
3c0104e
lint
edknv Feb 16, 2023
c8949e2
Merge branch 'main' into fea-sok-integration-wj
edknv Mar 1, 2023
b17fea4
Merge branch 'main' into fea-sok-integration-wj
edknv Mar 6, 2023
296a151
Merge branch 'main' into fea-sok-integration-wj
rnyak Mar 8, 2023
ef1c0bc
Merge branch 'main' into fea-sok-integration-wj
edknv Mar 8, 2023
b716adc
pip install sparse_operation_kit in tox.ini
edknv Mar 8, 2023
b1c1031
fix spelling
edknv Mar 8, 2023
91f25d1
resolve init method issue
Mar 12, 2023
d0671c9
fix schema issue
Mar 12, 2023
0eb266d
add indices and weights in from_pretrained
Mar 12, 2023
e9cc754
schema issue in from_pretrained
Mar 12, 2023
5532ab7
init method in DET
Mar 12, 2023
c9d3baf
tensor type in sok assing
Mar 12, 2023
33abba0
resolve config passing
Mar 15, 2023
4a9465d
Merge branch 'main' into fea-sok-integration-wj
WonderingWJ Mar 15, 2023
b2a01c2
lint
edknv Mar 17, 2023
b862a2a
Merge branch 'main' into fea-sok-integration-wj
edknv Mar 17, 2023
0fd059a
install sok in tox
edknv Mar 17, 2023
fcaefc3
Merge branch 'main' into fea-sok-integration-wj
edknv Mar 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove some unused variable
wenjingz committed Dec 28, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 97d51f585f7f1d9f00f45012db169f8963bf6c81
1 change: 1 addition & 0 deletions merlin/models/tf/distributed/embedding.py
Original file line number Diff line number Diff line change
@@ -141,6 +141,7 @@ def call(self, inputs, combiners, training=True):
@classmethod
def from_pretrained(
cls,
dim: int,
data: list,
trainable=True,
name=None,
7 changes: 0 additions & 7 deletions tests/unit/tf/horovod/test_embedding.py
Original file line number Diff line number Diff line change
@@ -30,9 +30,6 @@ def test_sok_dynamic_variables(self, dim):

rows = [65536 * 10, 65536]
cols = [128, 4]
combiners = ["sum", "sum"]
batch_size = 65536
iters = 100
initial_vals = [13, 17]

# sok variables
@@ -75,10 +72,6 @@ def test_distributed_variables(self, dim):

rows = [65536 * 10, 65536]
cols = [128, 4]
hotness = [10, 3]
combiners = ["sum", "sum"]
batch_size = 65536
iters = 100
initial_vals = [13, 17]

# initial value of embedding table