Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lightgcn fix #602

Merged
merged 6 commits into from
Mar 18, 2024
Merged

Lightgcn fix #602

merged 6 commits into from
Mar 18, 2024

Conversation

theisjendal
Copy link
Contributor

@theisjendal theisjendal commented Mar 14, 2024

Description

LightGCN is a simplified version of NGCF, however, some of the simplifications made in LightGCN where not included.
Therefore:

  • I have removed normalization between layers as this is not used by LightGCN.
  • Fixed the sum constant used for adding layers to 1/(L+1) and instead of 1/(l+1) where L is the number of layers and l is the current layer index.
  • Previous version of LightGCN didn't support blocks. It now supports DGL blocks.
  • Fixed requirements versions as newer version of DGL/Torch created an error.
  • I moved edge normalization to graph creation making computations in LightGCN easier/simpler.

Checklist:

  • I have added tests.
  • I have updated the documentation accordingly.
  • I have updated README.md (if you are adding a new model).
  • I have updated examples/README.md (if you are adding a new example).
  • I have updated datasets/README.md (if you are adding a new dataset).

theisjendal and others added 4 commits March 14, 2024 09:45
Removed normalization for layers, not used for lgcn.
Fixed sum weight constant to num layers instead of cur layer index.
Allow lgcn to take blocks.
Fixed requirement error caused by newer dgl versions.
Moved edge normalization to graph for easier use.
@theisjendal
Copy link
Contributor Author

theisjendal commented Mar 14, 2024

@tqtg as per #600 (comment), I've created a pull request for LightGCN only.

@lthoang lthoang requested a review from tqtg March 14, 2024 14:40
@tqtg
Copy link
Member

tqtg commented Mar 14, 2024

thanks @theisjendal for clarifying all the changes. They all make sense to me after revising the authors' implementation of LightGCN. Could you also try to compare the updated version and the old one using this example with the same hyper-parameters? I'm curious about the changes in accuracy and speed of convergence.

@theisjendal
Copy link
Contributor Author

theisjendal commented Mar 15, 2024

@tqtg, here are the results:
OLD
Training: 11%, 110/1000 [13:45<1:50:48, 7.47s/iter, loss=0.00479]
Early stopping:

  • best epoch = 61, stopped epoch = 111
  • best monitored value = 0.247716 (delta = -0.005755)

NEW
Training: 62%, 619/1000 [1:11:29<44:35, 7.02s/iter, loss=0.00469]
Early stopping:

  • best epoch = 570, stopped epoch = 620
  • best monitored value = 0.244275 (delta = -0.001430)

VALIDATION:

            | NDCG@20 | Recall@20 | Time (s)
------------ + ------- + --------- + --------
LightGCN-Old |  0.1534 |    0.2420 |   4.9250
LightGCN-New |  0.1552 |    0.2428 |   4.8821

TEST:

             | NDCG@20 | Recall@20 | Train (s) | Test (s)
------------ + ------- + --------- + --------- + --------
LightGCN-Old |  0.1698 |    0.2503 |  831.2188 |   5.2217
LightGCN-New |  0.1717 |    0.2581 | 4294.9170 |   5.2097

@tqtg
Copy link
Member

tqtg commented Mar 15, 2024

LGTM. Feel free to merge this PR.

@lthoang lthoang merged commit a2ee37e into PreferredAI:master Mar 18, 2024
14 checks passed
@darrylong darrylong added the models New models, changes to models label Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
models New models, changes to models
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants