Reproduce Issues

[zhuhm1996]

Hi, I used the same codes and datasets to tune the parameters provided by the paper. The random seed is set by 0. The followings are the results:

Where the first line represents results from the paper and the second line represents experimental results I conducted.
As you can see, I can not reproduce the results of the paper on many datasets. Would you tell me how to reproduce your results?

[xptree]

@zhuhm1996 Could you share your hyper-param for row 2, especially for RDT-B and RDT-M5K. We can only achieve 77 in RDT-B and 49 in RDT-M5K.

Thanks!

[cruyffturn]

@zhuhm1996 I think there is a typo in the table. The first five columns are social network datasets and the latter four columns are bioinformatics datasets. In the "EXPERIMENTS" section of the paper it's stated that different hidden units are searched for the two different categories.

[weihua916]

Hi, apologies for the delayed response. Two points that are often missed are:

• Have you correctly specified --graph_pooling_type? We used the sum graph pooling for bio/chemistry datasets and the mean graph pooling for social network datasets.
• Have you added --deg_as_tag (use the node degree as the input node feature) for IMDB and COLLAB?

Besides, those datasets are outdated and are too small to rigorously compare different models. I'd suggest working on more interesting and modern graph datasets like https://ogb.stanford.edu .