Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About the recreation of the paper experiment #4

Open
NguyenTuan9347 opened this issue Jun 23, 2024 · 5 comments
Open

About the recreation of the paper experiment #4

NguyenTuan9347 opened this issue Jun 23, 2024 · 5 comments

Comments

@NguyenTuan9347
Copy link

Hello when I read into your code specifically for all benchmark, data used for both evaluating and train is the same dataset and in the original paper there are no mention of the split ratio or where i can find the test and train dataset to get the result in your paper. Can you give me some guild on how I can get / recreate your test and train dataset. With warm regard.

@AmitRoy7781
Copy link
Collaborator

For this work, the combined reconstruction loss is used to calculate the roc-auc between normal and anomaly nodes without spliting into training, validation and test set. Thanks.

@apu20nam
Copy link

apu20nam commented Sep 3, 2024

@AmitRoy7781 @lipan00123 Please can you explain why you didn't split train and test. And if you are calculation roc-auc score between normal and anomaly nodes by giving whole nodes into the train then how can we say that this is best result .

@AmitRoy7781
Copy link
Collaborator

For this paper, we followed the same setting as the benchmarking outlier node detection (BOND) paper to compare with baseline GAD approaches, where the data is not split into training, validation, and test set and the best performance is reported over multiple runs.

For a practical scenario, the training of anomaly detection models is done on the available training data and since the model is unsupervised it can be applied to unseen graphs if it attains a certain level of performance on the training data.

@apu20nam
Copy link

@AmitRoy7781 @lipan00123 Please can you explain about the running time you mentioned in the paper is training time or inference time . if it is inference time how you applied because there has no mention about test data. Please explain it .

@AmitRoy7781
Copy link
Collaborator

@apu20nam We reported per epoch training time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants