Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to reproduce results #2

Open
robmarkcole opened this issue Oct 18, 2022 · 1 comment
Open

Unable to reproduce results #2

robmarkcole opened this issue Oct 18, 2022 · 1 comment

Comments

@robmarkcole
Copy link

Hi
I noticed the parameters in IQF-UseCase.ipynb will not produce the final plot. I have tried to reproduce but my plot results look quite different, with larger variation and noise_0 having highest loss. Can you advise how to reproduce the results?

ds_modifiers_list = [
    CustomNoiseModifier(params = {"sigma": 1*f})
    for f in [0,0.5,1,2,4,8, 16, 32] # add 16, 32
]

experiment = ExperimentSetup(
    experiment_name   = experiment_name,
    task_instance     = task,
    ref_dsw_train     = ds_wrapper_train,
    ref_dsw_val       = ds_wrapper_valid,
    ds_modifiers_list = ds_modifiers_list,
    repetitions       = 5 # increase from 1
)

image

@paugalles
Copy link

Yes, you are correct. The plot was not updated according to the latest setup.
The only randomness is fixed in this line here. Thus the solution is deterministic.

Since I am updating the use case as per #3 I will update the plot there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants