Edit LRFinder to have more than one parameter

[JakubDz2208]

Fixes #2703

Description:
Editing FastaiLRFinder to have more than one parameter

[vfdev-5]

Thanks for the PR @Jacob208M
I left some comments on how to improve the code.
We also need to write some tests. Please check https://github.com/pytorch/ignite/blob/master/tests/ignite/handlers/test_lr_finder.py

For development tips please see: https://github.com/pytorch/ignite/blob/master/CONTRIBUTING.md

[vfdev-5]

Thanks for the update @Jacob208M
We need to work more on the code to make it better. I left few comments on what to do. Feel free to ask questions if it is not clear.

[sadra-barikbin]

Hi @Jacob208M , thanks for this PR. Would you like any help?

[JakubDz2208]

@sadra-barikbin Can you review this changes is everything fine please?

[sadra-barikbin]

@sadra-barikbin Can you review this changes is everything fine please?

@Jacob208M I reviewed your changes and put some comments. Are they clear?

[JakubDz2208]

@sadra-barikbin Can you review this changes is everything fine please?

@Jacob208M I reviewed your changes and put some comments. Are they clear?

Yes, I'll manage it

[JakubDz2208]

@sadra-barikbin I just updated my code as you showed me. I haven't added ParamGroupScheduler because I wasn't sure about it.

[sadra-barikbin]

Sorry for delay. I left some comments and change requests.

[JakubDz2208]

@sadra-barikbin I did changes but now I got my tests failed with info

FAILED test_lr_finder.py::test_multi_group_optimizer[exp] - assert False FAILED test_lr_finder.py::test_multi_group_optimizer[linear] - TypeError: unsupported operand type(s) for -: 'list' and 'list'

Is it a problem with param_scheduler? What should I do with it?

[sadra-barikbin]

@sadra-barikbin I did changes but now I got my tests failed with info

FAILED test_lr_finder.py::test_multi_group_optimizer[exp] - assert False FAILED test_lr_finder.py::test_multi_group_optimizer[linear] - TypeError: unsupported operand type(s) for -: 'list' and 'list'

Is it a problem with param_scheduler? What should I do with it?

Thanks. Could you please push the changes to see where the error arises from?

[JakubDz2208]

@sadra-barikbin I did changes but now I got my tests failed with info
FAILED test_lr_finder.py::test_multi_group_optimizer[exp] - assert False FAILED test_lr_finder.py::test_multi_group_optimizer[linear] - TypeError: unsupported operand type(s) for -: 'list' and 'list'
Is it a problem with param_scheduler? What should I do with it?

Thanks. Could you please push the changes to see where the error arises from?

Done

[JakubDz2208]

@sadra-barikbin Hi, sorry for my absence, I was busy because my studies have started. I committed your changes but I still see there is an error, now it looks like that:
ERROR ignite.engine.engine.Engine:engine.py:1105 Current run is terminating due to exception: 'ParamGroupScheduler' object has no attribute 'get_param' ERROR ignite.engine.engine.Engine:engine.py:1000 Engine run is terminating due to exception: 'ParamGroupScheduler' object has no attribute 'get_param' ====================================================================================== short test summary info ====================================================================================== FAILED test_lr_finder.py::test_multi_group_optimizer[exp] - AttributeError: 'ParamGroupScheduler' object has no attribute 'get_param' FAILED test_lr_finder.py::test_multi_group_optimizer[linear] - AttributeError: 'ParamGroupScheduler' object has no attribute 'get_param'

Can you check it please?

[sadra-barikbin]

@sadra-barikbin Hi, sorry for my absence, I was busy because my studies have started. I committed your changes but I still see there is an error, now it looks like that: ERROR ignite.engine.engine.Engine:engine.py:1105 Current run is terminating due to exception: 'ParamGroupScheduler' object has no attribute 'get_param' ERROR ignite.engine.engine.Engine:engine.py:1000 Engine run is terminating due to exception: 'ParamGroupScheduler' object has no attribute 'get_param' ====================================================================================== short test summary info ====================================================================================== FAILED test_lr_finder.py::test_multi_group_optimizer[exp] - AttributeError: 'ParamGroupScheduler' object has no attribute 'get_param' FAILED test_lr_finder.py::test_multi_group_optimizer[linear] - AttributeError: 'ParamGroupScheduler' object has no attribute 'get_param'

Can you check it please?

get_param is now added to branch master in ignite. You should pull ignite changes to your master. Simply call git fetch && git pull origin master

[JakubDz2208]

I pulled new changes and now there is no error from get_param, but simple assertion False. I'm not sure why. Can you look at it?

[sadra-barikbin]

"Check code formatting" step in jobs is failing. Let's fix it first. flake8 which is a code style enforcement tool is raising those errors. You can see them by clicking Details in one of failed jobs bottom.

To install flake8 and other formatting tools being used by Ignite do:

./tests/run_code_style.sh install

You can see errors locally with:

./tests/run_code_style.sh lint

[vfdev-5]

@Jacob208M can you please follow and apply suggestions from this comment: #2704 (comment). so we can advance on this feature. Thanks!

[JakubDz2208]

I think is done, flake8 doesn't throw any errors, tests are passing. Could you look at this, please?

[vfdev-5]

@Jacob208M can you also run code formatting script:

./tests/run_code_style.sh fmt

[sadra-barikbin]

I forgot to tell you calling ./tests/run_code_style.sh fmt applies formatting rules automatically and there were no need to redefine variables with short names. Applying that command would result in:

def test_multiple_optimizers(
    lr_finder, dummy_engine_mulitple_param_groups, to_save_mulitple_param_groups, dataloader, step_mode
):
    start_lr = [0.1, 0.1, 0.01]
    end_lr = [1.0, 1.0, 1.0]

    with lr_finder.attach(
        dummy_engine_mulitple_param_groups,
        to_save_mulitple_param_groups,
        start_lr=start_lr,
        end_lr=end_lr,
        step_mode=step_mode,
    ) as trainer:
        trainer.run(dataloader)

[sadra-barikbin]

By the way for more information on Python style guides you could read PEP 8.

[vfdev-5]

@Jacob208M @sadra-barikbin Thank both of you guys for this PR !
Looks good to me