Make repos flake8 clean and enable it on CI

[Flamefire]

As discussed with @boegel it is a good idea to run flake8 on the whole code as part of CI to catch issues that hound misses, e.g. formatting making adjacent lines "invalid".

No not have 1 huge, unmanageable PR I propose to tackle 1 folder at a time. This has the benefit of being able to parallelize efforts. Example PR: easybuilders/easybuild-framework#3282

To get this done fast I propose to do that in a sprint with a couple people "cleaning" folders one by one. And finally enable flake8 as a Github Action on the repo root, see https://github.com/easybuilders/easybuild-framework/pull/3282/files#diff-fa7ae6d9a59a8cf2353dbb6eb115ff68

Preparation:

• Checkout develop branch and pull if required
• Upgrade flake8 and autopep8 (add --user if wanted or not using pyenv/virtualenv):
  • pip install --upgrade pip
  • pip install --upgrade flake8 autopep8
• Bonus: Install VSCode, the Python Plugin and enable autopep8 as the linter

Plan to do for each folder:

• autopep8 -r -i --max-line-length 120 <folder> to fix "simple" issues unlikely to cause problems
• Commit
• autopep8 -a -r -i --max-line-length 120 <folder> to fix harder issues which need to be checked at least
• Commit
• Iteratively run flake8 <folder> and fix issues
• Commit
• Open PR

On the start of the sprint all folders will be listed here or in the channel and each assigned to 1 developer. It might make sense to have 1 dedicated person for reviews

Non-goal:
This sprint should focus on getting the code to run with clean flake8. In the process many places for code improvements can be detected. In order to keep it easy for reviewers I propose to not fix or clean surrounding code unless this is introduced by the flake8/autopep8 change or helps for that and is sufficiently small (must be visibly correct on first glance).
Otherwise the amount of changes will be to much to handle for the reviewer.

Tasks

easybuild-framework repo (317 warnings)

• contrib (2 warnings) @boegel
• easybuild/ (77 warnings) => @wpoely86 @boegel (done, see make flake8 code style checks pass fully easybuild-framework#3417 => review/merge by @wpoely86 pending)
  • easybuild/base (no warnings)
  • easybuild/framework (10 warnings)
  • easybuild/scripts (10 warnings)
  • easybuild/toolchains (57 warnings)
  • easybuild/tools (done in fix code style issues in easybuild.tools + add flake8 linting test easybuild-framework#3282)
• test/ (238 warnings) => @boegel (done, see make flake8 code style checks pass in test/ easybuild-framework#3416)
  • test/framework/options.py (152 warnings)
  • test/framework/{config,containers,docs,easyblock,easyconfigversion,filetools,general,github,lib}.py (37 warnings)
  • [x]test/framework/{parallelbuild,robot,systemtools,toolchain,toy_build,tweak,type_checking}.py (37 warnings)
  • [x]test/framework/sandbox (12 warnings)

easybuild-easyblocks repo (562 warnings)

=> assigned to @Flamefire

• work alphabetically, top to bottom (a -> z)

• please open PRs per letter subdir, for easy testing with --include-easyblocks-from-pr

• @wpoely86 and @boegel can help out after tackling easybuild-framework, and work backwards (from z -> a)

• easybuild/easyblocks (547 warnings)

  • easybuild/easyblocks/__init__.py (1 warnings)
  • easybuild/easyblocks/0 (no warnings)
  • easybuild/easyblocks/a (46 warnings)
  • easybuild/easyblocks/b (18 warnings)
  • easybuild/easyblocks/c (29 warnings)
  • easybuild/easyblocks/d (59 warnings)
  • easybuild/easyblocks/e (14 warnings)
  • easybuild/easyblocks/f (16 warnings)
  • easybuild/easyblocks/g (31 warnings)
  • easybuild/easyblocks/generic (12 warnings)
  • easybuild/easyblocks/h (11 warnings)
  • easybuild/easyblocks/i (16 warnings)
  • easybuild/easyblocks/j (0 warnings)
  • easybuild/easyblocks/k (0 warnings)
  • easybuild/easyblocks/l (18 warnings)
  • easybuild/easyblocks/m (56 warnings)
  • easybuild/easyblocks/n (54 warnings)
  • easybuild/easyblocks/o (10 warnings)
  • easybuild/easyblocks/p (27 warnings)
  • easybuild/easyblocks/q (2 warnings)
  • easybuild/easyblocks/r (21 warnings) => @wpoely86
  • easybuild/easyblocks/s (42 warnings) => @wpoely86
  • easybuild/easyblocks/t (19 warnings) => @wpoely86
  • easybuild/easyblocks/{u,v} (2 warnings) => @wpoely86
  • easybuild/easyblocks/w (42 warnings) => @wpoely86
  • easybuild/easyblocks/x (no warnings)
  • easybuild/easyblocks/y (no warnings)
  • easybuild/easyblocks/z (no warnings)

• setup.py (2 warnings) => @wpoely86

• test/ (14 warnings) => @wpoely86

easybuild-easyconfigs repo (36 warnings)

• find . -name '*.py' | xargs flake8
  • test/ (12 warnings) (done, see fix code style issues in test/ easybuild-easyconfigs#11146)
  • easybuild/easyconfigs/l/LAMMPS/lammps_vs_yaff_test_single_point_energy.py (24 warnings) (done, see Enable Flake8 on CI and fix issues easybuild-easyconfigs#11147)

easybuild repo (30 warnings)

• find . -name '*.py' | xargs flake8 (done, see Enable flake8 on CI and fix issues #644)

[boegel]

Agreed on the last point, when such stuff comes into view, please open an issue to follow up on it!

[boegel]

We're running flake8 in CI in all easybuild-* repos now, so closing this...