Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite Makefile shellcheck rule to find all shellscripts #10512

Closed
wants to merge 1 commit into from

Conversation

gdevenyi
Copy link
Contributor

@gdevenyi gdevenyi commented Jun 29, 2020

Motivation and Context

As noted by #10510 a number of bashisms and related issues have crept into the shell scripts in the codebase. The current code does not successfully locate all shell scripts, and hence doesn't test all of them.

Description

Rewrote the find code which locates shell scripts to explicitly pattern patch against the shebang of every file in order to locate sh/bash/dash/ksh scripts.

How Has This Been Tested?

Tested locally on Ubuntu 18.04 it finds more shell scripts now, such as vdev_id

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation (a change to man pages or other documentation)

Checklist:

  • My code follows the ZFS on Linux code style requirements.
  • I have updated the documentation accordingly.
  • I have read the contributing document.
  • I have added tests to cover my changes.
  • I have run the ZFS Test Suite with this change applied.
  • All commit messages are properly formatted and contain Signed-off-by.

@gdevenyi
Copy link
Contributor Author

This change causes the make shellcheck to fail due to a large number of new issues it identifies, in particular, all of the tests have a huge number of note: Double quote to prevent globbing and word splitting. [SC2086]

We could either exclude this check (probably bad idea, I think vdev_id needs to be fixed here) or fix all those files....

shellcheck.log

@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Jun 30, 2020
@behlendorf
Copy link
Contributor

We could either exclude this check (probably bad idea, I think vdev_id needs to be fixed here) or fix all those files....

Initially we excluded some shell scripts while we focused on getting the rest ship shape. The intention was to eventually expand shellcheck to all the scripts as you've done in this PR. However, due to the number of scripts which need to be fixed I think a reasonable next step would be expand the shellcheck to all scripts except the .ksh scripts for now. We can get them in a subsequent pass.

@gdevenyi
Copy link
Contributor Author

Okay, I will re-adjust the code to exclude ksh files.

@behlendorf
Copy link
Contributor

Okay, I will re-adjust the code to exclude ksh files.

Thanks. You'll also need to make sure this PR fixes whatever issues are uncovered in the other scripts before we can merge it. Otherwise we'll break the style check for everyone.

@gdevenyi
Copy link
Contributor Author

Thanks. You'll also need to make sure this PR fixes whatever issues are uncovered in the other scripts before we can merge it. Otherwise we'll break the style check for everyone.

I'll do my best to address the issues. It stands at 1766 excluding the tests so it'll take a while.

@gdevenyi
Copy link
Contributor Author

Update, not as bad, only 259, all the shell scripts in config/ are auto-generated.

We'll have to fix those by auditing the m4 files, which for now I think is out-of-scope of this.

Rewrite the find command which determines which files are
passed to shellcheck in order to include any file with a
shell-like shebang, since not all shell scripts have a
".sh" or ".ksh" extension. Exclude tests until huge
number of issues can be addressed.

Signed-off-by: Gabriel A. Devenyi <gdevenyi@gmail.com>
@mschilli87
Copy link
Contributor

@gdevenyi: I should have some hours to spare this weekend an am familiar with POSIX shell. So if you have trivial work left by Friday evening and think another pair of hands would be useful, feel free to contact me. I'm just not sure if my help would be worth the extra effort for synchronizing between two people changing the code. So no hard feelings if I don't hear from you. 😉

@gdevenyi gdevenyi marked this pull request as draft August 20, 2020 15:09
@gdevenyi
Copy link
Contributor Author

Converting to a draft to avoid accidental merging like #10513 before the bugs are fixed.

Considering #10736 I will not attempt to fix it up since it has a big rewrite and defer to that PR.

@gdevenyi
Copy link
Contributor Author

@mschilli87 I would be happy to accept PRs on this branch on my fork if you want to help contribute to the fixes!

@gdevenyi
Copy link
Contributor Author

Ref: https://github.com/dylanaraps/pure-sh-bible

nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 25, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 26, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 27, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 30, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 31, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 31, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 31, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
nabijaczleweli added a commit to nabijaczleweli/zfs that referenced this pull request May 31, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
@behlendorf behlendorf closed this in c3ef9f7 Jun 1, 2021
behlendorf pushed a commit to behlendorf/zfs that referenced this pull request Jun 8, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
Closes openzfs#12101
behlendorf pushed a commit to behlendorf/zfs that referenced this pull request Jun 9, 2021
This checks every file it checked (and a few more),
but explicitly instead of "if it works it works" best-effort
(which wasn't that good anyway)

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Closes openzfs#10512
Closes openzfs#12101
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Code Review Needed Ready for review and testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants