Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hcllint: add a linting check for HCL files #7791

Merged
merged 1 commit into from
Apr 23, 2020
Merged

hcllint: add a linting check for HCL files #7791

merged 1 commit into from
Apr 23, 2020

Conversation

tgross
Copy link
Member

@tgross tgross commented Apr 23, 2020

Running make dev runs hclfmt, but this isn't checked as part of CI. That makes it possible to merge un-formatted HCL and Nomad jobspecs that later will make for dirty git staging areas when developers pull master. (Examples: #7584 #7011 #6724)

This adds a make hcllint target and wires it up to CI. Note that hclfmt doesn't return a non-0 exit code if it makes changes, which is why we can't just re-use make hclfmt here.


Looks like the following when run:

▶ make hcllint
6a7
>
17c18
< }
\ No newline at end of file
---
> }
6a7
>
11a13
>
15a18
>
19a23
>
26a31
>
33a39
>
42c48
< }
\ No newline at end of file
---
> }
6a7
>
8c9,10
<         sidecar_service {}
---
>         sidecar_service = {}
>
15c17
< }
\ No newline at end of file
---
> }
make: *** [hcllint] Error 1

▶ echo $?
2

▶ make hclfmt
--> Formatting HCL

▶ make hcllint

▶ echo $?
0

@tgross tgross force-pushed the hcllint branch 3 times, most recently from 7c88e9d to fb66f9c Compare April 23, 2020 14:44
@tgross tgross requested review from notnoop and shoenig April 23, 2020 14:44
GNUmakefile Outdated Show resolved Hide resolved
@tgross tgross force-pushed the hcllint branch 2 times, most recently from 84e41bd to f4463b2 Compare April 23, 2020 14:51
@notnoop
Copy link
Contributor

notnoop commented Apr 23, 2020

It would be nice if the output emitted the name of files that need to be formatted, rather than plain diff.

When faced with a similar issue in managing protobuf generation being out of sync, I used another way in https://github.com/hashicorp/nomad/blob/v0.11.1/GNUmakefile#L205-L207 . Since we effectively always want to run generation and they are idempotent, we can just run generation and see if any files changed. In CI it will emit the failed files. When run locally, it will update and report the files, and you can just commit then without any additional steps. What do you think about that approach?

GNUmakefile Outdated
@find . -path ./terraform -prune -o -name 'upstart.nomad' \
-prune -o \( -name '*.nomad' -o -name '*.hcl' \) \
-exec bash -c "diff {} <(hclfmt {})" ';' \
| tee /dev/stderr | wc -l | head -1 | grep -q ' *0$$'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point it might be worth to make a hcldiff.sh with some comments 😅

@shoenig
Copy link
Member

shoenig commented Apr 23, 2020

Thanks for doing this, @tgross !!

@tgross
Copy link
Member Author

tgross commented Apr 23, 2020

I like @notnoop's suggested approach, and that'll read easier in the Makefile too. Fix incoming.

GNUmakefile Outdated
@@ -206,6 +206,10 @@ check: ## Lint the source code
@$(MAKE) proto
@if (git status | grep -q .pb.go); then echo the following proto files are out of sync; git status |grep .pb.go; exit 1; fi

@echo "==> Check format of jobspecs and HCL files..."
@$(MAKE) hclfmt
@if (git status -s | grep -q '\.hcl\|\.nomad'); then echo the following HCL files are out of sync; git status | grep '\.hcl\|\.nomad'; exit 1; fi
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@notnoop how's this look? I ended up using git status -s to just get the files because otherwise the whole output invariably seemed to include nomad and I was getting false positives 😀

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.. You can also do a git diff --name-only to print out only the file changes. We probably could do that with the pb.go check.

Also, I find using -e a tiny bit more readable, e.g. grep -e '\.hcl$' -e '\.nomad$'?

Running `make dev` runs `hclfmt`, but this isn't checked as part of
CI. That makes it possible to merge un-formatted HCL and Nomad
jobspecs that later will make for dirty git staging areas when
developers pull master.

This changeset adds HCL linting to the `make check` target.

@echo "==> Check format of jobspecs and HCL files..."
@$(MAKE) hclfmt
@if (git status -s | grep -q -e '\.hcl$$' -e '\.nomad$$'); then echo the following HCL files are out of sync; git status -s | grep -e '\.hcl$$' -e '\.nomad$$'; exit 1; fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only need a single $ here.

Suggested change
@if (git status -s | grep -q -e '\.hcl$$' -e '\.nomad$$'); then echo the following HCL files are out of sync; git status -s | grep -e '\.hcl$$' -e '\.nomad$$'; exit 1; fi
@if (git status -s | grep -q -e '\.hcl$' -e '\.nomad$'); then echo the following HCL files are out of sync; git status -s | grep -e '\.hcl$' -e '\.nomad$'; exit 1; fi

Copy link
Member Author

@tgross tgross Apr 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought so at first but when I tested it, I realized Make wants you to escape the $, so the anchor was being ignored without that and the results were wrong.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:til: Thanks for your testing :). Ship it!

@tgross tgross merged commit 8af65c5 into master Apr 23, 2020
@tgross tgross deleted the hcllint branch April 23, 2020 18:32
@github-actions
Copy link

github-actions bot commented Jan 9, 2023

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants