`terraform modules` cmd human view #36062

Maed223 · 2024-11-19T15:42:01Z

Implements a human view for what was previously a machine readable view only command terraform modules -json. The human view shows the nested hierarchy/composition of modules in configuration rather than the flat list given by the json alternative, as well as additional info about the provided version constraints for the module. Documentation and the command's help output has been updated in accordance with this addition.

To accommodate this nested format, some reworking of resolver.go has been done to produce a manifest with a nested structure of records (whereas before a flat list was given).

Previous context on the `modules` command

The modules command was initially implemented (PR here) in order to provide a means of seeing the collection of modules in current use. This need arose as a part of the module deprecation/revocation project where a list of modules currently in config was needed (as a source of truth to check for deprecations/revocations in the agent). We found that modules.json provided a historical list of all modules to ever be in configuration which wasn't suitable for our use case. This command provides the means to view the modules currently in config by cross checking this historical list with the current state of configuration. As this output (for our purposes) was only being consumed by the agent, the human view had been omitted in the initial implementation. This PR seeks to close that gap.

Target Release

1.11.x

Draft CHANGELOG entry

NEW FEATURES

A human readable view for the terraform modules command.

github-actions · 2024-11-19T16:10:53Z

The equivalence tests will be updated. Please verify the changes here.

github-actions · 2024-11-19T16:12:49Z

The equivalence tests will be updated. Please verify the changes here.

rkoron007

Looks great! Added a small opportunistic suggestion

website/docs/cli/commands/modules.mdx

github-actions · 2024-11-20T22:19:58Z

The equivalence tests will be updated. Please verify the changes here.

rkoron007

Approved with some small tweaks for style!

rkoron007 · 2024-11-21T22:23:24Z

website/docs/cli/commands/modules.mdx

+The following example is a representation of the human readable output that `terraform modules` returns. The nested structure represents the 
+module composition/heirarchy within your configuration.
+
+```


Should this be shell? Or something like it? We wanna make sure we get syntax highlighting if we can!

Suggested change

```

```shell-session

Included 👍 e2c5b39

rkoron007 · 2024-11-21T22:25:08Z

website/docs/cli/commands/modules.mdx


 We will introduce new major versions only within the bounds of
 [the Terraform 1.0 Compatibility Promises](/terraform/language/v1-compatibility-promises).

 ## Output Format

+The following example is a representation of the human readable output that `terraform modules` returns. The nested structure represents the 


I like putting the "what" first before diving into the example, but let me know what you think!

Suggested change

The following example is a representation of the human readable output that `terraform modules` returns. The nested structure represents the

The `terraform modules` command returns a nested structure, representing module composition and hierarchy within your configuration.

The following example demonstrates what the `terraform modules` command returns without any additional flags:

(make sure to remove the line below if you accept this suggestion!

Included 👍 e2c5b39

uturunku1

Great job!

One inconsistency I was noticing between this cmd and other similar cmds that have have a json view is that when we pass the wrong argument for "-json", then we get different types of outputs. See picture:

I think the most accurate output from those 3 is the one for the "schema" cmd. The one for "show" is completely off, and the one for "modules" is slightly deceiving. Though the good thing is that it mentions that you can run "-help" and that is how the user can figure out that the proper argument is "-json".

But I was curious if you have a suggestion for a better error we could output instead "Too many command line arguments". This is not a blocker by the way. Aside from this question, I am happy with the output you have implemented, and I really like how the tree presents nested modules.

internal/moduleref/record.go

internal/command/views/modules.go

sebasslash

Looking good, but found a bug while testing ⬇️

internal/moduleref/resolver.go

internal/command/views/modules.go

sebasslash

Tested the changes, looking great now. I would address some of @rkoron007 comments on docs as well before merging.

sebasslash · 2024-12-06T21:30:56Z

internal/moduleref/resolver.go

+func (r *Resolver) findAndTrimReferencedEntries(cfg *configs.Config, parentRecord *Record, parentKey *string) {
+	var name string
+	var versionConstraints version.Constraints
+	if parentKey != nil {


parentKey does not need to be a pointer, we can safely check if the string is empty since no module except the root module will have an empty key.

sebasslash

Re-approving, great work here 👍

rkoron007

Looks great! Thank you for all your work on this!

rkoron007 · 2024-12-10T20:50:13Z

website/docs/cli/commands/modules.mdx

+    └── "my_local_module_b"[./path/to/local/module_a/module_b]
+        └── "my_local_module_c"[./path/to/local/module/module_a/module_b/module_c]
+```
+
 The following example is a representation of the JSON output format that `terraform modules -json` returns.

 ```javascript


I think you can just use json if this doesn't specifically need to be JavaScript:

Suggested change

```javascript

```json

radeksimko

LGTM on the basis that the human-readable output is not covered by compatibility guarantees, so my questions can be addressed in a separate PR at any time later.

radeksimko · 2024-12-16T10:15:33Z

internal/command/views/modules.go

+			// Avoid rendering the version constraint if an exact version is given i.e. 'version = "1.2.3"'
+			if childNode.VersionConstraints != nil && childNode.VersionConstraints.String() != childNode.Version.String() {
+				item += fmt.Sprintf(" (%s)", childNode.VersionConstraints.String())
+			}


What is the motivation behind this? I'd assume that version constraints will never be so long that they would worsen the readability but correct me if I'm wrong.

It's more so to handle the case where the version constraint is specified to the exact version which would lead to a bit of redundancy– such as "mod-key' [mod-source] 1.0.0 (1.0.0). Felt that is was simpler to just omit the constraint in this case as it isn't clear that it is a version constraint. Though it might have made more sense to just append a "=" here to make the constraint more clear.

I don't think it's a redundancy because a constraint is a different thing to the version actually installed. Relatedly, there can be many semantically equal version constraints which are represented by the same canonical format via String().

For example:

1.0.0

=1.0.0

= 1.0.0

These are all equivalent but I'm not sure they would result in the same behaviour as above, which would further confuse users.

I would just keep it simple and always render it.

radeksimko · 2024-12-16T10:17:28Z

internal/command/views/modules.go

 	return 0
 }

+func populateTreeNode(tree treeprint.Tree, node *moduleref.Record) {
+	for _, childNode := range node.Children {
+		item := fmt.Sprintf("\"%s\"[%s]", childNode.Key, childNode.Source.String())


I'd have expected %q instead of \"%s\" but I'm more curious if there is a reason for quoting the module key in the human output at all? 🤔

If I understand the code correctly I don't think this will fail, but %q should still be used to produce correctly quoted strings. (BTW you don't need to call String() for string formatting, that is what the fmt.Stringer interface does for you).

github-actions · 2024-12-16T15:34:38Z

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

jbardin

Sorry, just dropping in couple late comments, but nothing that can't be fixed up later.

jbardin · 2024-12-16T18:31:09Z

internal/command/views/modules.go

 	return 0
 }

+func populateTreeNode(tree treeprint.Tree, node *moduleref.Record) {
+	for _, childNode := range node.Children {
+		item := fmt.Sprintf("\"%s\"[%s]", childNode.Key, childNode.Source.String())


If I understand the code correctly I don't think this will fail, but %q should still be used to produce correctly quoted strings. (BTW you don't need to call String() for string formatting, that is what the fmt.Stringer interface does for you).

jbardin · 2024-12-16T18:43:44Z

internal/moduleref/resolver.go

+	}
+	key := strings.Join(cfg.Path, ".")
+
+	for entryKey, entry := range r.internalManifest {


Generating ordered lists of values from a map like this is going to produce inconsistent output. While it may be that the order doesn't matter, users will expect to see it presented consistently, so we typically will sort the slices before presentation.

Maed223 force-pushed the TF-19313/modules-cmd-human-view branch from 0778f7a to 2f7081c Compare November 19, 2024 15:43

vercel bot deployed to Preview November 19, 2024 15:48 View deployment

Maed223 force-pushed the TF-19313/modules-cmd-human-view branch from 2f7081c to 272130e Compare November 19, 2024 16:07

Maed223 changed the title ~~Tf 19313/modules cmd human view~~ terraform modules cmd human view Nov 19, 2024

Maed223 marked this pull request as ready for review November 19, 2024 16:09

Maed223 requested review from a team as code owners November 19, 2024 16:09

vercel bot deployed to Preview November 19, 2024 16:13 View deployment

rkoron007 previously approved these changes Nov 19, 2024

View reviewed changes

website/docs/cli/commands/modules.mdx Show resolved Hide resolved

Maed223 dismissed rkoron007’s stale review via 4cba6bb November 20, 2024 22:16

Maed223 force-pushed the TF-19313/modules-cmd-human-view branch from 272130e to 4cba6bb Compare November 20, 2024 22:16

vercel bot deployed to Preview November 20, 2024 22:23 View deployment

Maed223 added 3 commits November 21, 2024 12:54

Add human view for modules cmd

8d0db63

Testing updates

b2a90cf

Docs update for modules cmd human view

e4b4aa8

Maed223 force-pushed the TF-19313/modules-cmd-human-view branch from 4cba6bb to e4b4aa8 Compare November 21, 2024 17:54

vercel bot deployed to Preview November 21, 2024 18:00 View deployment

Maed223 requested a review from rkoron007 November 21, 2024 18:14

rkoron007 previously approved these changes Nov 21, 2024

View reviewed changes

uturunku1 self-requested a review November 22, 2024 21:29

uturunku1 reviewed Nov 22, 2024

View reviewed changes

internal/moduleref/record.go Show resolved Hide resolved

internal/command/views/modules.go Outdated Show resolved Hide resolved

sebasslash requested changes Nov 26, 2024

View reviewed changes

internal/moduleref/resolver.go Outdated Show resolved Hide resolved

internal/command/views/modules.go Outdated Show resolved Hide resolved

Maed223 added 2 commits December 4, 2024 11:43

Use mod key to match module

40d1628

align modules cmd with formating seen in providers cmd

9a3cedb

Maed223 dismissed rkoron007’s stale review via 9a3cedb December 5, 2024 18:12

vercel bot deployed to Preview December 5, 2024 18:18 View deployment

sebasslash previously approved these changes Dec 6, 2024

View reviewed changes

some tweaks to the docs

e2c5b39

Maed223 dismissed sebasslash’s stale review via e2c5b39 December 9, 2024 15:11

Maed223 requested a review from rkoron007 December 9, 2024 15:12

vercel bot deployed to Preview December 9, 2024 15:16 View deployment

sebasslash approved these changes Dec 9, 2024

View reviewed changes

rkoron007 approved these changes Dec 10, 2024

View reviewed changes

radeksimko approved these changes Dec 16, 2024

View reviewed changes

Maed223 merged commit e6bd282 into main Dec 16, 2024
8 checks passed

Maed223 deleted the TF-19313/modules-cmd-human-view branch December 16, 2024 15:34

jbardin reviewed Dec 16, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`terraform modules` cmd human view #36062

`terraform modules` cmd human view #36062

Maed223 commented Nov 19, 2024 •

edited

Loading

github-actions bot commented Nov 19, 2024

github-actions bot commented Nov 19, 2024

rkoron007 left a comment

github-actions bot commented Nov 20, 2024

rkoron007 left a comment

rkoron007 Nov 21, 2024

Maed223 Dec 9, 2024

rkoron007 Nov 21, 2024

rkoron007 Nov 21, 2024

Maed223 Dec 9, 2024

uturunku1 left a comment

sebasslash left a comment

sebasslash left a comment

sebasslash Dec 6, 2024

sebasslash left a comment

rkoron007 left a comment

rkoron007 Dec 10, 2024

radeksimko left a comment

radeksimko Dec 16, 2024

Maed223 Dec 16, 2024

radeksimko Dec 16, 2024

radeksimko Dec 16, 2024

jbardin Dec 16, 2024

github-actions bot commented Dec 16, 2024

jbardin left a comment

jbardin Dec 16, 2024

jbardin Dec 16, 2024

-The following example is a representation of the human readable output that `terraform modules` returns. The nested structure represents the
+The `terraform modules` command returns a nested structure, representing module composition and hierarchy within your configuration.
+The following example demonstrates what the `terraform modules` command returns without any additional flags:

terraform modules cmd human view #36062

terraform modules cmd human view #36062

Conversation

Maed223 commented Nov 19, 2024 • edited Loading

Previous context on the modules command

Target Release

Draft CHANGELOG entry

NEW FEATURES

github-actions bot commented Nov 19, 2024

github-actions bot commented Nov 19, 2024

rkoron007 left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 20, 2024

rkoron007 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

uturunku1 left a comment

Choose a reason for hiding this comment

sebasslash left a comment

Choose a reason for hiding this comment

sebasslash left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebasslash left a comment

Choose a reason for hiding this comment

rkoron007 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radeksimko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Dec 16, 2024

jbardin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

`terraform modules` cmd human view #36062

`terraform modules` cmd human view #36062

Maed223 commented Nov 19, 2024 •

edited

Loading

Previous context on the `modules` command