Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hclwrite: Allow updating block type and labels #340

Merged
merged 3 commits into from
Aug 21, 2020

Conversation

minamijoyo
Copy link
Contributor

Fixes #338

Add methods to update block type and labels to enable us to refactor HCL configurations such as renaming Terraform resources.

  • *Block.SetType(typeName string)
  • *Block.SetLabels(labels []string)

Some additional notes about SetLabels:
Since we cannot assume that old and new labels are equal in length, remove old labels and insert new ones before TokenOBrace.

To implement this, I also added the following methods.

  • *nodes.Insert(pos *node, c nodeContent) *node
  • *nodes.InsertNode(pos *node, n *node) *node

They are similar to the existing Append / AppendNode, but insert a node before a given position.

While implementing Block.SetLabels(), I found a new hclwrite parser bug.

The NewBlock() method records positions of TokenOBrace / TokenCBrace.
Nevertheless when generating blocks via hclwrite.ParseConfig(), they were not recorded.

The position of TokenOBrace is needed for Block.SetLabels(), so I also fixed this existing bug.

@apparentlymart
Copy link
Contributor

Hi @minamijoyo! Thanks for working on this, and sorry for the slow response. (I was on a vacation.)

Overall this looks good to me! I have some thoughts about the new Insert and InsertNodes methods, though.

The existing strategy for inserting tokens into the middle of a construct is for the insertion area to be represented by a separate child *node. We can see this in the existing code in the Block.body field: all of the tokens representing the portion between the two braces are separated into a child node, and then when we want to append new items to the body we can append their tokens to that child node.

When hclwrite prepares the final result (in hclwrite.File.Bytes) it walks the whole tree and collects all of the tokens in traversal order, and the node representing the block itself has the Block.body node before the Block.close node, and so the tokens are assembled in the correct order.

I think we could take a similar approach with the labels here. Currently labels are not a separate node but are rather just a nodeSet of various nodes that belong directly to the block's node. But if we were to change this internal structure so that labels is of type *node, just like body, then we could implement adding and removing labels the same way as we implement adding and removing body items, similar to how Body.Clear and Body.AppendBlock work.

I don't mean to suggest adding a new public type Labels to the package, though: instead I think the clearing and appending to the new *node representing all of the labels together would happen in the implementation of the SetLabels function, so that the external-facing interface is always to read and write all of the labels together.

What do you think about that? I would prefer to use this strategy because it is consistent with the solution to similar problems in this package, but I wonder if you already considered this approach and found a problem with it that I have not noticed.

Thanks again for working on this!

@minamijoyo
Copy link
Contributor Author

@apparentlymart Thank you for your reply.

To be honest, I implemented it simply referring to the existing implementation, so I didn't come up with changing the type of labels from nodeSet to *node. However, it sounds good to me for modeling the labels as a new node type. In general, it's a so-called first class collection pattern, and elements of block other than labels are already represented as a node. I think it's easier to handle labels if they are grouped as a node.
I'm not sure what will cause other matters in the case of the labels. I don't have much knowledge about the hcl internals, sorry... What can I do to move this pull request forward?

@minamijoyo
Copy link
Contributor Author

@apparentlymart I would appreciate if you could review it.
Please let me know if there is anything I can help you with.

minamijoyo and others added 3 commits August 21, 2020 09:21
Fixes hashicorp#338

Add methods to update block type and labels to enable us to refactor HCL
configurations such as renaming Terraform resources.

- `*Block.SetType(typeName string)`
- `*Block.SetLabels(labels []string)`

Some additional notes about SetLabels:

Since we cannot assume that old and new labels are equal in length,
remove old labels and insert new ones before TokenOBrace.

To implement this, I also added the following methods.

- `*nodes.Insert(pos *node, c nodeContent) *node`
- `*nodes.InsertNode(pos *node, n *node) *node`

They are similar to the existing Append / AppendNode,
but insert a node before a given position.
… in parser

While implementing Block.SetLabels(), I found a new hclwrite parser bug.

The NewBlock() method records positions of TokenOBrace / TokenCBrace.
Nevertheless when generating blocks via hclwrite.ParseConfig(),
they were not recorded.

The position of TokenOBrace is needed for Block.SetLabels(),
so I also fixed this existing bug.
All of the other subdivisions of a block were already nodes, but we'd
represented the labels as an undifferentiated set of nodes belonging
directly to the block's child node list.

Now that we support replacing the labels in the public API, that's a good
excuse to refactor this slightly to make the labels their own node. As
well as being consistent with everything else in Block, this also makes
it easier to implement the Block.SetLabels operation because we can
just  change the children of the labels node, rather than having to
carefully identify and extract the individual child nodes of the block
that happen to represent labels.

Internally this models the labels in a similar sort of way as the content
of a body, although we've kept the public API directly on the Block type
here because that's a more straightforward model for the use-cases we
currently know and matches better with the API of hcl.Block. This is just
an internal change for consistency.

I also added a few tests for having comments interspersed with labels
while I was here, because that helped to better exercise the new
parseBlockLabels function.
@apparentlymart
Copy link
Contributor

Hi @minamijoyo! Sorry for the very long delay in replying to you here. Unfortunately I had some unexpected high-priority projects in the meantime that meant I couldn't spend time on HCL.

Your changes here look great to me, including all of the tests which made it very easy to review. The (pre-existing) inconsistency with Block.labels being a nodeSet instead of a node was bothering me, so I've pushed a further commit here to refactor that to be a node like I was describing in my earlier comment, but the core functionality here is still what you implemented, just arranged slightly differently.

I'm going to merge this now. I'm going to wait to make a new HCL release because we may merge some other changes before tagging, but it should be in a tagged release soon.

Thanks again!

@apparentlymart apparentlymart merged commit 7d8f0ff into hashicorp:hcl2 Aug 21, 2020
@minamijoyo
Copy link
Contributor Author

@apparentlymart Thank you for your support and refactoring 🎉 🎉 🎉

minamijoyo added a commit to minamijoyo/hcledit that referenced this pull request Aug 22, 2020
Previously the block mv command depends on the forked version of the
hclwrite package. A good news is the patch have been merged into
upstream. So we no longer need the forked version, and simply use it
with upstream.
hashicorp/hcl#340
bflad added a commit that referenced this pull request May 23, 2024
Reference: #340
Reference: #680

Similar to `(*hclwrite.Block).SetType()`, this new method enables in-place renaming of an attribute.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants