Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support multiple encoded blocks #127

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

support multiple encoded blocks #127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e78e766
support multiple encoded blocks
wershlak Oct 31, 2020
ee7fcb8
add missing newline
wershlak Nov 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 17 additions & 19 deletions lib/puppet-syntax/hiera.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,30 +42,28 @@ def check_eyaml_data(name, val)
end

def check_eyaml_blob(val)
return unless val =~ /^ENC\[/

val.sub!('ENC[', '')
# strip newlines and extra spaces
val.gsub!(/\n/, '')
val.gsub!(/\s+/, '')
if val !~ /\]$/
return "has unterminated eyaml value"
else
val.sub!(/\]$/, '')
method, base64 = val.split(/,/)
if base64 == nil
base64 = method
method = 'PKCS7'
end

encodes = val.scan(/ENC\[.*?\]/)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why .*? as a regex? The ? doesn't make sense to me. Also wondering why you don't drop the ENC part via groups:

[1] pry(main)> 'ENC[PKCS7,aGVsbG8sf&IHdvcmxk==]'.scan(/ENC\[(.+)\]/)
=> [["PKCS7,aGVsbG8sf&IHdvcmxk=="]]
Suggested change
encodes = val.scan(/ENC\[.*?\]/)
encodes = val.scan(/ENC\[(.*)\]/)

You can even already do the splitting here:

Suggested change
encodes = val.scan(/ENC\[.*?\]/)
encodes = val.scan(/ENC\[([^,]+),?(.+)?\]/)

Note I made the last part optional so you can still detect an invalid format.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/voxpupuli/hiera-eyaml/blob/master/lib/hiera/backend/eyaml_backend.rb#L101-L103

The ? didn't really make sense to me either but it seemed appropriate since that's what hiera-eyaml uses.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Looks like it was introduced in voxpupuli/hiera-eyaml@169ae64 where it previously was value.start_with?('ENC[').

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm all for improving it and my preference would be to do a scan once but I didn't find something that works

encodes = val.scan(/ENC\[([^,]+),?(.+)?\]/)

This fails the tests. Take the example ENC[KMS,aGVsbG8sIdGHdvcmxk==]ENC[KMS,aGVsbG8sIdGHdvcmxk==]

Gives us these match groups.

1. | KMS
2. | aGVsbG8sIdGHdvcmxk==]ENC[KMS,aGVsbG8sIdGHdvcmxk==

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for me:

data = 'ENC[KMS,aGVsbG8sIdGHdvcmxk==]ENC[KMS,aGVsbG8sIdGHdvcmxk==]'
data.scan(/ENC\[([^,]+),?([^\]]+)?\]/)

Returns [["KMS", "aGVsbG8sIdGHdvcmxk=="], ["KMS", "aGVsbG8sIdGHdvcmxk=="]]

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @wershlak @ekohl

Following-up your conversation about the question mark ? in the REGEX, it seems that we can remove it right from the upstream project, here is the PR to the change:

voxpupuli/hiera-eyaml#313

Thanks!


return "has unknown eyaml method #{method}" unless ['PKCS7','GPG','GKMS','KMS'].include? method
return "has unpadded or truncated base64 data" unless base64.length % 4 == 0
# Return if there's no encoded material
return if encodes.length == 0

# Base64#decode64 will silently ignore characters outside the alphabet,
# so we check resulting length of binary data instead
pad_length = base64.gsub(/[^=]/, '').length
if Base64.decode64(base64).length != base64.length * 3/4 - pad_length
return "has corrupt base64 data"
encodes.each do |n|
match = n.match(/ENC\[(.+),(.+)\]/)
if match == nil
return "has invalid eyaml encoded format"
end

return "has unknown eyaml method #{match[1]}" unless ['PKCS7','GPG','GKMS','KMS'].include? match[1]

return "has unpadded or truncated base64 data" unless match[2].length % 4 == 0

return "has corrupt base64 data" unless match[2].match?(/^[a-zA-Z0-9+\/=]+$/)
end
return
end

def check(filelist)
Expand Down
17 changes: 7 additions & 10 deletions spec/fixtures/hiera/hiera_bad.eyaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
acme::warning1: ENC[unknown-method,aGVsbG8sIHdvcmxk]
acme::warning2: ENC[PKCS7,aGVsbG8sIHdvcmxk
acme::warning3: ENC[PKCS7,aGVsbG8sIHdvcmxk==]
acme::warning4: ENC[PKCS7,aGVs!!!!bG8sIHdvcmxk]
acme::warning1: ENC[unknown-method,aGVsbG8sIHdvcmxk] # unknown method
acme::warning2: ENC[PKCS7,aGV&&&sbG8sIHdvcmxk] # unpadded or truncated base64 data
acme::warning3: ENC[PKCS7,aGVsbG8sIHdvcmxk==] # unpadded or truncated base64 data
acme::warning4: ENC[PKCS7,aGVsbG8sf&IHdvcmxk==] # corrupt base64 data
acme::warning5:
key1: foo
key2: ENC[PKCS7,aGVs!!!!bG8sIHdvcmxk]
key2: ENC[PKCS7,aGVs!!!!bG8sIHdvcmxk] # corrupt base64 data
acme::warning6:
hash_key:
- element1
Expand All @@ -17,8 +17,5 @@ acme::warning6:
ENC[PKCS7,
aGVs!!!!bG8sIHdvcmxk
]
acme::good1: >
ENC[PKCS7,
aGVsbG8sIHdvcmxk]
acme::good2: ENC[GPG,aGVsbG8sIHdvcmxkIQ==]
acme::good3: ENC[GPG,aGVsbG8sIHdvcmxkISE=]
# corrupt base64 data
acme::warning7: ENC[KMSaGVsbG8sIdGHdvcmxk==] # has invalid eyaml encoded format
9 changes: 9 additions & 0 deletions spec/fixtures/hiera/hiera_good.eyaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
acme::good1: ENC[KMS,aGVsbG8sIdGHdvcmxk==]
acme::good2: ENC[PKCS7,aGVsbG8sf3IHdvcmxk==]
acme::good3: ENC[KMS,aGVsbG8sIdGHdvcmxk==]ENC[KMS,aGVsbG8sIdGHdvcmxk==]
acme::good4: >
ENC[PKCS7,
aGVsbG8sIHdvcmxk]
acme::good5: ENC[GPG,aGVsbG8sIHdvcmxkIQ==]
acme::good6: ENC[GPG,aGVsbG8sIHdvcmxkISE=]
12 changes: 10 additions & 2 deletions spec/puppet-syntax/hiera_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@
expect(res).to be == []
end

it "should return nothing from valid EYAML" do
files = fixture_hiera('hiera_good.eyaml')
res = subject.check(files)
expect(res).to be == []
end


it "should return an error from invalid YAML" do
hiera_yaml = RUBY_VERSION =~ /1.8/ ? 'hiera_bad_18.yaml' : 'hiera_bad.yaml'
files = fixture_hiera(hiera_yaml)
Expand Down Expand Up @@ -45,19 +52,20 @@

it "should return warnings for bad eyaml values" do
hiera_yaml = 'hiera_bad.eyaml'
examples = 6
examples = 7
files = fixture_hiera(hiera_yaml)
res = subject.check(files)
(1..examples).each do |n|
expect(res).to include(/::warning#{n}/)
end
expect(res.size).to be == examples
expect(res[0]).to match('Key acme::warning1 has unknown eyaml method unknown-method')
expect(res[1]).to match('Key acme::warning2 has unterminated eyaml value')
expect(res[1]).to match('Key acme::warning2 has unpadded or truncated base64 data')
expect(res[2]).to match('Key acme::warning3 has unpadded or truncated base64 data')
expect(res[3]).to match('Key acme::warning4 has corrupt base64 data')
expect(res[4]).to match('Key acme::warning5\[\'key2\'\] has corrupt base64 data')
expect(res[5]).to match('Key acme::warning6\[\'hash_key\'\]\[2\] has corrupt base64 data')
expect(res[6]).to match('Key acme::warning7 has invalid eyaml encoded format')
end

it "should handle empty files" do
Expand Down