Chef Policyfile archive truncates cookbook file

[PowerSchill]

Description

When creating a policyfile archive using chef export -a on a policyfile that includes the Windows cookbook it will truncate the resources/share.rb file's last two lines. It only truncates it when creating the archive; Using chef export to create the directory does not have the issue.

If I remove the following lines: # Author:: Sölvi Páll Ásgeirsson (<solvip@gmail.com>) and # Copyright:: 2014-2017, Sölvi Páll Ásgeirsson. from the file, upload the cookbook, and regenerate the policyfile the lines are not truncated.

ChefDK Version

Chef Development Kit Version: 3.0.36

Platform Version

macOS Sierra 10.12.6

Replication Case

1. Create a policyfile that includes the Windows cookbook.
2. Create a policyfile archive using chef export -a
3. Extract the archive and examine the resources/share.rb file to verify the last two lines are not complete.

Expected results: Client executes policy.
Actual results: Error happens when executing policy on client.

Stacktrace

Please include the stacktrace.out output or link to a gist of it, if there is one.

[coderanger]

Wasn't able to repro it:

$ cat Policyfile.rb
name 'test'
default_source :supermarket
run_list 'windows'
$ chef install
Building policy test
Expanded run list: recipe[windows]
Caching Cookbooks...
Installing windows 4.3.2

Lockfile written to /Users/coderanger/src/tmp/cheftrunc/Policyfile.lock.json
Policy revision id: a856af71270c8eb34755a0f375cbfe3e22393293ca18b856be101c96ca6bad52
$ chef export -a vendor
Exported policy 'test' to /Users/coderanger/src/tmp/cheftrunc/vendor/test-a856af71270c8eb34755a0f375cbfe3e22393293ca18b856be101c96ca6bad52.tgz
$ tar xzvf vendor/test-a856af71270c8eb34755a0f375cbfe3e22393293ca18b856be101c96ca6bad52.tgz
x .
x .chef
x .chef/config.rb
x Policyfile.lock.json
x README.md
x cookbook_artifacts
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/CHANGELOG.md
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/CONTRIBUTING.md
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/README.md
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/powershell_helper.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/registry_helper.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/version.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/version_helper.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/windows_helper.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/windows_privileged.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/libraries/wmi_helper.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/metadata.json
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/providers
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/providers/dns.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/recipes
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/recipes/default.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/auto_run.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/certificate.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/certificate_binding.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/dns.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/feature.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/feature_dism.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/feature_powershell.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/font.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/http_acl.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/pagefile.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/path.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/printer.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/printer_port.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/share.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/shortcut.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/task.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/user_privilege.rb
x cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/zipfile.rb
x policies
x policies/test-a856af71270c8eb34755a0f375cbfe3e22393293ca18b856be101c96ca6bad52.json
x policy_groups
x policy_groups/local.json
$ tail cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/share.rb
    Chef::Log.debug("Running '#{revoke_command}' to revoke share permissions")
    powershell_out!(revoke_command)
  end

  # convert True/False into "$True" & "$False"
  def bool_string(bool)
    # bool ? 1 : 0
    bool ? '$true' : '$false'
  end
end

That content looks correct. Maybe the trigger is something more specific?

[PowerSchill]

Yeah, must be something special. I duplicated your setup and still get the same error.

tail cookbook_artifacts/windows-a8163f06b1cc66c849807ef0f2583e66a668a5f4/resources/share.rb
    revoke_command = "Revoke-SmbShareAccess -Name '#{new_resource.share_name}' -AccountName \"#{users.join(',')}\" -Force"
    Chef::Log.debug("Running '#{revoke_command}' to revoke share permissions")
    powershell_out!(revoke_command)
  end

  # convert True/False into "$True" & "$False"
  def bool_string(bool)
    # bool ? 1 : 0
    bool ? '$true' : '$false'
  en%

[coderanger]

Can you check if the cookbook is correct in your cache? Look in ~/.chefdk/cache/cookbooks. It would be ~/.chefdk/cache/cookbooks/windows-4.3.2-supermarket.chef.io/resources/share.rb for the latest version of the cookbook but you might have multiple in there.

[PowerSchill]

The cookbook is correct and if I do just an chef export it works. So it appears to be something with the tar'ing of the folder that is causing the issue.

[coderanger]

Can you upload the generated tarball somewhere? Might have some useful clues in it.

[PowerSchill]

Here it is: https://www.dropbox.com/s/q9mu08u35rut3zg/test-a856af71270c8eb34755a0f375cbfe3e22393293ca18b856be101c96ca6bad52.tgz?dl=0

[coderanger]

Tried unpacking that with multiple different Tar implementations and they all show the same corruption so almost certainly something in mixlib-archive. @thommay any ideas?

[fujin]

Confirming this - export -a generates the bogus json metadata file in the cookbook artifacts; while the directory mode works as expected.

Minimal repro case:

default_source :supermarket
name 'test'
cookbook 'acme'
run_list 'acme'

Generate two exported copies, one directory mode, one archive:

chef export test.rb ../test
chef export -a test.rb ../test2

Diff of resulting metadata.json:

diff -u ../test/cookbook_artifacts/acme-d837caba5824af07a895b8e313a07b2051a6ceef/metadata.json cookbook_artifacts/acme-d837caba5824af07a895b8e313a07b2051a6ceef/metadata.json
--- ../test/cookbook_artifacts/acme-d837caba5824af07a895b8e313a07b2051a6ceef/metadata.json      2018-06-20 15:25:16.000000000 -0700
+++ cookbook_artifacts/acme-d837caba5824af07a895b8e313a07b2051a6ceef/metadata.json      2018-06-20 15:25:22.000000000 -0700
@@ -36,4 +36,3 @@
   "gems": [

   ]
-}

[fujin]

Potentially related to non-ASCII character in the metadata.json:

ggrep --color='auto' -P -n "[^\x00-\x7F]"

Looks to be the multi-byte single quote https://raw.githubusercontent.com/schubergphilis/chef-acme/master/README.md right towards the end:

echo 'Let’s' | hexdump -c
0000000   L   e   t   200 231   s  \n
0000008

[coderanger]

Less terrible workaround, add this to your knife.rb:

require "mixlib/archive/lib_archive"
Mixlib::Archive::LibArchive.define_singleton_method(:new) { raise LoadError }

[amunoz951]

So I'm running into this same problem in MacOS but not in windows. In MacOS, if I export the archive, the metadata.json from the audit cookbook (v5.0.2) gets truncated but not if I export to a directory. In windows it works fine both ways. I tried adding the mixlib lines above to my knife.rb but that did not help. I'm running chefdk 3.1.0. There don't appear to be any special characters where it truncates (\nWITHOUT WARRANTIES OR CONDITIONS OF ANY). Any suggestions?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.