Duplicate testing when verifier specified in suite definition

[daxroc]

Description

Using inspec - TestKitchen, I’m seeing duplicate inspec executions for the included tests. Single suite with path defined to test/recipes/ containing two files with one control in each but see four controls executed in the kitchen test/verify output.

  - name: master
    run_list:
      - recipe[example::install_master]
      - recipe[example::configure_master]
    verifier:
      inspec_tests:
        - test/recipes/master

One observation I've noted is the duplicated paths

-----> Verifying <master-ubuntu-1604>...
D      Initialize InSpec
       Use `/Users/droche/work/infra-auto/cookbooks/example/test/recipes/master` for testing
D      Running tests from: ["/Users/droche/work/infra-auto/cookbooks/example/test/recipes/master", "test/recipes/master"]

Recreate

Create a stock cookbook using chef generate cookbook duptest
Create a suite directory under test/recipes/<suite> and move the default_test.rb into this directory

Output

kitchen verify def --log-level=debug
-----> Starting Kitchen (v1.11.1)
D      [Vagrant command] BEGIN (vagrant --version)
D      [Vagrant command] END (0m0.47s)
D      Berksfile found at /Users/droche/work/infra-auto/cookbooks/duptest/Berksfile, loading Berkshelf
D      Berkshelf 4.3.5 library loaded
-----> Verifying <default-ubuntu-1604>...
D      Initialize InSpec
       Use `/Users/droche/work/infra-auto/cookbooks/duptest/test/recipes/default` for testing
D      Running tests from: ["/Users/droche/work/infra-auto/cookbooks/duptest/test/recipes/default", "test/recipes"]

Target:  ssh://vagrant@127.0.0.1:2202

  ✔  User root should exist
  ✔  Port 80 should not be listening
  ○  User root should exist; User root This is an example test, r... (1 skipped)
     This is an example test, replace with your own test.
  ○  Port 80 should not be listening; Port 80 This is an example ... (1 skipped)
     This is an example test, replace with your own test.

Summary: 4 successful, 0 failures, 4 skipped
       Finished verifying <default-ubuntu-1604> (0m0.42s).
-----> Kitchen is finished. (0m1.78s)

Workaround

Removning the definition of the verifier removes the duplicate executions when the test/recipes/<suite> directory is used.