Consider the use of InSpec for BDD testing?

[anweiss]

Looks like Chef came out with a pretty slick open source compliance-as-code framework and DSL called InSpec (http://inspec.io/) which supports a number of OS's. This could be a nice complement to OpenControl in the form of BDD testing. Some ideas below:

• Tap in to InSpec by way of a Compliance Masonry plugin
• Include InSpec DSL resources alongside component.yaml definitions

[pburkholder]

+1 on doing research along those lines. Some random GitHub folks and I did some POC work around STIGS at https://github.com/inspec-stigs. It's remarkably easy to write new Inspec resources, and it doesn't require an agent -- just SSH or WinRM to the target system.

…

On Wed, Mar 15, 2017 at 11:05 AM, Andrew Weiss ***@***.***> wrote: Looks like Chef came out with a pretty slick compliance-as-code framework and DSL called InSpec (http://inspec.io/) which supports a number of OS's. This could be a nice complement to OpenControl in the form of BDD testing. Some ideas below: - Tap in to InSpec by way of a Compliance Masonry plugin - Include InSpec DSL resources along component.yaml definitions — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#19>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABJ7CkeyFzJMc7RvzFxTKaB-HiXsn7ooks5rl_5KgaJpZM4MeEhI> .

-- Peter Burkholder, Innovation Specialist -- 18F/GSA http://18f.gsa.gov peter.burkholder@gsa.gov / +1-202-709-2028 Calendar free/busy: https://calendar.google.com/calendar/embed?src=peter.burkholder%40gsa.gov "Improve the public’s experience with the government by helping agencies build, buy and share technology that allows them to become more accessible, efficient, and effective." -- GSA TTS Mission

[mogul]

It's remarkably easy to write new Inspec resources, and it doesn't require an agent -- just SSH or WinRM to the target system.

We previously talked about adding a "boundary" to the schema so that it's obvious where BDD-ish tests should be run. Then the boundary can be associated with its own test code which essentially acts as a pre-condition stanza. In BDD this would basically prepend "GIVEN a vantage point from which to run tests on components within boundary X" as a Background for all tests run on components inside that boundary, where that stanza maps to "get a shell/WinRM on host Y", where Y is essentially a bastion host within that boundary. (GIVENs that cover jumps to further hosts within that boundary would be associated with the component, not the boundary, to retain the ability to refactor and remix the hierarchy.) On Wed, Mar 15, 2017 at 9:39 AM, Peter Burkholder (@pburkholder) < notifications@github.com> wrote:

…

+1 on doing research along those lines. Some random GitHub folks and I did some POC work around STIGS at https://github.com/inspec-stigs. It's remarkably easy to write new Inspec resources, and it doesn't require an agent -- just SSH or WinRM to the target system. On Wed, Mar 15, 2017 at 11:05 AM, Andrew Weiss ***@***.***> wrote: > Looks like Chef came out with a pretty slick compliance-as-code framework > and DSL called InSpec (http://inspec.io/) which supports a number of > OS's. This could be a nice complement to OpenControl in the form of BDD > testing. Some ideas below: > > - Tap in to InSpec by way of a Compliance Masonry plugin > - Include InSpec DSL resources along component.yaml definitions > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#19>, or mute the thread > <https://github.com/notifications/unsubscribe- auth/ABJ7CkeyFzJMc7RvzFxTKaB-HiXsn7ooks5rl_5KgaJpZM4MeEhI> > . > -- Peter Burkholder, Innovation Specialist -- 18F/GSA http://18f.gsa.gov ***@***.*** / +1-202-709-2028 <(202)%20709-2028> Calendar free/busy: https://calendar.google.com/calendar/embed?src=peter.burkholder%40gsa.gov "Improve the public’s experience with the government by helping agencies build, buy and share technology that allows them to become more accessible, efficient, and effective." -- GSA TTS Mission — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC6kpZ7ol7ZM4XZkaDht4Zoxspo62cAks5rmBRUgaJpZM4MeEhI> .

[anweiss]

@mogul could you maybe provide a concrete example of what you've described? I'm looking through what was started with the cloud.gov compliance examples and came across this snippet from one of the component.yaml files:

...

verifications:
- key: POLICY_DOC
  name: Policy Document
  path: https://github.com/18F/compliance-docs/blob/master/AT-Policy.md
  type: URL
- description: "GIVEN the github link - <policy> THEN the policy has been updated\
    \ within the last 180 days \n"
  key: Policy_Update_Test
  last_run: 2016-04-07 13:25:17.456091
  name: 18F Policies Update
  path: BDD/policies.feature
  test_passed: false
  type: TEST

In essence, could we hook the verifications map in to compliance-masonry plugins for various auditing tools (e.g. InSpec, OpenSCAP, etc)?

[mogul]

I didn't write that snippet; I believe @geramirez (or maybe @jcscottiii) did, and would better be able to answer questions about it.

In general, yeah, the verifications were intended to give a pointer to automated verification implementation. (I believe the way this worked was that the test_passed and last_run YAML elements were actually filled in when the tests were run.) It should be made generic to whatever tools make sense; BDD/Gherkin is just where I was starting from and what I was familiar with.

[shawndwells]

No activity on this discussion for over a year. Closing for inactivity. Feel free to reopen as appropriate!

[git-ingham]

At a high level, InSpec looks a lot like SCAP. Admittedly, SCAP is complex, but by the time you meet a DISA stig with InSpec, I suspect that the complexity will be similar.

Not reopening, just adding a comment.