RFC Platforms

[arlimus]

v0.4-2017.04.24 Complete version for now; not covering custom transport targets in this RFC
work items are tracked below in #1661 (comment) and will be moved into a milestone

1. Summary

This RFC describes InSpec's platform definition. Platforms are the sum of all the things that InSpec can test. It is designed to support all types of platforms, from operating systems, network equipment, to APIs, databases, and cloud systems.

This is not a breaking change. This RFC contains a detailed explanation of goals and supported fields.

Platforms in InSpec are fundamentally separated in operating systems (OS) and application interfaces (APIs). [See section 2]

Platforms are used in platform definitions and resources to declare what these can support. [See sections 3 + 4]

2. Platform definition and list

Platforms describe the things that InSpec can test. There are specific platforms (e.g. Red Hat Linux 7.0) and more general definitions that act as a grouping mechanism (e.g. Linux OS). Platforms are overall grouped in operating systems and APIs. Operating systems are built on the file, command, and os resources. APIs provide common tests around requests on different layers (e.g. testing SSL and HTTP).

Every platform is uniquely identified by the combination of name and release. A specific platform may have a number of platform family entries associated with it. (See section 2.2)

A core goal of this RFC is the ability to scan e.g. HTTP endpoints, APIs, SSL, and more without being disconnected from the scan-target(1). At the moment the details are hidden in code while the profile is executed on a host.

(1) At the moment a user may write a profile that targets a remote host (--target ssh://something) but really opens a connection to test an API (like AWS/Azure/VMware). Disconnected points to the difference in scan-target and actually tested object.

2.1 Naming conventions

InSpec's platform types are oriented towards Serverspec compatibility. This provides both simplicity in the OS world, as well as a few limitations and differences in convention to other tools. As for limitations: Shorthand specifiers like redhat would have a better replacement with rhel, considering there is an OS group called redhat and other products in this space like redhat-openstack.

A long-term transitional goal is to (1) introduce clear identifiers for new platforms, (2) add new identifiers to platforms that are loose and start to deprecate old identifiers, (3) remove old identifiers. This RFC will not deprecate or remove identifiers yet (steps 2+3) and only introduce new terms (step 1).

2.2 Platform definition

Platforms are mainly defined in Train. API platforms may be extended beyond that in InSpec.

Platform definitions are done via:

Train::Platforms.name('platform-identifier'[, condition: ['...' | /.../] ])
             [ .title('platform-title') ]
             [ .in_family('platform-family') ]

Train::Platforms.family('family-identifier')
             [ .title('platform-title') ]
             [ .in_family('platform-family') ]

Examples:

Train::Platforms.name('centos').title('CentOS Linux').is_a('redhat')
Train::Platforms.family('redhat').in_family('linux')

Some platforms might change families on certain conditions:

Train::Platforms.name('backtrack').title('BackTrack Linux')
Train::Platforms.name('backtrack', release: '< 4').in_family('slackware')
Train::Platforms.name('backtrack', release: '>= 4').in_family('debian')

2.3 Platform detection

Platform detection may result in very complex tests for the OS layer. For this reason, most detection is handled in Train's OS layer. API's may go beyond Train's detection and provide detection mechanisms during the defintion layer.

Detection is handled via:

Train::Platforms.[name|family]('identifier')
               # run the detection if the above was found
               .detect {
                 ...
                 return nil # in case the detection fails
                 return PlatformInfo.new(name, release[, ...]) # if found
               }

This leaves room to define more control elements (e.g. before, after, and possibly priority in the future). An example for a custom API resource:

Train::Platforms.family('http')
               .detect {
                 c = @backend.request('get', '/version').response.body
                 j = JSON.load(c)
                 return true if j['name'] == 'myApp'
               }

If the platform is not detected, the Train::Exceptions::PlatformDetectionFailed will be raised.

2.4 Operating systems

The list of platform families with platform names is:

• redhat family: redhat, oracle, centos, amazon, scientific, xenserver, wrlinux
• debian family: debian, ubuntu, linuxmint, raspbian
• suse family: suse, opensuse
• bsd family: freebsd netbsd openbsd darwin
• solaris family: solaris smartos omnios openindiana opensolaris nexentacore
• windows family: windows

Additionally the following platform families are larger groups:

• linux family: linux, alpine, arch, coreos, exherbo, gentoo, slackware, redhat, debian, suse
• unix family: unix, aix, hpux, linux, solaris, bsd

All operating system platforms are part of the os platform family. More operating systems may be added. The underlying abstraction is handled by Train, specifically train-os. It abstracts file, command, and the os resources.

Note: A linux platform family does not contain another linux platform family, but rather the generic linux platform (if we detected Linux but don't know which one).

2.5 APIs

InSpec's API platforms are quickly evolving. To accommodate different platforms, profiles have the ability to define their platform support in terms of names and family.

The following is a list of initial platforms:

• http family: most APIs fall in this category, including all cloud products listed below
• ssl family: covering both ssl and tls in their various releases
• sql family: postgresql, mysql, oracle-sql, mssql
• tcp and udp families: to combine all low-level APIs

Specific product families:

• aws family
• azure family
• openstack family

3. Platform in profiles

Profiles can specify a list of supported platforms. This is done via the supports field and includes:

• os-name: for OS-specific platform names
• os-family: for OS-specific platform families
• platform: to target platform families or names
• platform-name: for specific platform names
• platform-family: for general platform families
• release: for the platform version

The support entries are combined via OR and each entry is made up of fields combined via AND. Example:

# This profile will support any version of BSD and RHELv6.
# i.e. Platform  matches  (family: bsd) OR (name: redhat AND release: 6)
- supports:
  - os-family: bsd
  - os-name: redhat
    release: 6

4. Platforms in resources

Resources can specify a list of supported platforms. This is done via the supports field. It behaves just like supports in inspec.yml (section above).

Due to the Ruby language, a few names needed adjustments: os_name, os_family, platform, platform_name, platform_family

Full example:

# This resource will support any version of BSD and RHELv6.
# i.e. Platform  matches  (family: bsd) OR (name: redhat AND release: 6)
class GordonConf < Inspec.resource(1)
  name 'gordon_config'
  supports os_family: 'bsd'
  supports os_name: 'redhat', release: 6

4.1 Resources in profiles

Resources may be used in profiles that don't support them. For example: The Windows resource registry_key may be ineffective in a profile that only supports redhat.

To detect this and help users, inspec check provides warnings about using resources in a profile whose platform definition does not support them. For example:

Warning: You are trying to use 1 resource that is only supported on Windows. This profile does not support Windows. Either remove the resource, extend it to support your platform, or add the platform to the list of supported systems in your "inspec.yml".
    Resource: kernel_parameter

If the profile does not have a list of supported systems defined, inspec check will provide a warning and recommendation for a list of supported systems:

Warning: This profile does not define a list of supported platforms. Based on the resources you use, this is a recommended setting for "inspec.yml":
    - supports:
      - platform-family: linux

5. The platform resource

To accommodate these platforms and provide information the new platform resource is introduced. It is governed by the Train library for both OSs and APIs.

The os resource will remain active but will only function for platforms that are part of the os platform family.

Resource specification:

platform.params   ->  name:       platform name
                      release:    platform release version
                      arch:       optional, applicable for OS
                      family:     optional, see section 2.1
                      [may include more fields depending on platforms]
                      [you can also access these fields directly e.g. platform.name]

platform.families ->  full list of all platform families that apply

The os resource mimics the platform resource.

6. Targeting platforms

InSpec profiles and interactive commands are targeted towards platforms, i.e. a profile is run with the purpose to test a platform. The mechanism for describing the target does not uniquely identify platform. It does provide some indication on the platform.

SSH example: Running InSpec on a remote host via SSH may see a target specified as ssh://hostname. This target may lead to multiple specific platforms that are tested, e.g. RedHat, Ubuntu, or even Solaris. Train helps to identify the platform that is described by the InSpec target. SSH will always describe an OS target. However, it also serves as a target for profiles testing the SSH platform, i.e. the application itself.

HTTP example: InSpec may be provided a remote HTTP/HTTPS endpoint as a target, via http://hostname:port. A multitude of platforms may be identified by this endpoint. It is part of the detection mechanism in Train and beyond to choose which resources and profiles apply.

6.1 Custom transports

Specific APIs may be defined by a custom transport target. These help in providing useful shorthands for shared functionality. For example, a cloud platform 'XYZ' (to stay vendor neutral) may define its target as xyz://accesskey:secret@region. These can be defined alongside platform defintions.

Custom transport targets will be defined in a separate RFC. Until then, please use existing Train transports for operating systems and APIs.

6.2 Train-API

The Train-API subsystem supports non-OS platforms targets. It provides the tcp, udp, and unix transports for low-level socket access, as well as http and https for high-level transport implementations.

These are defined via the URI schema:

mandatory minimum:
[tcp|upd|unix|http|https]://host

all options:
[tcp|upd|unix|http|https]://[user[:password]@]host[:port]][/path][?query][#fragment]

Some of these fields may not apply to certain transports. There may be resources that are based on the tcp transport but don't process a path or query parameter (e.g. for low-level interface testing).

The tcp, udp, and unix transports will all provide a low-level socket object with a basic communication interface (using an abstraction on top of the underlying Socket object). It is exposed to all InSpec resources via inspec.backend including all configuration options.

The http and https transports offer a simpler interface for all RESTful endpoints, exposed as the inspec.backend object to InSpec resources. Apart from the simple request it is also possible to use HTTP verbs (get, post, put, options, delete, head, trace).

7. InSpec platform utility

InSpec provides all platforms with grouping criteria via:

inspec schema platforms

Please note that this is a hidden command, as it is only required for dev integrations.

The schema provides all information on currently supported platforms and may go beyond this document.

Example output:

[
  // The platform name centos is in the redhat platform family
  { "name": "centos", "families": ["redhat"] },
  // The platform family aws is in the http platform family
  { "family": "aws", "families": ["http"] }
  ...
]

8. Annex / Q&A

Q: I see a redhat platform and a redhat platform family. Won't that lead to conflicts?
A: See InSpec's naming conventions, section 2.1. Confusion may arise from grouping platforms, e.g. the linux family includes redhat. The convention is to include platform families whenever available and platforms by name if not.

Q: Will the os resource be removed?
A: This RFC does not remove or deprecate any part of the current InSpec runtime. We may deprecate this resource in the future and appreciate more feedback to guide this decision.

Q: Will os-family, os-name, and os fields in inspec.yml be removed?
A: This RFC does not remove or deprecate any part of the current InSpec runtime. We may deprecate these fields in the future and appreciate more feedback to guide this decision.

Q: What happens if I use 2 conflicting resources in the same profile? For example: AWS and Azure?
A: This is supported both in a profile as well as through profile inheritance. InSpec will warn if the profile doesn't configure a list of supported platforms (see section 4.1). If a resource is used that only runs on a platform that is not in the list of supported platforms, users will receive a warning as well (section 4.1). This use-case is especially relevant if you consider e.g. scanning a Database, which is made up of on-system configuration as well as run-time tests and may (in the future) combine two targets.

[adamleff]

In section 2:

without being disconnected from the scan-target.

This could probably use some additional explanation as to what you're trying to convey.

[adamleff]

Section 2.3: I'd love to see us raise a specific exception if we can't detect the platform (maybe something like Train::Exceptions::PlatformDetectionFailed?) rather than returning nil.

[adamleff]

Sections 3 and 4: 💯 -- cannot wait for this level of control. Will provide a much better UX to both resource and profile creators as well as those that consume them.

[adamleff]

Thank you so much, @arlimus, for writing this up. This sounds great.

[aaronlippold]

I feel like this could also connect/link to the other discussion chain of knowing when you are inside a container, a vm or on a host - i.e. as part of the platform data that is returned - i.e. #1005.

[arlimus]

@adamleff Thank you for the feedback, all added into the current version!

This RFC is now complete. I removed the custom transport target definition (e.g. aws://...) for now as it requires a larger change in the next step.

[adamleff]

@arlimus LGTM 👍

[arlimus]

Starting to cover work items:

• create Train-API
• add train platform definition
• add train platform detection
• add platform keyword to Train and InSpec
• add platform fields in inspec.yml (definition and check)
• add supports keyword to InSpec resources (definition and check)
• add InSpec platform schema
• provide platform information in JSON formatter

[jquick]

Moving outstanding work here:
Train API - inspec/train#238
Add platforms to schema command - #2488