Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new Configuration object to support flexible expression of configuration details for tools and software #432

Closed
11 tasks done
sbarnum opened this issue Aug 4, 2022 · 19 comments · Fixed by #455 or #477
Closed
11 tasks done

Comments

@sbarnum
Copy link
Contributor

sbarnum commented Aug 4, 2022

Background

This change proposal is an extended revision to CP-405 and should completely replace it.

Need to represent multiple different configurations of the same tool, each referencing the same Tool object. Knowing what tool was run for each different configuration should be represented as a link in the graph, and not rely on the string name value. In addition, tool configuration needs an added property to reference an object that was provided as an argument/input to the Tool (e.g. so you could have a graph-object for NSRL-as-of-March-2022 or ClamAV signature-database version X, and show where in the graph that object had been involved in decision processes).

The need to express configuration details is not unique to Tool.
It is equally relevant to observable:Software.

A flexible and consistent approach is needed to specify configuration details (including literal key/value pair settings as well as more complex object values) in a way that supports both tool and software user cases.
Such an approach should seek to maximize brevity, consistency, and flexiblity while minimizing duplication throughout the UCO ontology.

Requirements

Requirement 1

UCO must be able to represent how a tool or software was configured at the time of its invocation.

Requirement 2

A tool or software configuration that uses an object (such as a file or service endpoint) must be representable similar to the literal-valued tool:itemValue.

Requirement 3

UCO must be able to represent that one too or softwarel was used under multiple configurations.

Requirement 4

UCO must be able to use one tool or software configuration in multiple invocations.
E.g. if a tool analyzer_engine has a configuration involving calling it on the command line with the flag --auto-detect-content-type, and the tool is called once per input file for two input files, those two tool invocations should have their action:instrument objects use the same ConfiguredTool.

Requirement 5

UCO must support looking up when a tool or software is used with a certain flag.
E.g. if a tool analyzer_engine can take the unparameterized (0-argument) command-line flag --auto-detect-content-type, a UCO graph should be reviewable for when analyzer_engine is configured to use that flag, and in what action:Actions that configuration was used.

Requirement 6

UCO must support looking up when a tool or software is used with a certain flag-value. This applies to object-values (such as a file or service end-point reference), and literal-values (such as a string).
E.g. if a tool analyzer_engine can take the parameterized (1-argument) command-line flag --assume-content-type, a UCO graph should be reviewable for when analyzer_engine is configured to use that flag, and in what action:Actions that configuration was used.

Requirement 7

One tool used in multiple configurations should be linkable between those configurations and the actions using those configurations.

Risk / Benefit analysis

Benefits

Permit flexible representation of various configurations of tools and software.
Permit representation of multiple different configurations of the same tool or software, while maintaining link to the same Tool or Software

Risks

The submitter is unaware of risks associated with this change.

Competencies demonstrated

Competency 1

When a single tool or software is used in a workflow with multiple configuration sections, determine which Tool was used to obtain all observables, independent of all different tool configurations used.

Competency Question 1.1

What Tool or Software was used to obtain all observables?

Result 1.1

Query returns a single result Tool or Software.

Competency 2

A version of a tool or software known to contain a defect or vulnerability should be locatable within a graph, to understand the impact of results.

Competency Question 2.1

What Tool or Software was used to obtain all observables?

Result 2.1

Query returns all results affected by the defect or vulnerability.

Competency Question 2.2

A Software version 1.0 is known to have a vulnerability. What related devices or systems may be vulnerable do to the Software under any configuration.

Result 2.2

Query returns all related objects that may be vulnerable due to their relationship with the affected Software.

Competency 3

When different versions of a resource (e.g., NSRL hashset, AntiVirus database) were provided in various Tool Configuration objects, determine which resource was provided as an argument to a given Tool Configuration object.

Competency Question 3.1

What resource was referenced in the itemObject property of a given Tool Configuration object?

Result 3.1

Query returns a single object that was referenced in the itemObject of the given Tool Configuration object.

Solution suggestion

  • Additions

    • Create new configuration namespace
    • Create new configuration:Configuration class as a subClassOf core:UcoObject
    • Create property shape for configuration:configurationEntry on configuration:Configuration with minCount=1
    • Create new configuration:itemObject property with range core:ClassBase (see CP 430)
    • Create new property shape for configuration:itemObject on configuration:ConfigurationEntry
  • Modifcations

    • Move tool:dependencies property from tool namespace to configuration namespace and rename to configuration:dependencies
    • Move tool:dependencies property shape from tool:ToolConfigurationTypeFacet class to configuration:Configuration class
    • Move tool:DependencyType class from tool namespace to configuration namespace and rename to configuration:Dependency
    • Move tool:dependencyDescription property from tool namespace to configuration namespace and rename to configuration:dependencyDescription
    • Move tool:dependencyType property from tool namespace to configuration namespace and rename to configuration:dependencyType
    • Move tool:usageContextAssumptions property from namespace class to configuration namespace and rename to configuration:usageContextAssumptions
    • Move tool:usageContextAssumptions property shape from tool:ToolConfigurationTypeFacet class to configuration:Configuration class
    • Move tool:ConfigurationSettingType from tool namespace to configuration namespace and rename to configuration:ConfigurationEntry
    • Move tool:itemName property from tool namespace to configuration namespace and rename to configuration:itemName
    • Move tool:itemDescription property from tool namespace to configuration namespace and rename to configuration:itemDescription
    • Move tool:itemValue property from tool namespace to configuration namespace and rename to configuration:itemValue
    • Move tool:itemType property from tool namespace to configuration namespace and rename to configuration:itemType
    • Move tool:configurationSettings property from tool namespace to configuration namespace and rename to configuration:configurationEntry
      • Modify its range to be configuration:ConfigurationEntry
    • Modify range of tool:buildConfiguration from tool:BuildConfigurationType to configuration:Configuration
    • Modify property shape for tool:buildConfiguration on tool:BuildInformationType to specify "sh:class configuration:Configuration" rather than "tool:BuildConfigurationType"
    • Modify all class and property definitions to abstract from just being about tools
  • Deletions

    • Remove tool:BuildConfigurationType
    • Remove tool:configurationSettingDescription
    • Remove tool:ToolConfigurationTypeFacet

Examples

Example of an Action using a Tool with a Configuration

{
  "@graph": [
    {
      "@id": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "@type": "uco-tool:AnalyticTool",
      "uco-core:name": "Hansken",
      "uco-tool:toolType": "DFaaS",
      "uco-tool:creator": "NFI",
      "uco-tool:version": "1.0"
    },
    {
      "@id": "kb:relationship-d6d18770-cb60-4779-b28a-905846dbd3f4",
      "@type": "uco-core:Relationship",
      "uco-core:source": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "uco-core:target": "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92",
      "uco-core:kindOfRelationship": "Has_Configuration",
      "uco-core:startTime": "2010-01-15T17:59:43.25Z",
      "uco-core:isDirectional": true
    },
    {
      "@id": "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92",
      "@type": "uco-configuration:Configuration",
      "uco-configuration:configurationEntry": [
          {
            "@id": "kb:ConfigurationEntry-bff6199a-dc44-40c7-b268-92bc32a7d0e1",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "classifier",
            "uco-tool:itemValue": "nfi-forensic"
          },
          {
            "@id": "kb:ConfigurationEntry-6bb9afaa-4cb0-4428-aa18-bdd197ddc8af",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "TrainingSet",
            "uco-tool:itemValue": "0.0.7"
          },
          {
            "@id": "kb:ConfigurationEntry-e6b4d8fb-756b-4eed-8cb7-296609614d41",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "Configuration File",
            "uco-tool:itemObject": "kb:File-1ac3c11e-b6af-49c3-bcc5-5ba61635398e"
          }
      ]
    },
    {
      "@id": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "@type": "uco-analysis:Analysis",
      "uco-core:name": "compute string similarity",
      "uco-core:startTime": "2022-05-10T08:49:00.00Z",
      "uco-core:endTime": "2022-05-10T09:54:00.00Z",
      "uco-action:isAutomated": true,
      "uco-action:location": "kb:ESC-6FAC81EF-0966-4F05-94BB-2A5D572513CA",
      "uco-action:performer": "kb:Analyst-13A167EE-D3B5-4AA4-B8BA-83C25F8B8FF4",
      "uco-action:instrument": [
        "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
        "kb:relationship-d6d18770-cb60-4779-b28a-905846dbd3f4",
        "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92"
      ],
      "uco-action:environment": "kb:Computer-533FA61A-BE79-469E-A05F-1A341848B925",
      "uco-action:object": [
        "kb:RasterPicture1-b67308c0-c31b-41a6-805a-10ec526ec8bc"
      ],
      "uco-action:result": [
        "kb:ProvenanceRecord-d628b0f6-686d-4d22-a577-ec737e5947bc",
        "kb:AnalyticResult-3205CB19-0820-4009-B70B-646DBD19598B"
      ]
    },
    {
      "@id": "kb:AnalyticResult-3205CB19-0820-4009-B70B-646DBD19598B",
      "@type": "analysis:AnalyticResult",
      "uco-analysis:originatingAnalysis": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "uco-core:hasFacet": [
        {
          "@id": "kb:ArtifactClassificationResultFacet-35d8d5e2-41bc-40d5-a7ea-8fcaddf1d646",
          "@type": "uco-analysis:ArtifactClassificationResultFacet",
          "uco-analysis:classification": [
            {
              "@id": "kb:ArtifactClassification-ef471242-ec3f-4aa4-b7aa-57b682c891c4",
              "@type": "uco-analysis:ArtifactClassification",
              "uco-analysis:class": "money",
              "uco-analysis:classificationConfidence": 0.997359037
            }
          ]
        }
      ]
    }
  ]
}

Example of Multiple Actions each using a Tool with different Configurations

{
  "@graph": [
    {
      "@id": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "@type": "uco-tool:AnalyticTool",
      "uco-core:name": "Hansken",
      "uco-tool:toolType": "DFaaS",
      "uco-tool:creator": "NFI",
      "uco-tool:version": "1.0"
    },
    {
      "@id": "kb:relationship-d6d18770-cb60-4779-b28a-905846dbd3f4",
      "@type": "uco-core:Relationship",
      "uco-core:source": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "uco-core:target": "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92",
      "uco-core:kindOfRelationship": "Has_Configuration",
      "uco-core:startTime": "2010-01-15T17:59:43.25Z",
      "uco-core:isDirectional": true
    },
    {
      "@id": "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92",
      "@type": "uco-configuration:Configuration",
      "uco-configuration:configurationEntry": [
          {
            "@id": "kb:ConfigurationEntry-bff6199a-dc44-40c7-b268-92bc32a7d0e1",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "classifier",
            "uco-tool:itemValue": "nfi-forensic"
          },
          {
            "@id": "kb:ConfigurationEntry-6bb9afaa-4cb0-4428-aa18-bdd197ddc8af",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "TrainingSet",
            "uco-tool:itemValue": "0.0.7"
          }
      ]
    },
    {
      "@id": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "@type": "uco-analysis:Analysis",
      "uco-core:name": "compute string similarity",
      "uco-core:startTime": "2022-05-10T08:49:00.00Z",
      "uco-core:endTime": "2022-05-10T09:54:00.00Z",
      "uco-action:isAutomated": true,
      "uco-action:location": "kb:ESC-6FAC81EF-0966-4F05-94BB-2A5D572513CA",
      "uco-action:performer": "kb:Analyst-13A167EE-D3B5-4AA4-B8BA-83C25F8B8FF4",
      "uco-action:instrument": [
        "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
        "kb:relationship-d6d18770-cb60-4779-b28a-905846dbd3f4",
        "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92"
      ],
      "uco-action:environment": "kb:Computer-533FA61A-BE79-469E-A05F-1A341848B925",
      "uco-action:object": [
        "kb:RasterPicture1-b67308c0-c31b-41a6-805a-10ec526ec8bc"
      ],
      "uco-action:result": [
        "kb:ProvenanceRecord-d628b0f6-686d-4d22-a577-ec737e5947bc",
        "kb:AnalyticResult-3205CB19-0820-4009-B70B-646DBD19598B"
      ]
    },
    {
      "@id": "kb:AnalyticResult-3205CB19-0820-4009-B70B-646DBD19598B",
      "@type": "analysis:AnalyticResult",
      "uco-analysis:originatingAnalysis": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "uco-core:hasFacet": [
        {
          "@id": "kb:ArtifactClassificationResultFacet-35d8d5e2-41bc-40d5-a7ea-8fcaddf1d646",
          "@type": "uco-analysis:ArtifactClassificationResultFacet",
          "uco-analysis:classification": [
            {
              "@id": "kb:ArtifactClassification-ef471242-ec3f-4aa4-b7aa-57b682c891c4",
              "@type": "uco-analysis:ArtifactClassification",
              "uco-analysis:class": "money",
              "uco-analysis:classificationConfidence": 0.997359037
            }
          ]
        }
      ]
    },
    {
      "@id": "kb:relationship-c6d98ad7-8714-4229-a8ee-c21f94ead315",
      "@type": "uco-core:Relationship",
      "uco-core:source": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "uco-core:target": "kb:Configuration-ec9cb6cf-a724-4e8b-95e2-f2a81c4e7073",
      "uco-core:kindOfRelationship": "Has_Configuration",
      "uco-core:startTime": "2014-01-15T17:59:43.25Z",
      "uco-core:isDirectional": true
    },
    {
      "@id": "kb:Configuration-ec9cb6cf-a724-4e8b-95e2-f2a81c4e7073",
      "@type": "uco-configuration:Configuration",
      "uco-configuration:configurationEntry": [
          {
            "@id": "kb:ConfigurationEntry-bff6199a-dc44-40c7-b268-92bc32a7d0e1",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "classifier",
            "uco-tool:itemValue": "nfi-forensic"
          },
          {
            "@id": "kb:ConfigurationEntry-6bb9afaa-4cb0-4428-aa18-bdd197ddc8af",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "TrainingSet",
            "uco-tool:itemValue": "0.0.9"
          }
      ]
    },
    {
      "@id": "kb:Analysis-8d982a24-96e7-4fa2-8b4d-7b6c8bdd7c1b",
      "@type": "uco-analysis:Analysis",
      "uco-core:name": "compute string similarity",
      "uco-core:startTime": "2022-05-11T10:00:00.00Z",
      "uco-core:endTime": "2022-05-11T11:04:00.00Z",
      "uco-action:isAutomated": true,
      "uco-action:location": "kb:ESC-6FAC81EF-0966-4F05-94BB-2A5D572513CA",
      "uco-action:performer": "kb:Analyst-13A167EE-D3B5-4AA4-B8BA-83C25F8B8FF4",
      "uco-action:instrument": [
        "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
        "kb:relationship-c6d98ad7-8714-4229-a8ee-c21f94ead315",
        "kb:Configuration-ec9cb6cf-a724-4e8b-95e2-f2a81c4e7073"
      ],
      "uco-action:environment": "kb:Computer-533FA61A-BE79-469E-A05F-1A341848B925",
      "uco-action:object": [
        "kb:RasterPicture1-b67308c0-c31b-41a6-805a-10ec526ec8bc"
      ],
      "uco-action:result": [
        "kb:ProvenanceRecord-0a370a22-fe76-48fb-af94-3c95f7f63287",
        "kb:AnalyticResult-b437c97e-31b8-4c43-9c4c-b39d8ab1c3c6"
      ]
    },
    {
      "@id": "kb:AnalyticResult-b437c97e-31b8-4c43-9c4c-b39d8ab1c3c6",
      "@type": "analysis:AnalyticResult",
      "uco-analysis:originatingAnalysis": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "uco-core:hasFacet": [
        {
          "@id": "kb:ArtifactClassificationResultFacet-35d8d5e2-41bc-40d5-a7ea-8fcaddf1d646",
          "@type": "uco-analysis:ArtifactClassificationResultFacet",
          "uco-analysis:classification": [
            {
              "@id": "kb:ArtifactClassification-ef471242-ec3f-4aa4-b7aa-57b682c891c4",
              "@type": "uco-analysis:ArtifactClassification",
              "uco-analysis:class": "money",
              "uco-analysis:classificationConfidence": 0.975184401
            }
          ]
        }
      ]
    }
  ]
}

Example of a Software with a Configuration

{
  "@graph": [
    {
      "@id": "kb:Software-335a7fce-1cc2-465b-b052-f6c3943f386b",
      "@type": "uco-observable:Software",
      "core:name": "Chrome",
      "core:hasFacet": [
        {
          "@id": "kb:SoftwareFacet-27e838bd-9f20-40f8-a0f2-37a765235f2b",
          "@type": "uco-observable:SoftwareFacet",
          "uco-observable:manufacturer": "kb:Organization-12ec98d5-7e53-496c-9890-549417366048",
          "uco-observable:version": "103.0.5060.134 (Official Build) (x86_64)"
        }
      ]
    },
    {
      "@id": "kb:relationship-aaf75bd8-b199-4074-8269-1910a3681349",
      "@type": "uco-core:Relationship",
      "uco-core:source": "kb:Software-335a7fce-1cc2-465b-b052-f6c3943f386b",
      "uco-core:target": "kb:Configuration-6d62667e-3c67-43e0-96d3-7ac635db9a06",
      "uco-core:kindOfRelationship": "Has_Configuration",
      "uco-core:startTime": "2022-08-04T13:16:21.00Z",
      "uco-core:isDirectional": true
    },
    {
      "@id": "kb:Configuration-6d62667e-3c67-43e0-96d3-7ac635db9a06",
      "@type": "uco-configuration:Configuration",
      "uco-configuration:configurationEntry": [
          {
            "@id": "kb:ConfigurationEntry-227315d1-3257-4c43-b126-c3854ee22b66",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "Font size",
            "uco-tool:itemValue": "Medium (recommended)"
          },
          {
            "@id": "kb:ConfigurationEntry-53ed9b3e-1cf4-4060-8888-c0de764f2d5c",
            "@type": "uco-configuration:ConfigurationEntry",
            "uco-tool:itemName": "Basic spell check",
            "uco-tool:itemValue": "On"
          }

      ]
    },
    {
      "@id": "kb:Organization-12ec98d5-7e53-496c-9890-549417366048",
      "@type": "uco-identity:Organization",
      "core:name": "Google"
    }
  ]
}

Coordination

  • Tracking in Jira ticket OC-259
  • Administrative review completed
  • Requirements to be discussed in OC meeting, 2022-08-16
  • Requirements Review vote occurred, passing, on 2022-08-16
  • Requirements development phase completed.
  • Solution announced to OCs on 2022-08-22
  • Solutions Approval to be discussed in OC meeting, 2022-08-25
  • Solutions Approval vote occurred, passing, on 2022-08-25
  • Solutions development phase completed.
  • Implementation merged into develop
  • Milestone linked
  • Documentation logged in pending release page
@ajnelson-nist
Copy link
Contributor

@sbarnum , this proposal complicates the review process. You were not the one who filed Issue 405, and this issue performs a few expansions - one to "union-expand" the applicability of the configuration from 405, from Tool to Tool or Software (which I'm pretty much fine with), but then also create an entire namespace devoted to effectively one object. I haven't gone through and parsed your semi-formal language description and I am still addressing other filed proposals, so I don't entirely know the extent of what your sketched solution entails.

Unless the proposer for 405 agrees to withdraw that proposal and let this supersede, we will continue with reviewing 405, and treat 432 as an independent follow-on proposal.

Rather than ask us to mentally parse your language into mental UML models, I encourage you to draft your proposal in a file drafting.ttl in this directory, to demonstrate its feasibility:

https://github.com/casework/CASE-Examples/tree/master/examples/illustrations/configured_tool

That directory contains a tested transcription of the example in your last comment from 405. If you run make after writing drafting.ttl and revising configured_tool.json, you'll see how well the draft implementation works out.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 15, 2022

Eoghan asked for us to try to combine efforts on this.
I believe that 405 falls short of what is necessary and that 432 covers all of 405 plus more. More that has been needed for a long while. As such I believe it makes sense to pursue 432 rather than 405 separately.
I will work to pull together technical implementation to hopefully clear up any lack of clarity or confusion.

@ajnelson-nist
Copy link
Contributor

@sbarnum , I agree at a high level that it would be better for 432 to be pursed. However, I don't have the bandwidth to test 432 in addition to some nuances I thought needed to be explored in 405.

405 has a ready-for-solutions-approval vote PR, 447.

If you can build an implementation yourself starting from 447's last commit as a baseline, I'm happy to consider Issue 432 a fast-track proposal.

@ajnelson-nist
Copy link
Contributor

@sbarnum - correction, this is now the most recent commit. I hadn't added definition comments or rdfs:labels as I was testing, but they're in that new commit now.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 15, 2022

I created a branch off of the commit you posted an hour ago and created updated drafting.ttl and configured_tool.json files to align with CP-432.

Unfortunately, git is not letting me push the branch back up.
I also tried attaching the two files here in case I cannot get git to play well but git does not want to allow ttl or json files.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 15, 2022

running make throws an error about rdf-toolkit.jar which I assume is likely something to do with my environment which is never stable (see issues above with git). :-(

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 15, 2022

I am going to try to branch and modify the files directly on github using browser and see if that works.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 15, 2022

I am crossing my mental streams between UCO ontology and CASE-Examples. :-(

I will simply create a branch of UCO ontology with implementation for 432

@ajnelson-nist
Copy link
Contributor

Re:

running make throws an error about rdf-toolkit.jar which I assume is likely something to do with my environment which is never stable (see issues above with git). :-(

Two candidate causes:

  1. Your Java environment is still Java 8. See if java -version tells you that you're on version 11 or newer. This is required in UCO's ecosystem since this issue.
  2. Your local UCO clone has the old rdf-toolkit.jar at lib/rdf-toolkit.jar. Basically, if the mtime is older than around 2 weeks ago, you should run these commands, and then run make again:
rm .lib.done.log
rm lib/rdf-toolkit.jar

make clean had purposefully avoided deleting rdf-toolkit.jar because it was assumed to be a stable-enough resource that it'd be more harm than good deleting it on every make clean. You've hit the once-in-about-3-years case where that might not have been true.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 15, 2022

This branch Feature-Issue-432 was created off your last commit above and contains implementation changes to tool.ttl and adds configuration.ttl

@plbt5
Copy link
Contributor

plbt5 commented Aug 16, 2022

I found this in literature that can be useful as (part of the) solution:
Product configuration knowledge modeling using ontology web language. Please have a look at how the authors define a general configuration ontology (Section 3.) that -or parts thereof- might be reused.

@ajnelson-nist ajnelson-nist added this to the UCO 1.0.0 milestone Aug 17, 2022
@ajnelson-nist ajnelson-nist linked a pull request Aug 18, 2022 that will close this issue
11 tasks
ajnelson-nist added a commit that referenced this issue Aug 21, 2022
This patch is a manual adjustment to apply UCO current ontology spelling
and linking practices.

* While the `rdfs:label` for the `owl:Ontology` includes the `uco-`
  prefix on the namespace short form, the prefix encoded in Turtle for
  concepts has not included the `uco-` prefix.
* `rdfs:seeAlso` has been used when drafting ontology concepts in files
  named `drafting.ttl`, as a part of CASE development practice, to tie
  the concept to its originating work-anchor (whether Jira ticket,
  Github Issue, or other).  However, by the time concepts migrate into
  the UCO or CASE ontology Turtle files, `rdfs:seeAlso` is not used to
  reference originating tickets.

The first issue caused a breakage in normalization, because the
`@prefix` declaration didn't quite agree with all of the UCO concept
references.  So, this patch applies the manual spelling changes, and a
follow-on patch will normalize the new `configuration.ttl` file.

References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 21, 2022
References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 21, 2022
References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 21, 2022
References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 21, 2022
Because Sean used seeAlso on a few other properties, I was able to catch
a point of development interest.  It seems we're moving towards more
fleshed-out citations, which I'm happy to support.

References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 21, 2022
This patch is motivated by several matters:
1. This class will be introduced (under another name) in the resolution
   of Issue 430.  So, entailing it in this proposal is a risk to this
   proposal's acceptance.
2. The semantics of using the higher-class-than-UcoObject are not
   defined.  Under what conditions is it OK to use a `types:Hash` object
   with this property?  What about a `types:Dictionary`,
   `types:ControlledDictionaryEntry`, or `types:ThreadItem`?

References to `core:ClassBase` will be restored (under its agreed-upon
name alteration, `core:UcoThing`) only after explanation and
demonstration of the semantic issues.

References:
* #430
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
An `sh:xone` pattern was introduced as part of Issue 405.  Sean removed
it when migrating concepts over to the new `configuration` namespace,
and neither documented why nor updated the unit test that expected it.

This patch restores the addition.

References:
* #405
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
This patch should restore CI passing.

A follow-on patch will regenerate Make-managed files.

References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
This keeps with the allowance made by Issue 428.

This also supports that some configurations may truly have 0 things
specified, which could be phrased as operating in "All-defaults" mode.

References:
* #428
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
This aligns cardinality of `itemValue` with `itemObject`, and permits
setting multiple values for a parameter, such as multiple independent
filter patterns.

References
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
Sean removed `tool:ConfiguredTool` and `tool:isConfigurationOf` from the
implementation for Issue 405, without explanation.  His revised
demonstration of how to represent a tool being used with a configuration
within an action included this excerpt (not complete UCO JSON-LD, but
sufficient to demonstrate the design issue):

```json
[
    {
      "@id": "kb:relationship-d6d18770-cb60-4779-b28a-905846dbd3f4",
      "@type": "uco-core:Relationship",
      "uco-core:source": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "uco-core:target": "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92",
      "uco-core:kindOfRelationship": "Has_Configuration",
      "uco-core:startTime": "2010-01-15T17:59:43.25Z",
    },
    {
      "@id": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "@type": "uco-analysis:Analysis",
      "uco-core:name": "compute string similarity",
      "uco-action:instrument": [
        "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
        "kb:relationship-d6d18770-cb60-4779-b28a-905846dbd3f4",
        "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92"
      ]
    }
]
```

`tool:ToolConfigurationTypeFacet` was also removed, and I agree that
that is obviated with the `Configuration` class.  Because that `Facet`
is removed, another mechanism is needed to link a configuration to a
`Tool` or a `Software`.

This patch restores `ConfiguredTool`, and adds a similar class
`ConfiguredSoftware` to meet the expanded objective of Issue 432.

`isConfigurationOf` is restored as a more generically-specified
property, leaving type-scoping to the SHACL associations.

`configuration:usesConfiguration` is added to link a `ConfiguredX`
instance to a `Configuration`.

With the above new and restored properties and classes, the above
JSON-LD snippet is now expressible like this, most notably removing the
complexity of references in `action:instrument`:

```json
[
    {
      "@id": "kb:configured-tool-c6718138-e953-4731-bf52-d7d89eae9c85",
      "@type": [
          "uco-tool:AnalyticTool",
          "uco-tool:ConfiguredTool",
      ],
      "uco-configuration:isConfigurationOf": "kb:AnalyticTool-DAE5EE58-E5ED-4588-93BE-CDEC6FAA9C6A",
      "uco-configuration:usesConfiguration": "kb:Configuration-4bc42c81-e9e7-4f48-b066-17e7fdca0f92",
      "uco-core:objectCreatedTime": "2010-01-15T17:59:43.25Z",
    },
    {
      "@id": "kb:Analysis-7cd51fa7-63ee-4f40-a482-9ce8333c7556",
      "@type": "uco-analysis:Analysis",
      "uco-core:name": "compute string similarity",
      "uco-action:instrument": "kb:configured-tool-c6718138-e953-4731-bf52-d7d89eae9c85"
    }
]
```

Tests are updated to exercise the new classes and properties.

A follow-on patch will regenerate Make-managed files.

References:
* #405
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
References:
* #405
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit that referenced this issue Aug 22, 2022
Issue 430 was presumed in some of the new `sh:nodeKind` specifications.
This patch relaxes those to `sh:BlankNodeOrIRI` as used everywhere else
in UCO before the passage of 430.  Once 430 is passed, this patch will
be reverted.

References:
* #430
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
@ajnelson-nist
Copy link
Contributor

@plbt5 , with apologies, I have too much release preparation to do to find time to review your suggested material. If you have a summary of the impact of the material that would re-shape the implementation of Issues #405 or this 432, please provide it as a pull request.

OCs: I've looked through @sbarnum 's implementation, and found he reverted some of the implementation I had done for 405. The changes are restored, with the largest documentation in this patch. I am significantly concerned with the usage of a core:Relationship as an action:instrument value, and I also think that his suggested usage of a core:Relationship is incorrect in the context of defining a configured tool (or software) object. A ConfiguredTool (/ConfiguredSoftware) object feels like an inherent object ( @plbt5 likely has a better term for this), in that it does not make sense to describe without the non-configured object. (Paul's usual example here is something like the color of a red car, where the car's redness does not exist if the car does not exist.)

The implementation will undergo testing tomorrow, but please weigh in if you have a preference for Sean's pattern (especially referring to his full JSON-LD snippet in the Issue's description) over the ConfiguredX subclasses.

The pull request's commits each have a log message documenting the commit's effects. Please refer to any of the commits if you observe an issue.

ajnelson-nist added a commit to casework/CASE-Archive that referenced this issue Aug 22, 2022
No effects were observed on Make-managed files.

References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/CASE-Examples that referenced this issue Aug 22, 2022
A follow-on patch will regenerate Make-managed files.

References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/CASE-Examples that referenced this issue Aug 22, 2022
References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/casework.github.io that referenced this issue Aug 22, 2022
A follow-on patch will regenerate Make-managed files.

References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/casework.github.io that referenced this issue Aug 22, 2022
References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/CASE-Examples that referenced this issue Aug 22, 2022
A follow-on patch will regenerate Make-managed files.

References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/CASE-Examples that referenced this issue Aug 22, 2022
References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/casework.github.io that referenced this issue Aug 22, 2022
A follow-on patch will regenerate Make-managed files.

References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/casework.github.io that referenced this issue Aug 22, 2022
References:
* ucoProject/UCO#432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
@ajnelson-nist
Copy link
Contributor

OCs: Impact on the CASE examples has been reviewed. The updates that will be necessary in support of the current state of implementation of Issue 432 are best illustrated in these two commits:

@ajnelson-nist
Copy link
Contributor

@sbarnum : I did not transcribe your JSON-LD example to CASE-Examples, because it presumed other in-flight proposals (JSON-LD context dictionary, and analysis namespace), some of which have received insufficient development to be scheduled for 1.0.0.

@ajnelson-nist
Copy link
Contributor

ajnelson-nist commented Aug 22, 2022

Question for the OCs, illustrating the "inherent" property configuration:usesConfiguration: Should Configuration objects be shareable?

Suppose a tool version 1.0 releases a version 1.1 with no API or CLI changes. Suppose also a configuration object kb:configuration-1 was used for the configured run of version 1. To you, is this acceptable usage?

[
    {
        "@id": "kb:tool-v10",
        "@type": "tool:Tool"
    },
    {
        "@id": "kb:tool-v11",
        "@type": "tool:Tool"
    },
    {
        "@id": "kb:configuration-1",
        "@type": "configuration:Configuration"
    },
    {
        "@id": "kb:configured-tool-v10",
        "@type": "tool:ConfiguredTool",
        "configuration:isConfigurationOf": "kb:tool-v10",
        "configuration:usesConfiguration": "kb:configuration-1"
    },
    {
        "@id": "kb:configured-tool-v11",
        "@type": "tool:ConfiguredTool",
        "configuration:isConfigurationOf": "kb:tool-v11",
        "configuration:usesConfiguration": "kb:configuration-1"
    }
]

If the committee thinks it not acceptable to permit this, then usesConfiguration should be declared, and tested as, an owl:InverseFunctionalProperty, as we had done for core:hasFacet in Issue 379.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 22, 2022

I would assert that it is not only desirable but necessary (for correct graph analysis and navigation) for a range of use cases beyond the simple example here, especially for Software (beyond just Tools). Think about things like DISA STIGs or the application space of NIST SP 800-128, etc.

@eoghanscasey
Copy link
Contributor

I agree with this combined solution to fulfill the requirements originally articulated in #405 and expanded in #455. Thank you Alex and Sean for working through this together to provide a robust solution.

@sbarnum
Copy link
Contributor Author

sbarnum commented Aug 25, 2022

The range for the itemObject property should be UcoThing to enable expression of structured configuration items.

I will provide an example lateer.

@ajnelson-nist
Copy link
Contributor

The range for the itemObject property should be UcoThing to enable expression of structured configuration items.

I will provide an example lateer.

This will need to be done in a later proposal in 1.x.0.

I described in the OCs call this morning that we're going into 1.0.0 with this on the tighter constraint of UcoObject. If we're wrong and need to expand, that is a backwards-compatible change. If this were reversed, we would not be able to tighten the constraint from UcoThing to UcoObject and call it a backwards-compatible change.

ajnelson-nist added a commit that referenced this issue Aug 30, 2022
This should have happened as part of resolving Issue 432.

No effects were observed on Make-managed files.

References:
* #432

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/CASE-Utilities-Python that referenced this issue Sep 2, 2022
RDFS and OWL are receiving aliases for in-common spelling in adopters'
code.  OWL also specifically got further support in some UCO issues.

This patch also adds a `Namespace` for the import of the Collections
Ontology, and the new UCO namespace `configuration`.

References:
* ucoProject/UCO#389
* ucoProject/UCO#406
* ucoProject/UCO#432
* ucoProject/UCO#437

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
ajnelson-nist added a commit to casework/CASE-Utilities-Python that referenced this issue Sep 2, 2022
RDFS and OWL are receiving aliases for in-common spelling in adopters'
code.  OWL also specifically got further support in some UCO issues.

This patch also adds a `Namespace` for the import of the Collections
Ontology, and the new UCO namespace `configuration`.

References:
* ucoProject/UCO#389
* ucoProject/UCO#406
* ucoProject/UCO#432
* ucoProject/UCO#437

Signed-off-by: Alex Nelson <alexander.nelson@nist.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants