Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: maintain the latest spdx model and provide conversions from previous #172

Merged
merged 1 commit into from
Jan 17, 2023
Merged

refactor!: maintain the latest spdx model and provide conversions from previous #172

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
25 changes: 8 additions & 17 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

# SPDX tools-golang

tools-golang is a collection of Go packages intended to make it easier for
`tools-golang` is a collection of Go packages intended to make it easier for
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be super helpful to link your usage repo somewhere in this README or as a comment in the root /spdx package as an example of how to use this new refactored version!

Copy link
Collaborator Author

@kzantow kzantow Jan 17, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with the usage repo is it's just a branch that will probably get deleted. Once this is merged and I've updated Syft, I could have a follow-on to the README that links directly to the Syft usage but probably shouldn't put any "permanent" links before that.

It should also be noted that I've updated all the examples -- is this enough? I could also add some more examples, like converting documents...

Go programs to work with [SPDX®](https://spdx.dev/) files.

## Recent news
Expand All @@ -23,15 +23,14 @@ of the SPDX specification, available at: https://spdx.dev/specifications
tools-golang provides the following packages:

* *spdx* - in-memory data model for the sections of an SPDX document
* *tvloader* - tag-value document loader
* *tvsaver* - tag-value document saver
* *rdfloader* - RDF document loader
* *json* - JSON document parser and writer
* *yaml* - YAML document parser and writer
* *tagvalue* - tag-value document reader and writer
kzantow marked this conversation as resolved.
Show resolved Hide resolved
* *rdf* - RDF document reader
* *json* - JSON document reader and writer
* *yaml* - YAML document reader and writer
* *builder* - builds "empty" SPDX document (with hashes) for directory contents
* *idsearcher* - searches for [SPDX short-form IDs](https://spdx.org/ids/) and builds SPDX document
* *idsearcher* - searches for [SPDX short-form IDs](https://spdx.org/ids/) and builds an SPDX document
* *licensediff* - compares concluded licenses between files in two packages
* *reporter* - generates basic license count report from SPDX document
* *reporter* - generates basic license count report from an SPDX document
* *spdxlib* - various utility functions for manipulating SPDX documents in memory
* *utils* - various utility functions that support the other tools-golang packages

Expand All @@ -40,15 +39,7 @@ directory.

## What it doesn't do

tools-golang doesn't currently do any of the following:

* work with files under any version of the SPDX spec prior to v2.1
* convert between different versions of SPDX documents (e.g., from 2.1 to 2.2)
* enable applications to interact with SPDX files without needing to care
(too much) about the particular SPDX file version

We are working towards adding functionality for all of these. Code contributions
are welcome!
`tools-golang` doesn't currently support files under any version of the SPDX spec prior to v2.1

## Documentation

Expand Down
181 changes: 21 additions & 160 deletions builder/build.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,90 +7,14 @@ package builder
import (
"fmt"

"github.com/spdx/tools-golang/builder/builder2v1"
"github.com/spdx/tools-golang/builder/builder2v2"
"github.com/spdx/tools-golang/builder/builder2v3"
"github.com/spdx/tools-golang/spdx/common"
"github.com/spdx/tools-golang/spdx/v2_1"
"github.com/spdx/tools-golang/spdx/v2_2"
"github.com/spdx/tools-golang/spdx/v2_3"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/common"
)

// ===== 2.1 builder =====

// Config2_1 is a collection of configuration settings for builder
// (for version 2.1 SPDX Documents). A few mandatory fields are set here
// so that they can be repeatedly reused in multiple calls to Build2_1.
type Config2_1 struct {
// NamespacePrefix should be a URI representing a prefix for the
// namespace with which the SPDX Document will be associated.
// It will be used in the DocumentNamespace field in the CreationInfo
// section, followed by the per-Document package name and a random UUID.
NamespacePrefix string

// CreatorType should be one of "Person", "Organization" or "Tool".
// If not one of those strings, it will be interpreted as "Person".
CreatorType string

// Creator will be filled in for the given CreatorType.
Creator string

// PathsIgnored lists certain paths to be omitted from the built document.
// Each string should be a path, relative to the package's dirRoot,
// to a specific file or (for all files in a directory) ending in a slash.
// Prefix the string with "**" to omit all instances of that file /
// directory, regardless of where it is in the file tree.
PathsIgnored []string

// TestValues is used to pass fixed values for testing purposes
// only, and should be set to nil for production use. It is only
// exported so that it will be accessible within builder2v1.
TestValues map[string]string
}

// Build2_1 creates an SPDX Document (version 2.1), returning that document or
// error if any is encountered. Arguments:
// - packageName: name of package / directory
// - dirRoot: path to directory to be analyzed
// - config: Config object
func Build2_1(packageName string, dirRoot string, config *Config2_1) (*v2_1.Document, error) {
// build Package section first -- will include Files and make the
// package verification code available
pkg, err := builder2v1.BuildPackageSection2_1(packageName, dirRoot, config.PathsIgnored)
if err != nil {
return nil, err
}

ci, err := builder2v1.BuildCreationInfoSection2_1(config.CreatorType, config.Creator, config.TestValues)
if err != nil {
return nil, err
}

rln, err := builder2v1.BuildRelationshipSection2_1(packageName)
if err != nil {
return nil, err
}

doc := &v2_1.Document{
SPDXVersion: "SPDX-2.1",
DataLicense: "CC0-1.0",
SPDXIdentifier: common.ElementID("DOCUMENT"),
DocumentName: packageName,
DocumentNamespace: fmt.Sprintf("%s%s-%s", config.NamespacePrefix, packageName, pkg.PackageVerificationCode),
CreationInfo: ci,
Packages: []*v2_1.Package{pkg},
Relationships: []*v2_1.Relationship{rln},
}

return doc, nil
}

// ===== 2.2 builder =====

// Config2_2 is a collection of configuration settings for builder
// (for version 2.2 SPDX Documents). A few mandatory fields are set here
// so that they can be repeatedly reused in multiple calls to Build2_2.
type Config2_2 struct {
// Config is a collection of configuration settings for builder.
// A few mandatory fields are set here
// so that they can be repeatedly reused in multiple calls to Build.
type Config struct {
// NamespacePrefix should be a URI representing a prefix for the
// namespace with which the SPDX Document will be associated.
// It will be used in the DocumentNamespace field in the CreationInfo
Expand All @@ -113,111 +37,48 @@ type Config2_2 struct {

// TestValues is used to pass fixed values for testing purposes
// only, and should be set to nil for production use. It is only
// exported so that it will be accessible within builder2v2.
// exported so that it will be accessible within builder.
TestValues map[string]string
}

// Build2_2 creates an SPDX Document (version 2.2), returning that document or
// Build creates an SPDX Document, returning that document or
// error if any is encountered. Arguments:
// - packageName: name of package / directory
// - dirRoot: path to directory to be analyzed
// - config: Config object
func Build2_2(packageName string, dirRoot string, config *Config2_2) (*v2_2.Document, error) {
func Build(packageName string, dirRoot string, config *Config) (*spdx.Document, error) {
// build Package section first -- will include Files and make the
// package verification code available
pkg, err := builder2v2.BuildPackageSection2_2(packageName, dirRoot, config.PathsIgnored)
pkg, err := BuildPackageSection(packageName, dirRoot, config.PathsIgnored)
if err != nil {
return nil, err
}

ci, err := builder2v2.BuildCreationInfoSection2_2(config.CreatorType, config.Creator, config.TestValues)
ci, err := BuildCreationInfoSection(config.CreatorType, config.Creator, config.TestValues)
if err != nil {
return nil, err
}

rln, err := builder2v2.BuildRelationshipSection2_2(packageName)
rln, err := BuildRelationshipSection(packageName)
if err != nil {
return nil, err
}

doc := &v2_2.Document{
SPDXVersion: "SPDX-2.2",
DataLicense: "CC0-1.0",
SPDXIdentifier: common.ElementID("DOCUMENT"),
DocumentName: packageName,
DocumentNamespace: fmt.Sprintf("%s%s-%s", config.NamespacePrefix, packageName, pkg.PackageVerificationCode),
CreationInfo: ci,
Packages: []*v2_2.Package{pkg},
Relationships: []*v2_2.Relationship{rln},
}

return doc, nil
}

// ===== 2.3 builder =====

// Config2_3 is a collection of configuration settings for builder
// (for version 2.3 SPDX Documents). A few mandatory fields are set here
// so that they can be repeatedly reused in multiple calls to Build2_3.
type Config2_3 struct {
// NamespacePrefix should be a URI representing a prefix for the
// namespace with which the SPDX Document will be associated.
// It will be used in the DocumentNamespace field in the CreationInfo
// section, followed by the per-Document package name and a random UUID.
NamespacePrefix string

// CreatorType should be one of "Person", "Organization" or "Tool".
// If not one of those strings, it will be interpreted as "Person".
CreatorType string
var packageVerificationCode common.PackageVerificationCode

// Creator will be filled in for the given CreatorType.
Creator string

// PathsIgnored lists certain paths to be omitted from the built document.
// Each string should be a path, relative to the package's dirRoot,
// to a specific file or (for all files in a directory) ending in a slash.
// Prefix the string with "**" to omit all instances of that file /
// directory, regardless of where it is in the file tree.
PathsIgnored []string

// TestValues is used to pass fixed values for testing purposes
// only, and should be set to nil for production use. It is only
// exported so that it will be accessible within builder2v3.
TestValues map[string]string
}

// Build2_3 creates an SPDX Document (version 2.3), returning that document or
// error if any is encountered. Arguments:
// - packageName: name of package / directory
// - dirRoot: path to directory to be analyzed
// - config: Config object
func Build2_3(packageName string, dirRoot string, config *Config2_3) (*v2_3.Document, error) {
// build Package section first -- will include Files and make the
// package verification code available
pkg, err := builder2v3.BuildPackageSection2_3(packageName, dirRoot, config.PathsIgnored)
if err != nil {
return nil, err
}

ci, err := builder2v3.BuildCreationInfoSection2_3(config.CreatorType, config.Creator, config.TestValues)
if err != nil {
return nil, err
}

rln, err := builder2v3.BuildRelationshipSection2_3(packageName)
if err != nil {
return nil, err
if pkg.PackageVerificationCode != nil {
packageVerificationCode = *pkg.PackageVerificationCode
}

doc := &v2_3.Document{
SPDXVersion: "SPDX-2.3",
DataLicense: "CC0-1.0",
doc := &spdx.Document{
SPDXVersion: spdx.Version,
DataLicense: spdx.DataLicense,
SPDXIdentifier: common.ElementID("DOCUMENT"),
DocumentName: packageName,
DocumentNamespace: fmt.Sprintf("%s%s-%s", config.NamespacePrefix, packageName, pkg.PackageVerificationCode),
DocumentNamespace: fmt.Sprintf("%s%s-%s", config.NamespacePrefix, packageName, packageVerificationCode),
CreationInfo: ci,
Packages: []*v2_3.Package{pkg},
Relationships: []*v2_3.Relationship{rln},
Packages: []*spdx.Package{pkg},
Relationships: []*spdx.Relationship{rln},
}

return doc, nil
Expand Down
12 changes: 6 additions & 6 deletions builder/builder2v3/build_creation_info.go → builder/build_creation_info.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
// SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later

package builder2v3
package builder

import (
"time"

"github.com/spdx/tools-golang/spdx/common"
"github.com/spdx/tools-golang/spdx/v2_3"
"github.com/spdx/tools-golang/spdx"
"github.com/spdx/tools-golang/spdx/v2/common"
)

// BuildCreationInfoSection2_3 creates an SPDX Package (version 2.3), returning that
// BuildCreationInfoSection creates an SPDX Package, returning that
// package or error if any is encountered. Arguments:
// - packageName: name of package / directory
// - code: verification code from Package
// - namespacePrefix: prefix for DocumentNamespace (packageName and code will be added)
// - creatorType: one of Person, Organization or Tool
// - creator: creator string
// - testValues: for testing only; call with nil when using in production
func BuildCreationInfoSection2_3(creatorType string, creator string, testValues map[string]string) (*v2_3.CreationInfo, error) {
func BuildCreationInfoSection(creatorType string, creator string, testValues map[string]string) (*spdx.CreationInfo, error) {
// build creator slices
creators := []common.Creator{
// add builder as a tool
Expand All @@ -39,7 +39,7 @@ func BuildCreationInfoSection2_3(creatorType string, creator string, testValues
created = testVal
}

ci := &v2_3.CreationInfo{
ci := &spdx.CreationInfo{
Creators: creators,
Created: created,
}
Expand Down
24 changes: 12 additions & 12 deletions ...er/builder2v2/build_creation_info_test.go → builder/build_creation_info_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
// SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later

package builder2v2
package builder

import (
"testing"
)

// ===== CreationInfo section builder tests =====
func TestBuilder2_2CanBuildCreationInfoSection(t *testing.T) {
func TestBuilderCanBuildCreationInfoSection(t *testing.T) {
creatorType := "Organization"
creator := "Jane Doe LLC"
testValues := make(map[string]string)
testValues["Created"] = "2018-10-20T16:48:00Z"

ci, err := BuildCreationInfoSection2_2(creatorType, creator, testValues)
ci, err := BuildCreationInfoSection(creatorType, creator, testValues)
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
Expand All @@ -35,13 +35,13 @@ func TestBuilder2_2CanBuildCreationInfoSection(t *testing.T) {
}
}

func TestBuilder2_2CanBuildCreationInfoSectionWithCreatorPerson(t *testing.T) {
func TestBuilderCanBuildCreationInfoSectionWithCreatorPerson(t *testing.T) {
creatorType := "Person"
creator := "John Doe"
testValues := make(map[string]string)
testValues["Created"] = "2018-10-20T16:48:00Z"

ci, err := BuildCreationInfoSection2_2(creatorType, creator, testValues)
ci, err := BuildCreationInfoSection(creatorType, creator, testValues)
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
Expand All @@ -60,13 +60,13 @@ func TestBuilder2_2CanBuildCreationInfoSectionWithCreatorPerson(t *testing.T) {
}
}

func TestBuilder2_2CanBuildCreationInfoSectionWithCreatorTool(t *testing.T) {
func TestBuilderCanBuildCreationInfoSectionWithCreatorTool(t *testing.T) {
creatorType := "Tool"
creator := "some-other-tool-2.1"
creator := "some-other-tool"
testValues := make(map[string]string)
testValues["Created"] = "2018-10-20T16:48:00Z"

ci, err := BuildCreationInfoSection2_2(creatorType, creator, testValues)
ci, err := BuildCreationInfoSection(creatorType, creator, testValues)
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
Expand All @@ -80,18 +80,18 @@ func TestBuilder2_2CanBuildCreationInfoSectionWithCreatorTool(t *testing.T) {
if ci.Creators[0].Creator != "github.com/spdx/tools-golang/builder" {
t.Errorf("expected %s, got %s", "github.com/spdx/tools-golang/builder", ci.Creators[0])
}
if ci.Creators[1].Creator != "some-other-tool-2.1" {
t.Errorf("expected %s, got %s", "some-other-tool-2.1", ci.Creators[1])
if ci.Creators[1].Creator != "some-other-tool" {
t.Errorf("expected %s, got %s", "some-other-tool", ci.Creators[1])
}
}

func TestBuilder2_2CanBuildCreationInfoSectionWithInvalidPerson(t *testing.T) {
func TestBuilderCanBuildCreationInfoSectionWithInvalidPerson(t *testing.T) {
creatorType := "Whatever"
creator := "John Doe"
testValues := make(map[string]string)
testValues["Created"] = "2018-10-20T16:48:00Z"

ci, err := BuildCreationInfoSection2_2(creatorType, creator, testValues)
ci, err := BuildCreationInfoSection(creatorType, creator, testValues)
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
Expand Down
Loading