OSS Review Toolkit

Linux (OpenJDK 8)	Windows (Oracle JDK 9)

Introduction

The goal of the OSS Review Toolkit (ORT) is to verify Free and Open Source Software licence compliance by checking project source code and dependencies.

At a high level, it works by analyzing the source code for dependencies, downloading the source code of the dependencies, scanning all source code for license information, and summarizing the results.

The different tools that make up ORT are designed as libraries (for programmatic use) with minimal command line interfaces (for scripted use, doing one thing and doing it well).

The toolkit is envisioned to consist of the following libraries:

• Analyzer - determines dependencies of a software project even when multiple package managers are used. No changes to software project required.
• Downloader - fetches the source code based on output generated by the Analyzer.
• Scanner - wrapper around existing copyright/license scanners which takes as input the output of the Downloader and produces standardized output such as SPDX.
• Evaluator * - Evaluates the scan results from Scanner as OK or NOT OK based on user specified approval/rejection ruleset.
• Advisor * - Retrieves security advisories based on Analyzer results.
• Reporter * - Summarizes the output from Analyzer, Scanner and Evaluator in an interactive UI that shows the identified copyrights, licenses and NOT OK issues.
• Documenter * - Generates the outcome of the review, e.g. Open Source notices and annotated SPDX files that can be included with your deliverable.

* Libraries to be completed by Q3 2018.

ORT Prerequisites

To be able to use ORT:

• Install OpenJDK 8 or Oracle JDK 8u161 or later (not the JRE as you need the javac compiler) is installed and the JAVA_HOME environment variable set.
• Set the locale of your system is set to en_US.UTF-8, using other locales might lead to issues with parsing the output of external tools.
• Install git
• Upload your SSH public key to GitHub -- https://github.com/settings/keys (Note that you must have an account on GitHub to do this)
• Install Yarn -- see https://yarnpkg.com/lang/en/docs/install/
• Install Node.js and NPM -- for example, instructions for Ubuntu 17.10 are on http://www.ubuntugeek.com/install-latest-version-of-node-js-and-npm-on-ubuntu-17-10-server.html

ORT Installation

To install the OSS Review Toolkit:

3. Clone this repository recursively, i.e. with submodules (git clone --recurse-submodules).

4. Change into the directory to which you cloned the repository on your machine and run ./gradlew installDist to set up the build environment (e.g. get Gradle, etc.) and build/install the start scripts for ORT. The individual start scripts can then be run directly from their respective locations as follows:

   • ./analyzer/build/install/analyzer/bin/analyzer
   • ./graph/build/install/graph/bin/graph
   • ./downloader/build/install/downloader/bin/downloader
   • ./scanner/build/install/scanner/bin/scanner

However, if this is your first encounter with ORT, please do not run these commands just yet.

Package Managers Supported by ORT

ORT is able to detect the following package managers/build systems and query them for depedencies:

• Gradle
• Maven
• SBT
• NPM
• PIP

License Scanners Supported by ORT

ORT comes with some example implementations for wrappers around license/copyright scanners:

• lc
• Licensee
• ScanCode

Remote Caches Supported by ORT

For reusing already known scan results, ORT works with the following backend as a remote cache:

• Artifactory

Main ORT Packages

Analyzer

The Analyzer determines the dependencies of software projects in the specified input directory (-i). It does so by querying the supported package manager it find in the directory. No modifications to your existing project source code, or especially to the build system, are necessary for that to work. The tree of transitive dependencies for each project is written out as ABCD-style YAML (or JSON, see -f) files to the specified output directory (-o) whose inner structure mirrors that of the input directory. The output files document the status quo of all package-related meta-data. They can and probably need to be further processed or manually edited before they are passed on to one of the other ORT tools.

Graph

The Graph tool visualizes the dependency information collected by the analyzer. It takes as input (-d) a file that lists the dependencies and displays them in graph form. The graph is interactive in the sense that nodes can be dragged & dropped with the mouse to rearrange them for a better overview.

Downloader

The Downloader clones the source code for all the dependencies identified by the analyzer. Its input (-d) a file that lists the dependencies. The packages are cloned to the output directory specified by the switch -o. The Downloader transparently normalizes URLs and uses the appropriate VCS tool to check out source code from version control.

Note that the Downloader is integrated into the Scanner, so there is normally no need to run it on its own.

Scanner

The Scanner tool wraps the underlying license/copyright scanners in a common API, providing a consistent way to run them and allowing for their results to be compared easily. If passed a dependencies analysis file (-d), the Scanner automatically downloads the source code of the dependencies via the Downloader and scans them. To prevent downloading and scanning of any previously scanned sources, the Scanner can be configured (-c) to use a remote cache, hosted e.g. on Artifactory or S3 (not yet implemented). For example, the following YAML file contains the configuration of an Artifactory cache:

scanner:
  cache:
    type: Artifactory
    url: "https://artifactory.domain.com/artifactory/generic-repository-name"
    apiToken: $ARTIFACTORY_API_KEY

Getting Started with ORT

Please see GettingStarted.md for an introduction to the tools.

ORT Development

The toolkit is written in Kotlin and uses Gradle as the build system. We recommend the IntelliJ IDEA Community Edition as the IDE which can directly import the Gradle build files.

The most important root project Gradle tasks are listed in the table below.

Task	Purpose
assemble	Build the JAR artifacts for all projects
detektCheck	Run static code analysis on all projects
test	Run unit tests for all projects
funTest	Run functional tests for all projects
installDist	Build all projects and install the start scripts for distribution

ORT License

Copyright (c) 2017-2018 HERE Europe B.V.

See the LICENSE file in the root of this project for license details.