version 2.1.0 (#3)

Xilinx · May 4, 2021 · d17744e · d17744e
1 parent 6b0a45a
commit d17744e
Show file tree

Hide file tree

Showing 55 changed files with 1,386 additions and 779 deletions.
diff --git a/README.md b/README.md
@@ -1,2 +1,26 @@
 # ROAST
-ROAST, a unified framework for component testing across system topologies addresses the challenges of testing a complex heterogenous adaptive computer acceleration platform (ACAP) against a wide range of scenarios, finding edge cases that would not have been usually covered with traditional testing methods. ROAST includes a modular plugin architecture built on Python for extensibility and scalability in building components, launching tests, and collecting results.
+ROAST is an open-source Python framework that simplifies the development of complex validation test suites. To accomplish this, ROAST provides a collection of interfaces that allows test developers to build
+test suites in a highly structured manner.
+
+Key features:
+
+- Compose systems from Xilinx or custom components
+- Define systems composed from various configuration sources
+- Heirarchical configuration system
+- Randomized data provider for randomized testing
+- Generic APIs for simplified usage and access
+- Plugin system for extensibility
+
+ROAST consists of three packages:
+
+- roast: Core functionality that includes interfaces, plugin framework, configuration
+  management, and more
+- roast-xilinx (recommended: plugin for roast that includes Xilinx specific functionality such as
+  board acquisition, PetaLinux building, Vitis wrapper, and more.
+- pytest-roast (optional): plugin for pytest that includes pytest fixtures for accelerated test
+  development when using pytest as a test runner. Features includes target automation, scenario
+  generation, and configuration generation.
+
+Example test flow with package breakdown:
+
+![ROAST packages](images/roast_packages.png)
diff --git a/docs/user/components.rst → docs/advanced/component_models.rst b/docs/user/components.rst → docs/advanced/component_models.rst
@@ -1,8 +1,8 @@
 .. _component-plugins-models:
 
-==============================
- Component Plugins and Models
-==============================
+=========================
+ Component Plugin Models
+=========================
 
 While there are several different methods to load code dynamically in Python, the best approach
 for ROAST is to built on top of setuptools `entry points`_. At a high level, the primary reasons
@@ -14,9 +14,6 @@ A plugin framework named `stevedore`_ includes the functionality needed by ROAST
 `stevedore`_ is maintained by the Redhat Openstack project, an extremely popular cloud
 computing platform.
 
-Plugin Model
-============
-
 When constructing systems, we need to establish a uniform methodology across all use cases
 so that end users can focus on test development rather than system construction. This is
 realized through the creation of plugin models that are defined to ensure an uniform API
@@ -26,6 +23,8 @@ Currently, there are two plugin models implemented - `TestSuite`_ and `System`_.
 interfaces are defined as ``TestSuiteBase`` (roast/component/testsuite.py) and ``SystemBase``
 (roast/component/system.py).
 
+.. _testsuite:
+
 TestSuite
 ---------
 
@@ -70,6 +69,8 @@ In this plugin model, there are four methods: :func:`configure`, :func:`build`,
 and :func:`run`. Each plugin that inherits from the base class will need to implement these
 methods.
 
+.. _system:
+
 System
 ------
 

diff --git a/docs/user/examples/component_plugins.rst → docs/advanced/component_plugins.rst b/docs/user/examples/component_plugins.rst → docs/advanced/component_plugins.rst
diff --git a/docs/advanced/index.rst b/docs/advanced/index.rst
@@ -0,0 +1,16 @@
+==========
+ Advanced
+==========
+
+In this section, the advanced topics will cover how plugins are used in ROAST.
+
+- **Component Plugin Models**: This covers how the two main component plugins, :ref:`testsuite` and
+  :ref:`system` are modeled.
+- **Component Plugins**: This covers how plugins are created and called.
+
+
+.. toctree::
+   :maxdepth: 3
+
+   component_models
+   component_plugins
diff --git a/docs/conf.py b/docs/conf.py
@@ -28,9 +28,9 @@
 author = "Xilinx"
 
 # The short X.Y version
-version = "1.0"
+version = "2.1"
 # The full version, including alpha/beta/rc tags
-release = "v1.0"
+release = "v2.1"
 html_last_updated_fmt = "%B %d, %Y"
 
 # -- General configuration ---------------------------------------------------

diff --git a/docs/images/roast_packages.png b/docs/images/roast_packages.png
diff --git a/docs/index.rst b/docs/index.rst
@@ -2,20 +2,39 @@
  ROAST -- Randomized Okaying Across System Topologies
 ======================================================
 
-ROAST, a unified framework for component testing across system topologies addresses the challenges
-of testing a complex heterogenous adaptive computer acceleration platform (ACAP) against a wide
-range of scenarios, finding edge cases that would not have been usually covered with traditional
-testing methods. ROAST includes a modular plugin architecture built on Python for extensibility
-and scalability in building components, launching tests, and collecting results.
+ROAST is an open-source Python framework that simplifies the development of complex validation
+test suites. To accomplish this, ROAST provides a collection of interfaces that allows test
+developers to build test suites in a highly structured manner.
+
+Key features:
+
+- Compose systems from Xilinx or custom components
+- Define systems composed from various configuration sources
+- Heirarchical configuration system
+- Randomized data provider for randomized testing
+- Generic APIs for simplified usage and access
+- Plugin system for extensibility
+
+ROAST consists of three packages:
+
+- **roast**: Core functionality that includes interfaces, plugin framework, configuration
+  management, and more
+- **roast-xilinx (recommended)**: plugin for roast that includes Xilinx specific functionality such as
+  board acquisition, PetaLinux building, Vitis wrapper, and more.
+- **pytest-roast (optional)**: plugin for pytest that includes pytest fixtures for accelerated test
+  development when using pytest as a test runner. Features includes board automation, scenario
+  generation, and configuration generation.
+
+Example test flow with package breakdown:
+
+.. image:: images/roast_packages.png
+
+Contents
+========
 
 .. toctree::
    :maxdepth: 3
 
    install/index
-   user/index
-   reference/index
-
-Indices and tables
-==================
-
-* :ref:`genindex`
+   tutorial/index
+   advanced/index
diff --git a/docs/install/index.rst b/docs/install/index.rst
@@ -4,10 +4,10 @@
 Installation
 ============
 
-Python Versions
----------------
+Versions
+--------
 
-ROAST requires Python 3.6+.
+ROAST supports Linux and requires Python 3.6+.
 
 Basic Installation
 ------------------
@@ -16,6 +16,10 @@ The easiest way to install it is using pip_::
 
   $ pip install roast
 
+To install the Xilinx plugin for roast::
+
+  $ pip install roast-xilinx
+
 Optionally, roast specific pytest fixtures can be installed through a pytest plugin::
 
   $ pip install pytest-roast

diff --git a/docs/reference/functions/pytest-roast.rst b/docs/reference/functions/pytest-roast.rst
@@ -4,12 +4,8 @@
 
 .. autofunction:: pytest_roast.create_configuration
 
-.. autofunction:: pytest_roast.get_cmdl_machine_opt
-
 .. autofunction:: pytest_roast.create_scenario
 
-.. autofunction:: pytest_roast.board_session
-
 .. autofunction:: pytest_roast.get_board
 
 .. autofunction:: pytest_roast.board
diff --git a/docs/user/examples/hello_world.rst → docs/tutorial/complex_structures.rst b/docs/user/examples/hello_world.rst → docs/tutorial/complex_structures.rst
@@ -1,89 +1,11 @@
-.. _hello-world:
+.. _complex-repository-structures:
 
-=============
- Hello World
-=============
+===============================
+ Complex Repository Structures
+===============================
 
-The objective of this tutorial is to build a test repository starting with a a basic test and
-expanding to more complex parameterized and categoried tests leveraging ROAST to generate its
-configuration.
-
-.. contents::
-  :local:
-
-Environment Setup
-=================
-
-While ROAST is test runner agnostic, this tutorial will use `pytest`_ and
-:ref:`pytest fixtures<pytest-fixtures>` developed specifically for ROAST. This means both ``roast``
-and ``pytest-roast`` packages must be installed. While optional, it is highly recommended that
-these are installed into a `virtual environment`_.
-
-Upon installation of ``pytest-roast``, ``pytest`` will automatically be installed as a dependency.
-
-.. note::
-   Please review :ref:`installation` instructions on how to install Python packages using pip.
-
-Repository Structure
-====================
-
-Tests can be structured in a number of ways. In this tutorial, we'll first start with a basic
-structure and then expand to parameterized and categorized tests.
-
-Basic
------
-
-Create a repository with the following structure and files::
-
-   basic/
-   └── tests/
-       ├── test_basic.py
-       └── conf.py
-
-conf.py
-
-.. code-block:: python
-
-    var = "hello world"
-
-test_basic.py
-
-.. code-block:: python
-
-    import pytest
-
-    def test_basic(create_configuration):
-        conf = create_configuration()
-        assert conf.var == "hello world"
-
-In **test_basic.py**, we can see that the ``create_configuration`` fixture is called which reads
-from **conf.py** and assigns it to the ``conf`` variable. The value of ``var`` is then accessed
-by accessing ``conf.var``.
-
-.. note::
-   For other methods of accessing configuration values, visit :ref:`configuration-system`.
-
-First, let's see what pytest collects as tests::
-
-    $ pytest --collect-only
-    ======= test session starts ========
-    ..
-    collected 1 item
-    <Module tests/test_basic.py>
-      <Function test_basic>
-
-    ====== no tests ran in 0.04s =======
-
-We can now execute the test::
-
-    $ pytest
-    ====== test session starts =======
-    ..
-    collected 1 item
-
-    tests/test_basic.py .       [100%]
-
-    ======= 1 passed in 0.18s ========
+In this tutorial, we're going to greatly expand the repository to accomodate for test variations
+and expand to complex parameterized and categoried tests.
 
 Parameterized
 -------------
@@ -169,9 +91,6 @@ parameter has a value of ``["parameter2"]`` and the configuration files from dir
 and **test/parameter2** will be used. The ``properties.expected`` value that is compared with
 ``config.var`` is ``"hello parameter2"``.
 
-.. note::
-   For more details on test parameterization, visit `pytest parameterization`_.
-
 Let's see what pytest collects as tests::
 
     $ pytest --collect-only
@@ -332,8 +251,4 @@ Let's now execute the tests::
     category/test_something.py ..      [ 50%]
     category/test_something_else.py .. [100%]
 
-    =========== 4 passed in 0.21s ===========
-
-.. _pytest: https://pytest.org/
-.. _virtual environment: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
-.. _pytest parameterization: https://docs.pytest.org/en/stable/parametrize.html
+    =========== 4 passed in 0.21s ===========
diff --git a/docs/user/configuration.rst → docs/tutorial/configuration.rst b/docs/user/configuration.rst → docs/tutorial/configuration.rst
@@ -23,7 +23,7 @@ Configuration files must be named **conf** and can be written in a number of for
 
 .. seealso::
 
-   :ref:`hello-world` tutorial for examples of project structures.
+   :ref:`getting-started` tutorial for examples of project structures.
 
 Single File
 ===========

diff --git a/docs/tutorial/examples_repository.rst b/docs/tutorial/examples_repository.rst
@@ -0,0 +1,13 @@
+.. _examples-repository:
+
+=====================
+ Examples Repository
+=====================
+
+In this repository, full working examples are available to showcase various features of ROAST. 
+These range from simple "hello word" applications to full Linux images.
+
+Varying styles of repository structures and test parameterization are provided to illustrate
+how test variants can be defined within a test suite.
+
+Visit https://github.com/Xilinx/roast-examples for details.