buildtesters · shahzebsiddiqui · Jan 8, 2023 · Jan 7, 2023 · Jan 7, 2023 · Jan 7, 2023
diff --git a/docs/buildspec_tutorial.rst b/docs/buildspec_tutorial.rst
@@ -1,17 +1,13 @@
-.. _writing_buildspecs:
+.. _buildspec_tutorial:
 
 Buildspec Tutorial
 ====================
 
 .. toctree::
  :maxdepth: 1
 
- buildspecs/global
  buildspecs/buildspec_overview
- buildspecs/customize_shell
  buildspecs/performance_checks
- buildspecs/test_dependency
- buildspecs/multi_executor
  buildspecs/compiler
  buildspecs/spack
 

diff --git a/docs/buildtest_site.rst b/docs/buildtest_site.rst
@@ -32,7 +32,7 @@ that provides how buildtest is configured at NERSC. Once you have defined your c
 Writing Test
 -------------
 
-If you are going to write test, we assume you have read :ref:`writing_buildspecs` section which covers
+If you are going to write test, we assume you have read :ref:`buildspec_tutorial` section which covers
 how to write buildspecs. You should consider reviewing the Schema Documentation: https://buildtesters.github.io/buildtest/
 which goes in detail about each schema and buildspec attributes.
 

diff --git a/docs/references.rst → docs/conferences.rst b/docs/references.rst → docs/conferences.rst
diff --git a/docs/gettingstarted/buildingtest.rst b/docs/gettingstarted/buildingtest.rst
@@ -4,7 +4,7 @@ Building Test via buildtest (``buildtest build``)
 =================================================
 
 This guide will get you familiar with buildtest command line interface. Once
-you complete this section, you can proceed to :ref:`writing buildspecs <writing_buildspecs>`
+you complete this section, you can proceed to :ref:`writing buildspecs <buildspec_tutorial>`
 section where we will cover how to write buildspecs.
 
 Once you install buildtest, you should find the `buildtest` command in your **$PATH**.

diff --git a/docs/index.rst b/docs/index.rst
@@ -76,9 +76,16 @@ automate module load testing. For more details on lmodule see https://github.com
 
 .. toctree::
  :maxdepth: 2
- :caption: Tutorial
+ :caption: Getting Started
 
  installing_buildtest
+ quick_start
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Tutorial
+
+
  getting_started
  buildspec_tutorial
 
@@ -101,12 +108,13 @@ automate module load testing. For more details on lmodule see https://github.com
  :maxdepth: 2
  :caption: Reference
 
+ writing_buildspecs
  features
  schema_examples
  facility_examples
  api
  command
- references
+ conferences
 
 .. toctree::
  :maxdepth: 2

diff --git a/docs/installing_buildtest.rst b/docs/installing_buildtest.rst
@@ -112,4 +112,4 @@ use buildtest. Shown below is the output
 
  .. command-output:: buildtest --help
 
-If you have got this far, please go to the next section on :ref:`getting_started`
+If you have got this far, you can check out the :ref:`quick_start` or :ref:`getting_started`
diff --git a/docs/quick_start.rst b/docs/quick_start.rst
@@ -0,0 +1,157 @@
+.. _quick_start:
+
+Quick Start Guide
+==================
+
+Building Test
+--------------
+
+The ``buildtest build`` command is used for running a test. There are several ways to run the test,
+the most common way to run arbitrary test is to specify file path::
+
+ buildtest build -b <file>
+
+To specify multiple files you can append ``-b`` option::
+
+ buildtest build -b <file1> -b <file2>
+
+To run all tests within a directory, specify path to directory, let's assume we want
+to run all tests in **examples** directory you can do the following::
+
+ buildtest build -b examples
+
+buildtest can exclude buildspecs via ``-x`` option which can operate on file or directory similar to ``-b``. This can be useful
+if you want to run tests in a directory but exclude one file. For example you can do the following to exclude test ``examples/hello.yml``::
+
+ buildtest build -b examples -x examples/hello.yml
+
+To build tests by tagname you can use the ``-t`` option where buildtest will discover all tests that belong to the corresponding tag.
+For instance to run all tests for tagname **network** you can do the following::
+
+ buildtest build -t network
+
+
+Inspecting a test
+--------------------
+
+Upon execution of test you can inspect test results for a given test, where **name** is the name of test::
+
+ buildtest inspect query <name>
+
+To see content of output and error file for a given test, you can run::
+
+ buildtest inspect query -o -e <name>
+
+You can inspect multiple tests simultaneously since test names are positional arguments,
+let's assume we have two test names **jobA** and **jobB**, we can do the following::
+
+ buildtest inspect query jobA jobB
+
+Every test has a unique identifier, if you want to select a specific test run you specify the test identifier such as following::
+
+ buildtest inspect query always_fail/c0e886c6-8a1f-4063-846f-d16bc4733b04
+
+Note buildtest has tab completion so test IDs will be auto-completed as you type.
+
+You can use regular expression with ``buildtest inspect query`` command which can be useful to select multiple
+tests. You must specify test names in quotes in order for buildtest to process regular expression::
+
+ buildtest inspect query "always_fail/(c0|8e)"
+
+To see a list of available tests you can run::
+
+ buildtest inspect query list
+
+You can retrieve path attributes for a given test via ``buildtest path`` command which expects a test name. To
+retrieve path for a test you can run::
+
+ buildtest path hello_world
+
+Note you can only specify one test name. You can retrieve path to output or error file by running::
+
+ buildtest path -o hello_world
+ buildtest path -e hello_world
+
+Querying Test Report
+----------------------
+
+buildtest will keep track of all tests in a report file, to see all tests you can run::
+
+ buildtest report
+
+To see all pass or failed tests you can run::
+
+ buildtest report --pass
+ buildtest report --fail
+
+If you want to filter tests by returncode you can use ``--filter`` field which expects a key=value pair. You can see
+all available filter fields via ``buildtest report --helpfilter``. To filter by returncode 1 you can do::
+
+ buildtest report --filter returncode=1
+
+If you want to see a summary of the report file you can run::
+
+ buildtest report summary
+
+Developing a test
+------------------
+
+When you are creating a test known as **buildspec** you will want to make sure buildspec is valid, this can
+be done by running **buildtest buildspec validate**. This command works similar to ``buildtest build`` where one
+can specify file or directory to search for buildspecs. To validate multiple buildspecs you can do the following::
+
+ buildtest buildspec validate -b <file1> -b <file2>
+ buildtest buildspec validate -b <dir1> -b <file1>
+
+buildtest can load all buildspecs in the cache so you can query cache to find all buildspecs, query all tags,
+search for test maintainers and much more. To see all available buildspecs in the cache you can run::
+
+ buildtest buildspec find
+
+If you want to see a summary of the buildspec cache you can run::
+
+ buildtest buildspec summary
+
+To see all invalid buildspecs in your cache you can run::
+
+ buildtest buildspec find invalid
+
+The ``buildtest buildspec edit-test`` and ``buildtest buildspec edit-file`` are builtin commands to open
+buildspec in editor similar to how one would open in your preferred editor. The one benefit of using these commands,
+is buildtest will validate the test after closing file. The difference between the two commands is one operates on test
+names while the later operates on filepaths. Shown below is example usage for both commands::
+
+ buildtest buildspec edit-test hello_world
+ buildtest buildspec edit-file examples/foo.yml
+
+To view content of buildspec, you can use ``buildtest buildspec show`` where argument is name of test. You can
+specify multiple test name to see content of all tests such as shown below::
+
+ buildtest buildspec show hello_world foo_bar
+
+The ``buildtest buildspec show-fail`` command will show content of all buildspecs that failed tests during execution which
+are all tests reported by ``buildtest report --fail``. Note this is not to be confused with all invalid buildspecs.
+
+Buildtest Configuration
+-------------------------
+
+The buildtest configuration file is used to :ref:`configuring_buildtest` for your system. In order to use buildtest, your
+configuration file must be valid. You can validate the configuration file by running::
+
+ buildtest config validate
+
+To view content of your configuration file you can run::
+
+ buildtest config view
+
+You can retrieve path to configuration file by running::
+
+ buildtest config path
+
+When building tests, you can specify configuration file to use via::
+
+ buildtest -c <config> build -b <file>
+
+To see a list of available :ref:`executors <configuring_executors>` defined in your configuration file you can run::
+
+ buildtest config executors
diff --git a/docs/writing_buildspecs.rst b/docs/writing_buildspecs.rst
@@ -0,0 +1,14 @@
+.. _writing_buildspecs:
+
+
+Writing Buildspecs
+===================
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Buildspec Reference
+
+ writing_buildspecs/global
+ writing_buildspecs/test_dependency
+ writing_buildspecs/multi_executor
+ writing_buildspecs/customize_shell
diff --git a/docs/buildspecs/customize_shell.rst → docs/writing_buildspecs/customize_shell.rst b/docs/buildspecs/customize_shell.rst → docs/writing_buildspecs/customize_shell.rst
diff --git a/docs/buildspecs/global.rst → docs/writing_buildspecs/global.rst b/docs/buildspecs/global.rst → docs/writing_buildspecs/global.rst
diff --git a/docs/buildspecs/multi_executor.rst → docs/writing_buildspecs/multi_executor.rst b/docs/buildspecs/multi_executor.rst → docs/writing_buildspecs/multi_executor.rst
diff --git a/docs/buildspecs/test_dependency.rst → docs/writing_buildspecs/test_dependency.rst b/docs/buildspecs/test_dependency.rst → docs/writing_buildspecs/test_dependency.rst