Add section about config files in the Launcher tutorial

docs/explanation/configs.rst

@@ -1,3 +1,5 @@
+.. _checkbox_configs:
+
 Checkbox Configs
 ^^^^^^^^^^^^^^^^
 
@@ -32,6 +34,70 @@ value resolution is as follows:
 3. config file from ``/etc/xdg``
 4. config file from ``$SNAP_DATA``
 
+Configuration checker
+=====================
+
+The values resolution order and the fact that configurations can be stored in
+so many different places may bring confusion when running Checkbox.
+
+Fortunately, the ``check-config`` command will list:
+
+- all the configuration files being used
+- for each section, the configured parameters being used
+- the origin of each of these customized parameters
+- an overall status report
+
+For example:
+
+.. code-block:: none
+
+ $ checkbox-cli check-config
+
+ Configuration files:
+ - /var/snap/checkbox/2799/checkbox.conf
+ - /home/user/.config/checkbox.conf
+ [config]
+ config_filename=checkbox.conf (Default)
+ (...)
+ [test plan]
+ filter=*wireless* From config file: /home/user/.config/checkbox.conf
+ forced=False (Default)
+ unit= (Default)
+ [test selection]
+ exclude= (Default)
+ forced=False (Default)
+ (...)
+ [environment]
+ STRESS_BOOT_ITERATIONS=100 From config file: /var/snap/checkbox/2799/checkbox.conf
+ (...)
+ [manifest]
+ No problems with config(s) found!
+
+A configuration file may have errors. Consider the following ``checkbox.conf``
+placed in ``/home/user/.config/``:
+
+.. code-block:: none
+
+ [tset plan]
+ filter = *wireless*
+
+ [test selection]
+ wrong_var = example
+
+When running the ``check-config`` command, the following will be reported:
+
+.. code-block:: none
+
+ Problems:
+ - Unexpected section [tset plan]. Origin: /home/user/.config/checkbox.conf
+ - Unexpected variable 'wrong_var' in section [test selection] Origin: /home/user/.config/checkbox.conf
+
+Indeed, there is a typo in the name of the ``[test plan]`` section, and
+a unknown variable is set in the ``[test selection]`` section. For more
+information on the available sections and variables, please check the
+:ref:`launcher` reference.
+
+
 Configs with Checkbox Remote
 ============================
 

docs/tutorial/using-checkbox/basic-launcher.rst

@@ -36,7 +36,7 @@ Save it, then launch Checkbox using this launcher:
 
 .. code-block:: none
 
- checkbox.checkbox-cli launcher mylauncher
+ $ checkbox.checkbox-cli launcher mylauncher
 
 The test plan selection screen should be much less intimidating now!
 
@@ -106,7 +106,7 @@ Run Checkbox with this modified version of the launcher:
 
 .. code-block:: none
 
- checkbox.checkbox-cli launcher mylauncher
+ $ checkbox.checkbox-cli launcher mylauncher
 
 Notice how none of the initial screens are shown and Checkbox immediately
 runs the TODO test plan. This is because:
@@ -195,6 +195,96 @@ In Checkbox language, submissions files are the HTML test report as well as
 an archive containing the test results and additional logs that might have
 been produced by the test cases.
 
+A note about config files
+=========================
+
+So far, you have customized Checkbox using a launcher file. It is also
+possible to put these options in a configuration file that Checkbox will use
+when it is launched. The main difference is that you don't have to specify
+the launcher when running Checkbox.
+
+Create the file ``~/.config/checkbox.conf`` and add the following content
+in it:
+
+.. code-block:: none
+
+ [launcher]
+ launcher_version = 1
+ app_id = com.canonical.certification:tutorial
+ stock_reports = text, submission_files
+
+ [test plan]
+ unit = com.canonical.certification::TODO
+ forced = yes
+
+ [test selection]
+ forced = yes
+
+ [environment]
+ TUTO = tutorial
+
+Now, run Checkbox without any argument:
+
+.. code-block:: none
+
+ $ checkbox.checkbox-cli
+
+You should see that Checkbox behaves exactly the same as in the previous
+section. It found the configuration from the ``~/.config/checkbox.conf``
+file and used it to automatically select the test plan and run it.
+
+Configuration files can be placed elsewhere on the system, and Checkbox
+will follow a certain resolution order to decide what configuration to
+use if more than one configuration files define the same key. Please check
+:ref:`checkbox_configs` for more information.
+
+Checkbox comes with a handy command to check what configuration is being used,
+and where it comes from. Run the following command:
+
+.. code-block:: none
+
+ $ checkbox.checkbox-cli check-config
+ Configuration files:
+ - /var/snap/checkbox/2799/checkbox.conf
+ - /home/user/.config/checkbox.conf
+ [config]
+ config_filename=checkbox.conf (Default)
+ [launcher]
+ app_id=com.canonical.certification:tutorial From config file: /home/user/.config/checkbox.conf
+ app_version= (Default)
+ launcher_version=1 From config file: /home/user/.config/checkbox.conf
+ local_submission=True (Default)
+ session_desc= (Default)
+ session_title=session title (Default)
+ stock_reports=text, submission_files From config file: /home/user/.config/checkbox.conf
+ [test plan]
+ filter=* (Default)
+ forced=True From config file: /home/user/.config/checkbox.conf
+ unit=com.canonical.certification::TODO From config file: /home/user/.config/checkbox.conf
+ [test selection]
+ exclude= (Default)
+ forced=True From config file: /home/user/.config/checkbox.conf
+ (...)
+ [environment]
+ STRESS_S3_WAIT_DELAY=120 From config file: /var/snap/checkbox/2799/checkbox.conf
+ (...)
+ TUTO=tutorial From config file: /home/user/.config/checkbox.conf
+ (...)
+ No problems with config(s) found!
+
+You can see:
+
+- a list of the configuration files being used
+- for each section, the configured parameters being used
+- the origin of each of these customized parameters
+- an overall status report ("No problems with config(s) found!")
+
+This can be really helpful when debugging a Checkbox run. For instance,
+looking at the output above, I can see that the ``STRESS_S3_WAIT_DELAY``
+environment variable is set to ``120`` because it is specified in
+a Checkbox configuration that comes with the snap version I'm using
+(``/var/snap/checkbox/2799/checkbox.conf``).
+
 Create an executable launcher
 =============================
 
@@ -230,13 +320,13 @@ Make the launcher executable:
 
 .. code-block:: none
 
- chmod +x mylauncher
+ $ chmod +x mylauncher
 
 Run it:
 
 .. code-block:: none
 
- ./mylauncher
+ $ ./mylauncher
 
 Checkbox runs exactly like before! The line we added is called a `shebang
 <https://en.wikipedia.org/wiki/Shebang_(Unix)>`_ and allows us to run