Running a Subset of the Test Suite (#544)

.github/workflows/full-test.yaml

@@ -33,5 +33,5 @@ jobs:
       timeout-minutes: 25
       r-version: ${{ matrix.r-version }}
       image: ${{ matrix.os }}
-      action: test-full
+      action: test-full -- --forbid-only
       coverage: false

.github/workflows/qa.yaml

@@ -54,7 +54,7 @@ jobs:
     if: ${{ github.event_name != 'pull_request' || github.event.pull_request.base.ref != 'main' }}
     uses: Code-Inspect/flowr/.github/workflows/run.yaml@main
     with:
-      action: test-full
+      action: test-full -- --forbid-only
       coverage: true
 
 

wiki/Capabilities.md

@@ -131,7 +131,6 @@ and the grouping with `{`<sup>T61</sup> and `}`<sup>T62</sup>, the normalization
 
 <li> <details> <summary><b>Separators</b>&emsp;(full)</summary>
 
-
 Separators help R to distinguish between otherwise ambiguous expressions
 like `a <- 1` and `a < -1`. They are partially defined by [Section 10.3.5](https://cran.r-project.org/doc/manuals/r-release/R-lang.html#Separators),
 with more separator tokens on the [Tokens](https://github.com/Code-Inspect/flowr/wiki/Tokens#tokens-used-to-delimit-parts-of-expressions) wiki page. The most common
@@ -142,7 +141,6 @@ this is [discouraged](https://style.tidyverse.org/syntax.html#semicolons), and n
 Nevertheless, the normalization of *flowR* does handle the use of semicolons, as well as `,`<sup>T59</sup>, in combination with arguments and parameters.
 </details></li>
 
-
 </ol>
 
 ### Values
@@ -274,13 +272,12 @@ special way (although we prevent assignments and references, similar to
 our handling of `NULL`).
 </details></li>
 
-
 </ol>
 
 ### Data Types
 
 R is dynamically typed and assigns a type to each R object, which can be
-queried at runtime with the typeof function. The list of existing types is
+queried at runtime with the `typeof` function. The list of existing types is
 documented at the beginning of [Section 2](https://cran.r-project.org/doc/manuals/r-release/R-lang.html#Objects).
 These types include symbols, primitive types like characters, booleans (logical values), integers,
 doubles, complex numbers, and more. Since the intricate details of these
@@ -420,7 +417,6 @@ corresponding change in the token type from `EQ_ASSIGN` to `EQ_SUB`
 and `EQ_FORMALS`.
 For another important differentiation when used in arguments, we recommend Section&nbsp;8.2.26 of the great [The R Inferno](https://www.burns-stat.com/pages/Tutor/R_inferno.pdf) by Burns.
 
-
 Besides using `=`, there is the older variant of the local equal assignment
 using an additional colon: `:=`. While (from [lintr](https://github.com/r-lib/lintr/blob/3d9e6d78efe7fc41d8b545a845a984a0821cfbbe/R/assignment_linter.R#L95-L96))
 > it’s [`:=`] extremely uncommon as a normal assignment operator

wiki/Core.md

@@ -32,4 +32,3 @@ If you want to execute a single step (for whatever reason) there are two ways to
 #### Step 4: Slicing on the Dataflow Graph
 
 #### Step 5: Reconstruct the R Code
-

wiki/Home.md

@@ -1,6 +1,5 @@
 [![flowR Logo](img/flowR.png)](https://github.com/Code-Inspect/flowr)
 
-
 ***This wiki page is currently under construction***
 
 *flowR* is a static [dataflow analyzer](https://en.wikipedia.org/wiki/Data-flow_analysis) and [program slicer](https://github.com/Code-Inspect/flowr/wiki/Terminology#program-slice) for the [*R*](https://www.r-project.org/) programming language (currently tested for versions `4.x`), and you have stumbled upon its wiki pages!

wiki/Interface.md

@@ -693,7 +693,6 @@ Within the started session, type the following message and press enter to see th
 {"type": "request-file-analysis","id":"0","filetoken":"x","content":"x <- 1\nx + 1"}
 ```
 
-
 ##### Using Python
 
 In python, a similar process would look like this.
@@ -735,6 +734,7 @@ sequenceDiagram
     end
     deactivate  Server
 ```
+
 </details>
 
 In order to slice, you have to send a file analysis request first. The `filetoken` you assign is of use here as you can re-use it to repeatedly slice the same file.
@@ -953,7 +953,6 @@ With a shell object (let's call it `shell`), you can execute R code by using `RS
 
 Besides that, the command `RShell::tryToInjectHomeLibPath` may be of interest, as it enables all libraries available on the host system.
 
-
 ### Slicing With The `SteppingSlicer`
 
 The main class that represents *flowR*'s slicing is the [`SteppingSlicer`](https://code-inspect.github.io/flowr/doc/classes/src_core_slicer.SteppingSlicer.html) class. With *flowR*, this allows you to slice code like this:
@@ -1095,7 +1094,6 @@ Whenever this name appears, you may substitute this with whatever name fits your
 3. **Add it to the feature-mapping**\
    Now, in the `feature.ts` file in `src/statistics/features`, add your feature to the `ALL_FEATURES` object.
 
-
 Now, we want to extract something. For the *example* feature created in the previous steps, we choose to count the amount of `COMMENT` tokens.
 So we define a corresponding [XPath](https://developer.mozilla.org/en-US/docs/Web/XPath) query:
 

wiki/Linting and Testing.md

@@ -4,6 +4,7 @@ For the latest code-coverage information, see [codecov.io](https://codecov.io/gh
   - [Functionality Tests](#functionality-tests)
     - [Test Structure](#test-structure)
     - [Writing a Test](#writing-a-test)
+    - [Running Only Some Tests](#running-only-some-tests)
   - [Performance Tests](#performance-tests)
 - [CI Pipeline](#ci-pipeline)
 - [Linting](#linting)
@@ -30,7 +31,7 @@ What may be counter-intuitive is that this does not run *all* tests by default b
 npm run test-full
 ```
 
-However, depending on your local R version, your network connection and potentially other factors, some tests may be skipped automatically as they do not apply to your current system setup (or can't be tested with the current prerequisites). Each test can speciy such requirements as part of the `TestConfiguration`, which is then enforced by the [`ensureConfig`](https://github.com/Code-Inspect/flowr/blob/main/test/functionality/_helper/shell.ts) function.
+However, depending on your local R version, your network connection and potentially other factors, some tests may be skipped automatically as they do not apply to your current system setup (or can't be tested with the current prerequisites). Each test can specify such requirements as part of the `TestConfiguration`, which is then enforced by the [`ensureConfig`](https://github.com/Code-Inspect/flowr/blob/main/test/functionality/_helper/shell.ts) function.
 It is up to the [ci](#ci-pipeline) to run the tests on different systems to ensure that those tests are ensured to run.
 
 #### Test Structure
@@ -49,7 +50,12 @@ Besides folders can (theoretically) arbitrarily structure their tests. We use th
 
 #### Writing a Test
 
-Currently this is heavily dependend on what you want to test (normalization, dataflow, quad-export, ...) and it is probably best to have a look at existing tests in that area to get an idea of what comfort functionality is available.
+Currently this is heavily dependent on what you want to test (normalization, dataflow, quad-export, ...) and it is probably best to have a look at existing tests in that area to get an idea of what comfort functionality is available.
+
+#### Running Only Some Tests
+
+To run only some of the tests, the mocha test runner allows several options.
+You can specify so [in the code](https://mochajs.org/#exclusive-tests) or [via the command line](https://mochajs.org/#-grep-regexp-g-regexp). If you modify the code, please make sure to remove the changes before committing!
 
 ### Performance Tests
 
@@ -70,7 +76,7 @@ We have several workflows defined in [.github/workflows](../.github/workflows/)
 We explain the most important workflows in the following:
 
 - [run.yaml](../.github/workflows/run.yaml) is a [reusable workflow](https://docs.github.com/en/actions/using-workflows/reusing-workflows) that we use *whenever* we want to run a *flowR* npm script in the pipeline (furthermore, this hides necessary extra steps for coverage, documentation, and benchmarks).
-- [qa.yaml](../.github/workflows/qa.yaml) is the main workflow which will run different steps depending on several factors. It is repsonsible for:
+- [qa.yaml](../.github/workflows/qa.yaml) is the main workflow which will run different steps depending on several factors. It is responsible for:
   - running the [functionality](#functionality-tests) and [performance tests](#performance-tests)
     - uploading the results to the [benchmark page](https://code-inspect.github.io/flowr/wiki/stats/benchmark) for releases
     - running the [functionality tests](#functionality-tests) on different operating systems (Windows, macOS, Linux) and with different versions of R

wiki/Overview.md

@@ -60,7 +60,6 @@ The [core](https://github.com/Code-Inspect/flowr/tree/main/src/core) module cont
 
 The [utility](https://github.com/Code-Inspect/flowr/tree/main/src/util) module is of no further interest for the usage of *flowR*
 
-
 The following sections explain how to use these features.
 
 ## Using *flowR*
@@ -129,7 +128,6 @@ Sleep 200ms
 
 The server allows to access the REPL as well (see the [interface](https://github.com/Code-Inspect/flowr/wiki/Interface) wiki page for more information).
 
-
 ## Calling the Scripts Directly
 
 This describes the old way of using *flowR* by creating and calling the respective scripts directly. Although this is no longer necessary, the scripts still remain, fully integrated into the REPL of *flowR* (you can access them simply by adding a colon `:` before the name).
@@ -216,7 +214,6 @@ See
 npm run stats -- --help
 ```
 
-
 ### Export RDF N-Quads
 
 If you know what the RDF N-Quads refer to, then you are good to go! If not, you do not have to worry - you probably do not need them (they are for a graph search that is based on *flowR*). See the following for more information:

wiki/Setup.md

@@ -25,7 +25,6 @@ This should drop you into *flowR*'s read-evaluate-print loop.
 Enter `:help` to receive more information and `:quit` to leave.
 Please remember, that you have to link external directories to make them available within the running container:
 
-
 ## ⚒️ Building From Scratch
 
 To use *flowR*, you need [*R*](https://www.r-project.org/) installed and on your path.
@@ -55,7 +54,6 @@ for(i in 1:(N-1)) product <- product * i
 cat("Product:", product, "\n")
 ```
 
-
 At the time of writing this, there is currently no page for frequently encountered errors. So just [message me](mailto:florian.sihler@uni-ulm.de) in case of problems.
 
 ## 📜 Developing for *flowR*

wiki/Terminology.md

@@ -1,6 +1,5 @@
 Based on the explanations in my [master's thesis](http://dx.doi.org/10.18725/OPARU-50107), this page collects the definitions for terms which are of interest in the context of *flowR*.
 
-
 <!-- TOC -->
 - [Program Slicing](#program-slicing)
   - [Slicing Criterion](#slicing-criterion)
@@ -13,7 +12,6 @@ Based on the explanations in my [master's thesis](http://dx.doi.org/10.18725/OPA
     - [Dynamic](#dynamic)
 <!-- TOC -->
 
-
 ## Program Slicing
 
 For more information on different slicing variants, refer to Section&nbsp;2.1.3 of my [master's thesis](http://dx.doi.org/10.18725/OPARU-50107).
@@ -36,7 +34,6 @@ Weiser originally defined it as a combination of a line number and a set of vari
 
 Furthermore, slicing criteria can be joined by the use of a `;`-semicolon. `10@sum;12@product` refers to the first occurrence of `sum` in line 10 and the first occurrence of `product` in line 12 as two variables to be used for slicing.
 
-
 ### Program Slice
 
 Based on a slicing criterion and a program, a program slice is a subset of the program that either
@@ -64,7 +61,7 @@ print(x + z)
 
 Let's slice for the last definition of `x` in line 9 (`x <- y`).
 
-* the smallest [backward slice](#backward-slice) is:
+- the smallest [backward slice](#backward-slice) is:
 
     ```R
     if(x > 4) {
@@ -79,17 +76,14 @@ Let's slice for the last definition of `x` in line 9 (`x <- y`).
     It does not contain `z <- 3` as it is not necessary to define `x` in the last line, however it includes the if's condition, as it decides the active branch and hence the active definition for `y` in the definition.
     If `x <- y` itself is to be included in the slice depends on the program slicer. *flowR* includes it.
 
-
-* the smallest [forward slice](#forward-slice) is:
+- the smallest [forward slice](#forward-slice) is:
 
     ```R
     print(x + z)
     ```
 
   It contains all statements affected by the definition of `x`. *flowR* does not support forward slicing in its current form.
 
-
-
 ### Slicing Direction
 
 #### Backward Slice
@@ -102,7 +96,6 @@ See the explanation of [program slices](#program-slice) for an example.
 Based on a [slicing criterion](#slicing-criterion) and a program, a forward slice is a subset of the program that might be influenced by the [slicing criterion](#slicing-criterion).
 See the explanation of [program slices](#program-slice) for an example.
 
-
 ### Information Used for Slicing
 
 #### Static

wiki/Thesis.md

@@ -23,7 +23,7 @@ The (sorted) results should be versioned and can be found in [scripts/top-r-down
     cd scripts/ && Rscript top-downloads.R
     ```
 
-1. If you haven't done so already, or updated the package list in the previous step, **download** the package sources.
+2. If you haven't done so already, or updated the package list in the previous step, **download** the package sources.
    For this, you can use the [scripts/download-top-pkg-sources.R](../scripts/download-top-pkg-sources.R) script.
    But **make sure**, that you set the `to` variable to the output path you want.
 
@@ -33,13 +33,13 @@ The (sorted) results should be versioned and can be found in [scripts/top-r-down
 
    Downloading and extracting the sources can take a while.
 
-2. Make sure you have the latest version of the *flowr* package installed.
+3. Make sure you have the latest version of the *flowr* package installed.
 
    ```shell
    npm ci
    ```
 
-3. Now you can run the statistics program on the downloaded sources.
+4. Now you can run the statistics program on the downloaded sources.
    You can do this in two ways (check `npm run stats -- --help` for more information).
    In any case, the extraction may take a long time, so be prepared for that!
    Furthermore, you may want to store the output of the tool as it provides additional information.
@@ -63,7 +63,7 @@ The (sorted) results should be versioned and can be found in [scripts/top-r-down
    Theoretically, you should be able to stop the extraction at any time and still get usable information with the next step,
    of course limited to only those files that have been processed so far.
 
-4. Afterward, your output folder should contain several folders with the recorded stats of all extracted features.
+5. Afterward, your output folder should contain several folders with the recorded stats of all extracted features.
    To make sense of them, you need to use the post-processor, which prints the summarized information to the command-line:
 
    ```shell