Moved all non-code artifacts required at runtime to src/main/resources to make JAR self-contained; reworked file loading accordingly

Author: xJREB

.gitignore

@@ -16,4 +16,4 @@ production/
 .factorypath
 .class
 bin/
-src/main/java/cli/docs/config.properties
+src/main/resources/config.properties

README.md

@@ -6,7 +6,7 @@ Currently, RESTRuler can parse the following Web API description languages:
 * [OpenAPI v2](https://swagger.io/specification/v2/)
 * [OpenAPI v3](https://github.com/OAI/OpenAPI-Specification)
 
-RESTRuler has been developed in the [Empirical Software Engineering Group](https://www.iste.uni-stuttgart.de/ese) at the University of Stuttgart, Germany, as a research prototype written in Java.
+RESTRuler has been developed in the [Empirical Software Engineering Group](https://www.iste.uni-stuttgart.de/ese) at the University of Stuttgart, Germany, as a research prototype written in Java (version >=18 needed).
 It is a command-line tool that takes the path or URL to an OpenAPI definition file as input and displays a list of design rule violations as output.
 Optionally, a Markdown report file can be generated with additional details and improvement suggestions.
 

build.gradle

@@ -5,7 +5,7 @@ plugins {
     id "com.adarshr.test-logger" version "4.0.0"
 }
 
-version = "2.0.0"
+version = "2.1.0"
 group = "rest-ruler"
 
 repositories {

docs/architecture.md

@@ -89,9 +89,9 @@ A good overview of the implemented rules can be found [here](./rules/README.md).
 
 ## Apache OpenNLP
 - [Apache OpenNLP](https://opennlp.apache.org)
-- [Model](models/en-pos-maxent.bin)
+- [Model](src/main/resources/models/en-pos-maxent.bin)
 
 ## Weka
 
 - [Weka](https://www.cs.waikato.ac.nz/ml/weka/)
-- [Model](models/request_model.dat)
+- [Model](src/main/resources/models/request_model.dat)

docs/rules/implemented-rules/CRUD-function-names-should-not-be-used-in-URIs.md

@@ -49,7 +49,7 @@ The following anti-patterns exemplify what not to do:
 * The parameters in curly brackets are excluded from the path and are therefore currently not checked
 * If more than these previously defined words are to be considered CRUD violations, perform the following steps:
    1. Add the appropriate words to the attribute `CRUD_OPERATIONS` in the `./src/main/java/cli/rule/rules/CRUDRule.java` 
-   2. Mine the words that have the appropriate words as substring from `./src/main/java/cli/docs/wordninja_words.txt` and add them to the list `./src/main/java/cli/docs/CRUD_words.txt` (Exclude similar words to still detect the violation; e.g. "gets" from "get")
+   2. Mine the words that have the appropriate words as substring from `./src/main/resources/wordninja_words.txt` and add them to the list `./src/main/resources/CRUD_words.txt` (Exclude similar words to still detect the violation; e.g. "gets" from "get")
 
 ### Future work
 

docs/rules/implemented-rules/File-extensions-should-not-be-included-in-URIs.md

@@ -27,12 +27,12 @@ On the Web, the period (.) character is commonly used to separate the file name
 
 ### What is checked
 
-The Path is divided into the individual segments. Then, to check if a segment ends with a dot and a file extension (.file-extension), a list [2] of 838 different file extensions is searched. If one of them matches the end of the segment, a rule violation is returned. This list can be extended very easily if file extensions are missed. The list is located here: `./src/main/java/cli/docs/file_extensions.txt`.
+The Path is divided into the individual segments. Then, to check if a segment ends with a dot and a file extension (.file-extension), a list [2] of 838 different file extensions is searched. If one of them matches the end of the segment, a rule violation is returned. This list can be extended very easily if file extensions are missed. The list is located here: `./src/main/resources/file_extensions.txt`.
 
 ### What is not checked
 
 * The path needs to end with .file-extension. If the path does not end with this, the rule is not violated.
-* It is possible that individual file extensions are not recognized because they have not been recorded in the system. To create a rule violation anyway, simply add the non-existing extensions to the list: `./src/main/java/cli/docs/file_extensions.txt`
+* It is possible that individual file extensions are not recognized because they have not been recorded in the system. To create a rule violation anyway, simply add the non-existing extensions to the list: `./src/main/resources/file_extensions.txt`
 
 ### Future work
 

docs/rules/implemented-rules/Request-Methods.md

@@ -32,7 +32,7 @@ Description from Massé [1].
 ### What is checked:
 
 * For each request, we check if the description or summary attribute corresponds to the request type.
-* To implement this, we used Weka [2] to train a Naive Bayes Multinomial classifier (`models/request_model.dat`) that predicts the request type based on the description or the summary. We curated a dataset of over 3k instances for this (`models/request-model-training-data.txt`).
+* To implement this, we used Weka [2] to train a Naive Bayes Multinomial classifier (`src/main/resources/models/request_model.dat`) that predicts the request type based on the description or the summary. We curated a dataset of over 3k instances for this (`model-training/request-model-training-data.txt`).
 * If the prediction doesn't match with the given request type, we throw a violation.
 
 ### What is not checked:

models/dataset/test_data.arff model-training/dataset/test_data.arff

@@ -1,59 +1,66 @@
 package cli.rule;
 
-import cli.rule.rules.SingularDocumentNameRule;
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+import java.util.logging.Level;
+import java.util.logging.Logger;
+import java.util.stream.Collectors;
+import org.apache.commons.lang3.tuple.ImmutablePair;
 import com.google.common.collect.Lists;
+import cli.rule.rules.SingularDocumentNameRule;
 import io.swagger.parser.OpenAPIParser;
 import io.swagger.v3.oas.models.OpenAPI;
 import io.swagger.v3.parser.core.models.SwaggerParseResult;
 import opennlp.tools.postag.POSModel;
 import opennlp.tools.postag.POSTaggerME;
 import opennlp.tools.tokenize.SimpleTokenizer;
-import org.apache.commons.lang3.tuple.ImmutablePair;
-
-import java.io.*;
-import java.nio.charset.Charset;
-import java.nio.file.Files;
-import java.nio.file.Paths;
-import java.util.*;
-import java.util.logging.Level;
-import java.util.logging.Logger;
-import java.util.stream.Collectors;
-import java.util.stream.Stream;
 
 public class Utility {
     private static final Logger logger = Logger.getLogger(Logger.GLOBAL_LOGGER_NAME);
-    private static final String PATH_TO_ENGLISH_DICTIONARY =
-            "src/main/java/cli/docs/wordninja_words.txt";
-    public static final String MODELS_EN_POS_MAXENT_BIN = "models/en-pos-maxent.bin";
+    private static final String PATH_TO_ENGLISH_DICTIONARY = "/wordninja_words.txt";
+
+    public static final String MODELS_EN_POS_MAXENT_BIN = "/models/en-pos-maxent.bin";
 
     private Utility() {
         throw new IllegalStateException("Utility class");
     }
 
     public static boolean getPathSegmentContained(String word, String filePath) {
         boolean isWordInDictionary = false;
-        try (FileReader fileReader = new FileReader(filePath)) {
-            try (Scanner scanner = new Scanner(fileReader)) {
-                while (scanner.hasNext()) {
-                    String wordFromDictionary = scanner.next();
-                    if (word.toLowerCase().contains(wordFromDictionary)) {
-                        isWordInDictionary = true;
-                        break;
-                    }
+        try (InputStream is = Utility.class.getResourceAsStream(filePath);
+                BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
+            String wordFromDictionary;
+            while ((wordFromDictionary = br.readLine()) != null) {
+                if (word.toLowerCase().contains(wordFromDictionary)) {
+                    isWordInDictionary = true;
+                    break;
                 }
             }
         } catch (Exception e) {
-            logger.severe("Error on checking if a word is contained in a dictionary: " + e.getMessage());
+            logger.severe(
+                    "Error on checking if a word is contained in a dictionary: " + e.getMessage());
         }
         return isWordInDictionary;
     }
 
     public static boolean getPathSegmentMatch(String word, String filePath) {
         boolean isWordInDictionary = false;
         try (Scanner scanner = new Scanner(new File(filePath))) {
-            if (scanner.useDelimiter("\\Z").next().matches(word)) isWordInDictionary = true;
+            if (scanner.useDelimiter("\\Z").next().matches(word))
+                isWordInDictionary = true;
         } catch (Exception e) {
-            logger.severe("Error on checking if a word is contained in a dictionary: " + e.getMessage());
+            logger.severe(
+                    "Error on checking if a word is contained in a dictionary: " + e.getMessage());
         }
         return isWordInDictionary;
     }
@@ -65,6 +72,7 @@ public static OpenAPI getOpenAPI(String path) {
 
     /**
      * Method to change the switchPathSegment from plural to singular a vice versa.
+     * 
      * @param equals
      * @return
      */
@@ -78,17 +86,17 @@ public static String getControlPathSegmentForRule(boolean equals) {
 
     /**
      * Get a token from a word using the nlp apache pos tagger library.
+     * 
      * @param pathSegment
      * @return
      */
-    public static String getTokenNLP(String pathSegment){
-        if(pathSegment.equals("")){
+    public static String getTokenNLP(String pathSegment) {
+        if (pathSegment.equals("")) {
             return null;
         }
         SimpleTokenizer tokenizer = SimpleTokenizer.INSTANCE;
         String[] tokens = tokenizer.tokenize(pathSegment);
-        try(InputStream modelIn = new FileInputStream(
-                MODELS_EN_POS_MAXENT_BIN);) {
+        try (InputStream modelIn = Utility.class.getResourceAsStream(MODELS_EN_POS_MAXENT_BIN);) {
             POSModel posModel = new POSModel(modelIn);
             POSTaggerME posTagger = new POSTaggerME(posModel);
             String tags[] = posTagger.tag(tokens);
@@ -102,6 +110,7 @@ public static String getTokenNLP(String pathSegment){
 
     /**
      * Given a word check if it is plural or singular based on token.
+     * 
      * @param token
      * @return
      */
@@ -119,6 +128,7 @@ public static String getTokenFromWord(String token) {
 
     /**
      * Split a string into words if possible using an english dictionary to match the words.
+     * 
      * @param sentence
      * @return
      * @throws IOException
@@ -127,22 +137,27 @@ public static List<String> splitContiguousWords(String sentence) throws IOExcept
         String splitRegex = "[^a-zA-Z0-9']+";
         Map<String, Number> wordCost = new HashMap<>();
         List<String> dictionaryWords = new ArrayList<>();
-        Stream<String> stringLines = null;
+        InputStream is = null;
+
         try {
-            stringLines = Files.lines(Paths.get(PATH_TO_ENGLISH_DICTIONARY), Charset.defaultCharset());
-            dictionaryWords = stringLines.collect(Collectors.toList());
+            is = Utility.class.getResourceAsStream(PATH_TO_ENGLISH_DICTIONARY);
+            BufferedReader br = new BufferedReader(new InputStreamReader(is));
+            dictionaryWords = br.lines().collect(Collectors.toList());
         } catch (Exception e) {
             logger.log(Level.SEVERE, "Error on getting the english dictionary file {e}", e);
         } finally {
-            if (stringLines != null) stringLines.close();
+            if (is != null) {
+                is.close();
+            }
         }
 
         double naturalLogDictionaryWordsCount = Math.log(dictionaryWords.size());
         long wordIdx = 0;
         for (String word : dictionaryWords) {
             wordCost.put(word, Math.log(++wordIdx * naturalLogDictionaryWordsCount));
         }
-        int maxWordLength = Collections.max(dictionaryWords, Comparator.comparing(String::length)).length();
+        int maxWordLength =
+                Collections.max(dictionaryWords, Comparator.comparing(String::length)).length();
         List<String> splitWords = new ArrayList<>();
         for (String partSentence : sentence.split(splitRegex)) {
             splitWords.add(split(partSentence, wordCost, maxWordLength));
@@ -152,12 +167,14 @@ public static List<String> splitContiguousWords(String sentence) throws IOExcept
 
     /**
      * Split a string into sub strings.
+     * 
      * @param partSentence
      * @param wordCost
      * @param maxWordLength
      * @return
      */
-    public static String split(String partSentence, Map<String, Number> wordCost, int maxWordLength) {
+    public static String split(String partSentence, Map<String, Number> wordCost,
+            int maxWordLength) {
         List<ImmutablePair<Number, Number>> cost = new ArrayList<>();
         cost.add(new ImmutablePair<>(0, 0));
         for (int index = 1; index < partSentence.length() + 1; index++) {
@@ -166,7 +183,8 @@ public static String split(String partSentence, Map<String, Number> wordCost, in
         int idx = partSentence.length();
         List<String> output = new ArrayList<>();
         while (idx > 0) {
-            ImmutablePair<Number, Number> candidate = bestMatch(partSentence, cost, idx, wordCost, maxWordLength);
+            ImmutablePair<Number, Number> candidate =
+                    bestMatch(partSentence, cost, idx, wordCost, maxWordLength);
             Number candidateCost = candidate.getKey();
             Number candidateIndexValue = candidate.getValue();
             if (candidateCost.doubleValue() != cost.get(idx).getKey().doubleValue())
@@ -175,7 +193,9 @@ public static String split(String partSentence, Map<String, Number> wordCost, in
             String token = partSentence.substring(idx - candidateIndexValue.intValue(), idx);
             if (token.equals("'") && !output.isEmpty()) {
                 String lastWord = output.get(output.size() - 1);
-                if (lastWord.equalsIgnoreCase("'s") || (Character.isDigit(partSentence.charAt(idx - 1)) && Character.isDigit(lastWord.charAt(0)))) {
+                if (lastWord.equalsIgnoreCase("'s")
+                        || (Character.isDigit(partSentence.charAt(idx - 1))
+                                && Character.isDigit(lastWord.charAt(0)))) {
                     output.set(output.size() - 1, token + lastWord);
                     newToken = false;
                 }
@@ -190,19 +210,22 @@ public static String split(String partSentence, Map<String, Number> wordCost, in
 
     /**
      * Get the best match for a word.
+     * 
      * @param partSentence
      * @param cost
      * @param index
      * @param wordCost
      * @param maxWordLength
      * @return
      */
-    public static ImmutablePair<Number, Number> bestMatch(String partSentence, List<ImmutablePair<Number, Number>> cost,
-                                                    int index, Map<String, Number> wordCost, int maxWordLength) {
-        List<ImmutablePair<Number, Number>> candidates = Lists.reverse(cost.subList(Math.max(0,
-                index - maxWordLength), index));
+    public static ImmutablePair<Number, Number> bestMatch(String partSentence,
+            List<ImmutablePair<Number, Number>> cost, int index, Map<String, Number> wordCost,
+            int maxWordLength) {
+        List<ImmutablePair<Number, Number>> candidates =
+                Lists.reverse(cost.subList(Math.max(0, index - maxWordLength), index));
         int enumerateIdx = 0;
-        ImmutablePair<Number, Number> minPair = new ImmutablePair<>(Integer.MAX_VALUE, enumerateIdx);
+        ImmutablePair<Number, Number> minPair =
+                new ImmutablePair<>(Integer.MAX_VALUE, enumerateIdx);
         for (ImmutablePair<Number, Number> pair : candidates) {
             ++enumerateIdx;
             String subsequence = partSentence.substring(index - enumerateIdx, index).toLowerCase();

models/dataset/test_data.txt model-training/dataset/test_data.txt

@@ -1,28 +1,34 @@
 package cli.rule.rules;
 
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
 import cli.analyzer.RestAnalyzer;
 import cli.rule.IRestRule;
 import cli.rule.Utility;
 import cli.rule.Violation;
-import cli.rule.constants.*;
+import cli.rule.constants.ErrorMessage;
+import cli.rule.constants.RuleCategory;
+import cli.rule.constants.RuleSeverity;
+import cli.rule.constants.RuleSoftwareQualityAttribute;
 import cli.utility.Output;
-
 import io.swagger.v3.oas.models.OpenAPI;
 
-import java.util.*;
-
 /**
  * Implementation of the rule: Underscores (_) should not be used in URI.
  */
 public class CRUDRule implements IRestRule {
     private static final String TITLE = "CRUD function names should not be used in URIs";
     private static final RuleCategory CATEGORY = RuleCategory.URIS;
     private static final RuleSeverity SEVERITY = RuleSeverity.ERROR;
-    private static final List<RuleSoftwareQualityAttribute> SOFTWARE_QUALITY_ATTRIBUTE = Arrays
-            .asList(RuleSoftwareQualityAttribute.USABILITY, RuleSoftwareQualityAttribute.MAINTAINABILITY);
-    private static final String[] CRUD_OPERATIONS = { "get", "post", "delete", "put", "create", "read", "update",
-            "patch", "insert", "select", "fetch", "purge", "retrieve", "add" };
-    private static final String PATH_TO_CRUD_DICTIONARY = "src/main/java/cli/docs/CRUD_words.txt";
+    private static final List<RuleSoftwareQualityAttribute> SOFTWARE_QUALITY_ATTRIBUTE =
+            Arrays.asList(RuleSoftwareQualityAttribute.USABILITY,
+                    RuleSoftwareQualityAttribute.MAINTAINABILITY);
+    private static final String[] CRUD_OPERATIONS = {"get", "post", "delete", "put", "create",
+            "read", "update", "patch", "insert", "select", "fetch", "purge", "retrieve", "add"};
+    private static final String PATH_TO_CRUD_DICTIONARY = "/CRUD_words.txt";
     private final List<Violation> violationList = new ArrayList<>();
     private boolean isActive;
 
@@ -61,8 +67,7 @@ public void setIsActive(boolean isActive) {
     }
 
     /**
-     * Checks if there is a violation against the CRUD rule. All paths and base URLs
-     * are checked.
+     * Checks if there is a violation against the CRUD rule. All paths and base URLs are checked.
      *
      * @param openAPI the definition that will be checked against the rule.
      * @return the list of violations.
@@ -101,16 +106,18 @@ public List<Violation> checkViolation(OpenAPI openAPI) {
      * Checks if the segment contains a CRUD operation
      *
      * @param segment the currently examined segment
-     * @param path    the whole request path
+     * @param path the whole request path
      */
     private void checkCRUDInSegment(String segment, String path) {
         for (String crudOperation : CRUD_OPERATIONS) {
             if (segment.toLowerCase().contains(crudOperation)) {
-                this.violationList.add(new Violation(this, RestAnalyzer.locMapper.getLOCOfPath(path), "URIS should not be " +
-                        "used " + "to " + "indicate that a CRUD function (" + crudOperation.toUpperCase() + ") is "
-                        + "performed, " + "instead HTTP request methods should be used for this.", path,
-                        ErrorMessage.CRUD));
+                this.violationList.add(new Violation(this,
+                        RestAnalyzer.locMapper.getLOCOfPath(path),
+                        "URIS should not be " + "used " + "to " + "indicate that a CRUD function ("
+                                + crudOperation.toUpperCase() + ") is " + "performed, "
+                                + "instead HTTP request methods should be used for this.",
+                        path, ErrorMessage.CRUD));
             }
         }
     }
-}
+}