[FLOSS P3] Kdb validate

doc/decisions/error_codes.md

@@ -58,7 +58,7 @@ Various projects and standards:
   Currently they have 43 classes which all come from SQLSTATE. Postgres also throws additional errors but have to subclass it to one of the current 43 classes and have a special naming convention which have to start with a `P` in the subclass.
 - [etcd](https://github.com/etcd-io/etcd):
   Etcd's approach for errors are tightly coupled to the programming language Go as well as the [gRPC](https://grpc.io/) standard which currently has
-  [16 codes](https://godoc.org/google.golang.org/grpc/codes) defined. Some of these errors are similar or identical to those which will be used in elektra.
+  [16 codes](https://pkg.go.dev/google.golang.org/grpc/codes?utm_source=godoc) defined. Some of these errors are similar or identical to those which will be used in elektra.
   Every error of etcd is associated with one of these categories and gets its own error message which is specified in [this](https://github.com/etcd-io/etcd/blob/master/etcdserver/api/v3rpc/rpctypes/error.go) file. This concept though does not allow easy subclassing which might be useful (eg. further split FailedPrecondition into more specific errors like semantic and syntactic errors)
 - [Windows Registry](https://docs.microsoft.com/en-us/windows/desktop/sysinfo/registry):
   The registry does not use any specific error concept but takes the standard [Win32 Error Codes](https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/18d8fbe8-a967-4f1c-ae50-99ca8e491d2d). These are neither hierarchical nor have any special ordering. Basically it is the same as elektra has now except for no duplicated

doc/help/kdb-validate.md

@@ -0,0 +1,39 @@
+# kdb-validate(1) - Validate key values
+
+## SYNOPSIS
+
+`kdb validate`
+
+## DESCRIPTION
+
+Validate the values of string keys below a given name using the loaded validation plugins (eg. range or validation) by reading all values, making them dirty by changing to another value, changing back to original and then writing that back to the key database.
+
+This command is useful for validating configuration files against
+their specifications.
+
+For keys to be validated, they must contain the 'check'-metakeys
+and the respective plugins for validation must be loaded
+for the backend that was used while mounting.
+If a validation is done while using `kdb set` or `kdb get`
+the same validation is also done by `kdb validate`
+Only string keys are validated! Binary keys are skipped!
+
+Use `-f` to do a write-test even if the previous read
+from the key database has issued warnings.
+
+## OPTIONS
+
+- `-d`,`--debug`:
+  Give debug information.
+- `-f`, `--force`:
+  Force writing the configuration even on warnings.
+- `-H`, `--help`:
+  Show the man page.
+- `-p <name>`, `--profile <name>`:
+  Use a different profile for kdb configuration.
+- `-v`, `--verbose`:
+  Explain what is happening.
+- `-V`, `--version`:
+  Print version info.
+- `-C <when>`, `--color <when>`:
+  Print `never/auto(default)/always` colored output.

doc/man/man1/kdb-validate.1

@@ -0,0 +1,38 @@
+.\" generated with Ronn-NG/v0.10.1
+.\" http://github.com/apjanke/ronn-ng/tree/0.10.1.pre1
+.TH "KDB\-VALIDATE" "1" "January 2022" ""
+.SH "NAME"
+\fBkdb\-validate\fR \- Validate key values
+.SH "SYNOPSIS"
+\fBkdb validate\fR
+.SH "DESCRIPTION"
+Validate the values of string keys below a given name using the loaded validation plugins (eg\. range or validation) by reading all values, making them dirty by changing to another value, changing back to original and then writing that back to the key database\.
+.P
+This command is useful for validating configuration files against their specifications\.
+.P
+For keys to be validated, they must contain the 'check'\-metakeys and the respective plugins for validation must be loaded for the backend that was used while mounting\. If a validation is done while using \fBkdb set\fR or \fBkdb get\fR the same validation is also done by \fBkdb validate\fR Only string keys are validated! Binary keys are skipped!
+.P
+Use \fB\-f\fR to do a write\-test even if the previous read from the key database has issued warnings\.
+.SH "OPTIONS"
+.TP
+\fB\-d\fR,\fB\-\-debug\fR
+Give debug information\.
+.TP
+\fB\-f\fR, \fB\-\-force\fR
+Force writing the configuration even on warnings\.
+.TP
+\fB\-H\fR, \fB\-\-help\fR
+Show the man page\.
+.TP
+\fB\-p <name>\fR, \fB\-\-profile <name>\fR
+Use a different profile for kdb configuration\.
+.TP
+\fB\-v\fR, \fB\-\-verbose\fR
+Explain what is happening\.
+.TP
+\fB\-V\fR, \fB\-\-version\fR
+Print version info\.
+.TP
+\fB\-C <when>\fR, \fB\-\-color <when>\fR
+Print \fBnever/auto(default)/always\fR colored output\.
+

doc/man/man7/elektra-plugins.7

@@ -1,6 +1,6 @@
 .\" generated with Ronn-NG/v0.10.1
 .\" http://github.com/apjanke/ronn-ng/tree/0.10.1.pre1
-.TH "ELEKTRA\-PLUGINS" "7" "July 2021" ""
+.TH "ELEKTRA\-PLUGINS" "7" "January 2022" ""
 .SH "NAME"
 \fBelektra\-plugins\fR \- plugins overview
 .P
@@ -94,7 +94,7 @@ augeas \fIaugeas/\fR reads/writes many different configuration files using the A
 .P
 Using semi\-structured data for config files, mainly suitable for spec\-namespace (put a focus on having nice syntax for metadata):
 .IP "\(bu" 4
-ni \fIni/\fR parses INI files based on (including metadata) ni \fIhttps://lab\.burn\.capital/chaz\-attic/bohr/\-/blob/main/include/bohr/ni\.h\fR\.
+ni \fIni/\fR parses INI files based on (including metadata)
 .IP "" 0
 .P
 Only suited for import/export:

doc/news/_preparation_next_release.md

@@ -150,7 +150,7 @@ _(Michael Tucek)_
 
 ## Tools
 
-- <<TODO>>
+- Implement `kdb validate <key>`, collect warnings and errors while kdb.get() and kdb.set(), see #3674 _(@flo91)_, _(@JakobWonisch)_
 - <<TODO>>
 - <<TODO>>
 

doc/tutorials/highlevel-bindings.md

@@ -56,7 +56,7 @@ direct mapping, significantly simplifies the generated code (and maybe also the
 
 ### Bindings for the `lowlevel` Library
 
-Your programming language of choice must provide a way to call into C code (like [cgo](https://golang.org/cmd/cgo/)).
+Your programming language of choice must provide a way to call into C code (like [cgo](https://pkg.go.dev/cmd/cgo/)).
 
 In general we prefer (in this order):
 

doc/tutorials/install-webui.md

@@ -12,7 +12,7 @@ Elektra-web requires:
 
 - [Elektra](https://libelektra.org/) with the [`yajl` plugin](https://master.libelektra.org/src/plugins/yajl/) installed
 - A recent [node.js](https://nodejs.org/en/) installation (at least 6.x)
-- [Go](https://golang.org/) with version > 1.13
+- [Go](https://go.dev/) with version > 1.13
 
 ## Building with elektra-web Tool
 

doc/tutorials/validation.md

@@ -301,3 +301,35 @@ rm $(kdb get system:/tests/userbackup)
 kdb rm system:/tests/specbackup
 kdb rm system:/tests/userbackup
 ```
+
+## Validate Existing Keys
+
+To check if an existing set of keys can be read and written with the current validation rules `kdb validate` should be used. Validate will read the values of all string keys under the point defined as argument in the command line, sets the key value to something different, then back to the original and finally writes that original value back to the key database. All loaded [validation plugins](/src/plugins/README.md) are now used to validate the values of keys with the necessary meta-keys (see above).
+
+Only string keys are validated! Binary keys are skipped!
+
+```sh
+# mount test config file and set a value
+sudo kdb mount range.ecf /tests/range range dump
+
+# set value
+kdb set user:/tests/range/value 5
+
+# add range check to all keys under /tests/range/
+kdb meta-set spec:/tests/range/_ check/range "1-10"
+
+# check if validate passes
+kdb validate /tests/range
+
+# set new key to invalid value (with kdb set -f)
+kdb set -f user:/tests/range/value2 11
+
+# validation fails now
+kdb validate /tests/range
+# RET:1
+
+# clean up
+kdb rm -r /tests/range/
+sudo kdb umount /tests/range
+
+```

src/libs/elektra/symbols.map

@@ -134,6 +134,27 @@ libelektra_1.0 {
 	ELEKTRA_WARNING_VALIDATION_SEMANTIC;
 	ELEKTRA_WARNING_VALIDATION_SYNTACTIC;
 
+	# export also names of errors and warnings
+	ELEKTRA_ERROR_CONFLICTING_STATE_NAME;
+	ELEKTRA_ERROR_INSTALLATION_NAME;
+	ELEKTRA_ERROR_INTERFACE_NAME;
+	ELEKTRA_ERROR_INTERNAL_NAME;
+	ELEKTRA_ERROR_OUT_OF_MEMORY_NAME;
+	ELEKTRA_ERROR_PLUGIN_MISBEHAVIOR_NAME;
+	ELEKTRA_ERROR_RESOURCE_NAME;
+	ELEKTRA_ERROR_VALIDATION_SEMANTIC_NAME;
+	ELEKTRA_ERROR_VALIDATION_SYNTACTIC_NAME;
+
+	ELEKTRA_WARNING_CONFLICTING_STATE_NAME;
+	ELEKTRA_WARNING_INSTALLATION_NAME;
+	ELEKTRA_WARNING_INTERFACE_NAME;
+	ELEKTRA_WARNING_INTERNAL_NAME;
+	ELEKTRA_WARNING_OUT_OF_MEMORY_NAME;
+	ELEKTRA_WARNING_PLUGIN_MISBEHAVIOR_NAME;
+	ELEKTRA_WARNING_RESOURCE_NAME;
+	ELEKTRA_WARNING_VALIDATION_SEMANTIC_NAME;
+	ELEKTRA_WARNING_VALIDATION_SYNTACTIC_NAME;
+
 	# kdb.h
 	elektraGOptsContract;
 	elektraGOptsContractFromStrings;

src/libs/tools/include/CMakeLists.txt

@@ -5,6 +5,13 @@ install (
 	DESTINATION include/${TARGET_INCLUDE_FOLDER}
 	COMPONENT libelektra-dev)
 
+file (GLOB ERROR_HDR_FILES errors/*.hpp)
+
+install (
+	FILES ${ERROR_HDR_FILES}
+	DESTINATION include/${TARGET_INCLUDE_FOLDER}/errors
+	COMPONENT libelektra-dev)
+
 file (GLOB HELPER_HDR_FILES helper/*.hpp)
 
 install (

src/libs/tools/include/errors/baseNotification.hpp

@@ -0,0 +1,107 @@
+#ifndef ELEKTRA_BASENOTIFICATION_HPP
+#define ELEKTRA_BASENOTIFICATION_HPP
+
+#include <kdbtypes.h>
+#include <key.hpp>
+#include <string>
+#include <utility>
+
+namespace kdb
+{
+namespace tools
+{
+namespace errors
+{
+/* Common abstract class for warnings and errors.
+ * Because warning and errors share the same data members,
+ * a method can accept a BaseNotification argument and
+ * the caller can create an Error or a Warning
+ * based on the provided object. */
+class BaseNotification
+{
+public:
+	/* constructor */
+	BaseNotification (std::string reason, std::string module, std::string file, std::string mountPoint, std::string configFile,
+			  kdb::long_t line);
+
+	/* setters */
+	void setData (const std::string & reason, const std::string & module, const std::string & file, const std::string & mountPoint,
+		      const std::string & configFile, kdb::long_t line);
+
+	/* get references (for setting and getting member values) */
+	std::string & reason ();
+	std::string & module ();
+	std::string & file ();
+	std::string & mountPoint ();
+	std::string & configFile ();
+	kdb::long_t & line ();
+	const std::string & reason () const;
+	const std::string & module () const;
+	const std::string & file () const;
+	const std::string & mountPoint () const;
+	const std::string & configFile () const;
+	const kdb::long_t & line () const;
+
+	/* fixed values per Class, taken from C-makro definitions in /src/include/kdberrors.h */
+	virtual std::string code () const = 0;
+	virtual std::string description () const = 0;
+
+	/* string representation */
+	friend std::ostream & operator<< (std::ostream & outputStream, const BaseNotification & eb);
+
+	/**
+	 * @brief Compare fields of notification objects
+	 *
+	 * Also incorporates the compare method to enable subclasses to add constraints to the comparison.
+	 *
+	 * @param other the notification to compare
+	 *
+	 * @return true if objects are equal
+	 */
+	bool operator== (const BaseNotification & other) const;
+	bool operator!= (const BaseNotification & other) const;
+
+protected:
+	BaseNotification () = default;
+
+	/**
+	 * @brief Compare to another notification object
+	 *
+	 * Is used in operator==.
+	 * Can be overloaded by subclasses to check additional constraints.
+	 * At least the types of the two objects that get compared should be checked for equality!
+	 *
+	 * @param other the notification to compare to
+	 *
+	 * @return true if objects are equal
+	 */
+	virtual bool compare (const BaseNotification & other) const;
+
+	/* Can be overwritten by subclasses to change the text representation */
+
+	/**
+	 * @brief Get a text representation of the notification.
+	 *
+	 * Is used in operator<<.
+	 * Can be overloaded by subclasses to append additional text.
+	 *
+	 * @param outputStream The stream to append the text to,
+	 * used by the operator `<<`
+	 *
+	 * @return The given stream with additional text appended.
+	 */
+	virtual std::ostream & toString (std::ostream & outputStream) const;
+
+private:
+	std::string m_reason;
+	std::string m_module;
+	std::string m_file;
+	std::string m_mountPoint;
+	std::string m_configFile;
+	kdb::long_t m_line = 0;
+};
+
+} // namespace errors
+} // namespace tools
+} // namespace kdb
+#endif // ELEKTRA_BASENOTIFICATION_HPP