From 43d61aa22d2a901ef87e679006593f00734e7d74 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:10:16 +0100 Subject: [PATCH 01/50] doc: add quick link --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index aad6c4cac67..9b3cf71b974 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,7 @@ applications' configurations, leveraging easy application integration. ## Often Used Links +- If you are new, start reading [Get Started](doc/GETSTARTED.md) - [Build server](https://build.libelektra.org/) - [Website](https://www.libelektra.org) - [API documentation](https://doc.libelektra.org/api/latest/html/) From 37a77f508691e209cee1af0fe7b94d03fc4ba096 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:11:00 +0100 Subject: [PATCH 02/50] doc: small improvements in get started --- doc/GETSTARTED.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/doc/GETSTARTED.md b/doc/GETSTARTED.md index 8200abd0e5d..9b71ddc8b23 100644 --- a/doc/GETSTARTED.md +++ b/doc/GETSTARTED.md @@ -59,10 +59,14 @@ cmake .. # watch output to see if everything needed is included # optionally run "ccmake .." to get an overview of the available build settings (needs cmake-curses-gui) cmake --build build -- -j5 - cmake --build build --target run_nokdbtests #optional: run tests ``` - With these commands you will be able to run the "Hello World!" example but usually you will need to use some of the [plugins](/src/plugins/README.md), tools and bindings of Elektra. Please take a look to the more detailed [compiling documentation](/doc/COMPILE.md). After you completed building Elektra on your own, you can execute these commands to install Elektra (please check the [installation documentation](/doc/INSTALL.md) if you are using Debian): + Optional you can also run tests, see [here for more information](/doc/TESTING.md): + ```sh + cmake --build build --target run_nokdbtests + ``` + + With these commands you will be able to run the "Hello World!" example but usually you will need to use some of the [plugins](/src/plugins/README.md), tools and bindings of Elektra. Please take a look to the more detailed [compiling documentation](/doc/COMPILE.md). After you completed building Elektra on your own, you can execute these commands to install Elektra (please check the [installation documentation](/doc/INSTALL.md) for the many available packages): ```sh sudo make install @@ -71,7 +75,12 @@ [Installation documentation](/doc/INSTALL.md) contains further information about available packages. + Optional you can also run tests to verify the installed Elektra, see [here for more information](/doc/TESTING.md): + ```sh + kdb run_nokdbtests + ``` + - Hello World! - Start with your very first Elektra application and follow these steps: + Start with your very first Elektra application in C and follow these steps: [Hello World!](/doc/tutorials/hello-elektra.md) From f0a6fc217b48ee3dbef65c7e1268c66093d1d4cc Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:11:35 +0100 Subject: [PATCH 03/50] doc: swap hello C and application integration tut --- doc/tutorials/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/tutorials/README.md b/doc/tutorials/README.md index b93e8d4e71b..fcb24da7739 100644 --- a/doc/tutorials/README.md +++ b/doc/tutorials/README.md @@ -19,8 +19,8 @@ application, that means, you want your application to participate in the global key database Elektra provides. -- [Integration of your Application](application-integration.md) -- [Hello, Elektra](hello-elektra.md) +- [Hello, Elektra in C](hello-elektra.md) +- [Integration of your C Application](application-integration.md) - [Plugins Introduction](plugins.md) - [Storage Plugins](storage-plugins.md) - [Compilation Variants of plugins](compilation-variants.md) (advanced topic) From 1f58971cf781ee53be664ab147c5c8b259f5422f Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:11:54 +0100 Subject: [PATCH 04/50] cmake: ignore two variables (showing up on my system) --- scripts/cmake/ElektraCache.cmake | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/cmake/ElektraCache.cmake b/scripts/cmake/ElektraCache.cmake index b9329ec5e56..2d2a47f6f7d 100644 --- a/scripts/cmake/ElektraCache.cmake +++ b/scripts/cmake/ElektraCache.cmake @@ -423,6 +423,8 @@ mark_as_advanced ( pkgcfg_lib__LIBSYSTEMD_PC_syst pkgcfg_lib__OPENSSL_crypto pkgcfg_lib__OPENSSL_ssl + pkgcfg_lib_PC_yaml-cpp_yaml-cpp + pkgcfg_lib__LIBSYSTEMD_PC_systemd SWIG_DIR SWIG_VERSION gtest_build_samples From cf07d1ed03618b2395839b9266be49988a6da82c Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:28:20 +0100 Subject: [PATCH 05/50] doc: write release notes --- doc/news/_preparation_next_release.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/news/_preparation_next_release.md b/doc/news/_preparation_next_release.md index 35c0bed65c8..f8cc4c9519c 100644 --- a/doc/news/_preparation_next_release.md +++ b/doc/news/_preparation_next_release.md @@ -147,7 +147,7 @@ you up to date with the multi-language support provided by Elektra. ## Documentation -- <> +- Finalize 1.0 decisions. _(Markus Raab)_ - <> - <> From 330eef557f5cb161d94e2ec898dac6cd8f097f1e Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:31:29 +0100 Subject: [PATCH 06/50] doc: decision for vendor_spec --- doc/decisions/vendor_spec.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/decisions/vendor_spec.md b/doc/decisions/vendor_spec.md index 793df8eba1f..3336fae9130 100644 --- a/doc/decisions/vendor_spec.md +++ b/doc/decisions/vendor_spec.md @@ -22,7 +22,10 @@ and being administer friendly. ## Decision -Provide means that a single specification can satisfy every distribution and administrator. +As found out during implementation of [specload](/src/plugins/specload), only a very limited subset can be modified safely, e.g.: + +- add/edit/remove `description`, `opt/help` and `comment` +- add/edit `default` ## Rationale From df569afc02d9f1ddbfa7e23b85a323bacb275bbf Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:33:38 +0100 Subject: [PATCH 07/50] doc: as discussed in meeting --- doc/decisions/semantics_name.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index 115ccd9da27..6172732d335 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -19,6 +19,13 @@ The alternative would be to have semantics in key names, with following advantag Do not encode any semantics into the key names. All semantic must be in metadata. +Nevertheless, there are guidelines (without any checks in `keySetBaseName`): + +- `#` is used to indicate that array numbers follow. +- `®` is used to indicate that some information was encoded in the key name. + This is usually only needed internally in storage plugins. +- `®elektra` is reserved, key names should not start with that sequence. + There are, however, rules and conventions which syntax to use for specific semantics. The `spec` plugin guards these rules. From 3308e72f953048f6bc2043a745c373c4004eb7e5 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:34:29 +0100 Subject: [PATCH 08/50] doc: change decision as discussed --- doc/decisions/holes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/holes.md b/doc/decisions/holes.md index 4c18232054a..bd77a072167 100644 --- a/doc/decisions/holes.md +++ b/doc/decisions/holes.md @@ -37,7 +37,7 @@ Support holes and values for non-leaves in a KeySet if the underlying format all If the underlying format does not support it and there is also not an obvious way how to circumvent it -- e.g., JSON which does not have comments -- holes and -values in non-leaves are not supported. +values in non-leaves can be supported with key names starting with ®elektra. ## Rationale From 3950728e1bf982dfcaf9fece00974dd17f9a346e Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 13:37:18 +0100 Subject: [PATCH 09/50] doc: update global_validation.md as discussed --- doc/decisions/global_validation.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/decisions/global_validation.md b/doc/decisions/global_validation.md index 698fc580d7a..cbc1abe05b4 100644 --- a/doc/decisions/global_validation.md +++ b/doc/decisions/global_validation.md @@ -24,10 +24,14 @@ with the respective `kdbGet`. ## Decision -- always load everything (with mmap) +Not supported, admins/maintainers need to stay with their spec within what applications request by `kdbGet`. ## Rationale +- it is too expensive to always load all keys (@mpranj: is this true?) +- problems in implementing to always get everything +- it makes the parameter to `kdbGet` basically useless + ## Implications ## Related Decisions From 3131c2d9fe581450b16f3c367286f8dad8f72ef7 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 15:23:29 +0100 Subject: [PATCH 10/50] doc: rework boolean --- doc/decisions/boolean.md | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/doc/decisions/boolean.md b/doc/decisions/boolean.md index e826eef6b9b..2e2db730959 100644 --- a/doc/decisions/boolean.md +++ b/doc/decisions/boolean.md @@ -6,46 +6,47 @@ Inconsistent use of bool in various parts of Elektra. ## Constraints +- needs to be string + ## Assumptions -- needs to be string -- convenience plugins can convert anything to 0 or 1 - type checker plugins can reject everything not 0 or 1 ## Considered Alternatives -- strictly only allow 0 and 1 (would move validation across the code) - only check presence or absence (no cascading override of already present key possible) -- use as in CMake (would move convenience across the code) +- use booleans as in CMake, which allows on/off, true/false, ... (would need convenience across the code) ## Decision -Use, depending on what your default should be: +Only the strings `0` and `1` are allowed in the `KeySet` for `type = boolean`. +Everything else should lead to errors in checkers (in `kdbSet`). + +Storage plugins are allowed any representation as suitable. -- 0 is false, everything else is true (default is true) -- 1 is true, everything else is false (default is false) +In the absence of the key, the default can be either: -Example: +- default is true: + `0` is false, everything else is true, or +- default is false: + `1` is true, everything else is false. + +Example for implementation in C: ```c if ( strcmp(keyString(k), "0")) {/*true*/} else {/*false*/} if (!strcmp(keyString(k), "1")) {/*true*/} else {/*false*/} ``` -In the documentation it should mention that a bool is used +In the spec/docu it should mention that a bool is used and which is the default. The type checker plugin should allow -- non-presence (default) +- non-presence (for default), if not required - the string "0" - the string "1" -The convenience plugin should transform (it might be combined with a plugin that transforms everything lower-case): - -- "false", "off", "no" to "0" -- "true", "on", "yes" to "1" - ## Rationale - most easy to implement From e07dac14ee8abdf15f77fd47eb5cd9e3400ddebe Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 18:09:52 +0100 Subject: [PATCH 11/50] doc: add 3 decisions --- doc/decisions/README.md | 9 ++++-- doc/decisions/ensure.md | 39 +++++++++++++++++++++++ doc/decisions/escaped_name.md | 47 ++++++++++++++++++++++++++++ doc/decisions/spec_expressiveness.md | 30 ++++++++++++++++++ 4 files changed, 122 insertions(+), 3 deletions(-) create mode 100644 doc/decisions/ensure.md create mode 100644 doc/decisions/escaped_name.md create mode 100644 doc/decisions/spec_expressiveness.md diff --git a/doc/decisions/README.md b/doc/decisions/README.md index f172fc49f6c..6afb3522111 100644 --- a/doc/decisions/README.md +++ b/doc/decisions/README.md @@ -41,24 +41,27 @@ section here. ## Decided - [Array for Warnings](warning_array.md) +- [Vendor Spec](vendor_spec.md) ## In Discussion - [Semantics for Name](semantics_name.md) - [Base Name](base_name.md) +- [Escaped Name](escaped_name.md) - [Store the escaped and/or unescaped key name](store_name.md) -- [Sources for Metadata](metadata_sources.md) - [Lookup every key](lookup_every_key.md) - [Holes and Non-leaf values](holes.md) -- [Arbitrary Meta-Data](arbitrary_metadata.md) +- [Sources for Metadata](metadata_sources.md) +- [Arbitrary Meta-Data](arbitrary_metadata.md) merge with metadata_sources.md - [Multiple File Backends](multiple_file_backends.md) - [Capabilities](capabilities.md) - [Array](array.md) - [Boolean](boolean.md) - [Global Plugins](global_plugins.md) -- [Vendor Spec](vendor_spec.md) +- [Ensure](ensure.md) - [Plugin Variants](plugin_variants.md) - [Global Validation](global_validation.md) +- [Spec Expressiveness](spec_expressiveness.md) ## Rejected diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md new file mode 100644 index 00000000000..f4101ff93a1 --- /dev/null +++ b/doc/decisions/ensure.md @@ -0,0 +1,39 @@ +# Ensure + +## Problem + +Applications want to ensure that some functionality (=global plugin) +is present in Elektra. + +## Constraints + +- keep notifications and highlevel API as-is + +## Assumptions + +## Considered Alternatives + +- integrate in kdbOpen + +## Decision + +Keep kdbEnsure but: + +- reduce for only global plugins, as the partial other functionality is confusing + and not needed +- find solution that list plugin is not needed +- only as implementation detail below libraries (e.g. like done for notification) + +## Rationale + +- allows interfaces that let the user do `kdbOpen` + +## Implications + +- gopts needs a nice wrapper library (low-prior) + +## Related Decisions + +- [Global Plugins](global_plugins.md) + +## Notes diff --git a/doc/decisions/escaped_name.md b/doc/decisions/escaped_name.md new file mode 100644 index 00000000000..c1949b50ca9 --- /dev/null +++ b/doc/decisions/escaped_name.md @@ -0,0 +1,47 @@ +# Escaped Name + +## Problem + +Currently we store both the escaped and unescaped name in a Key and in +a way that there cannot be an optimization not to store it (there are +functions handling out the pointer). + +## Constraints + +- allow implementations of Elektra to not store two names + +## Assumptions + +- the memory consumption of storing the name twice is considerable + +## Considered Alternatives + +- store it twice +- store it once but in one memory block + +## Decision + +- remove the escaped name from `_Key` +- `keyName` returns the unescaped name +- remove `keyUnescapedName`, `keyGetUnescapedNameSize`, `keyGetFullName`. +- reverse terminology: with "key name" we will refer to the unescaped (base) name, + the escaped name will be explicitly called "escaped key name". + "full name" (a variant of the escaped name) will not be used anymore. +- escaped name will be only present in `keyNew` and `keyGetEscapedName` +- remove `keyAddName` +- `keyDup` with arguments to filter which parts are copied + (to allow copy of keys where only the key name is copied) + +## Rationale + +## Implications + +- needs fix of OPMPHM +- Later modifications in keyNew to also accept key base names might be useful: + +- keyNew(KEY_VALUE, "...", KEY_NAME, "", "...", KEY_END) +- keyNew (KEY_NAME, KEY_NS_USER, "abc", "def", KEY_NAME_END, KEY_END); + +## Related Decisions + +## Notes diff --git a/doc/decisions/spec_expressiveness.md b/doc/decisions/spec_expressiveness.md new file mode 100644 index 00000000000..f7a071931a1 --- /dev/null +++ b/doc/decisions/spec_expressiveness.md @@ -0,0 +1,30 @@ +# Spec's Expressiveness + +## Problem + +Currently, you can easily come into wrong assumptions +that something would work in the specification. + +We need to find minimal requirements to implement a sane spec plugin. + +## Constraints + +## Assumptions + +## Considered Alternatives + +## Decision + +- no defaults for `sw/_/key` specifications + (default will not work for `ksLookup(/sw/sthg/key)`) +- plugins are not allowed to create keys + +The spec plugin should yield errors when it detects such situations. + +## Rationale + +## Implications + +## Related Decisions + +## Notes From cc280cc993835d8c559f32b09dbb2461820f66f8 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 3 Nov 2020 18:11:33 +0100 Subject: [PATCH 12/50] doc: merge similar decisions --- doc/decisions/README.md | 3 +-- doc/decisions/arbitrary_metadata.md | 4 ++++ doc/decisions/metadata_sources.md | 23 ----------------------- 3 files changed, 5 insertions(+), 25 deletions(-) delete mode 100644 doc/decisions/metadata_sources.md diff --git a/doc/decisions/README.md b/doc/decisions/README.md index 6afb3522111..e34ce4f9089 100644 --- a/doc/decisions/README.md +++ b/doc/decisions/README.md @@ -51,8 +51,7 @@ section here. - [Store the escaped and/or unescaped key name](store_name.md) - [Lookup every key](lookup_every_key.md) - [Holes and Non-leaf values](holes.md) -- [Sources for Metadata](metadata_sources.md) -- [Arbitrary Meta-Data](arbitrary_metadata.md) merge with metadata_sources.md +- [Arbitrary Meta-Data](arbitrary_metadata.md) - [Multiple File Backends](multiple_file_backends.md) - [Capabilities](capabilities.md) - [Array](array.md) diff --git a/doc/decisions/arbitrary_metadata.md b/doc/decisions/arbitrary_metadata.md index d3ec0d0b2c9..d541142d720 100644 --- a/doc/decisions/arbitrary_metadata.md +++ b/doc/decisions/arbitrary_metadata.md @@ -6,6 +6,8 @@ To make storage-plugins suitable for `spec` they need to be able to store all the metadata as specified in [METADATA.ini](/doc/METADATA.ini). Most file formats do not have support for that. +If metadata is merged from different sources, metadata from spec might end up somewhere else. + ## Constraints ## Assumptions @@ -19,6 +21,8 @@ Most file formats do not have support for that. ## Decision +Do not store metadata in any namespace but spec. + Use different storage plugins, or plugins with different configurations, for the `spec` namespace: diff --git a/doc/decisions/metadata_sources.md b/doc/decisions/metadata_sources.md deleted file mode 100644 index fe15c8134e5..00000000000 --- a/doc/decisions/metadata_sources.md +++ /dev/null @@ -1,23 +0,0 @@ -# Metadata Sources - -## Problem - -If metadata is merged from different sources, metadata from spec might end up somewhere else. - -## Constraints - -## Assumptions - -## Considered Alternatives - -## Decision - -Do not store metadata in any namespace but spec. - -## Rationale - -## Implications - -## Related Decisions - -## Notes From ca32fdcfe2f0c2423a70d4ab0cf336216a09d0ae Mon Sep 17 00:00:00 2001 From: "Restyled.io" Date: Tue, 3 Nov 2020 17:07:42 +0000 Subject: [PATCH 13/50] Restyled by prettier-markdown --- doc/GETSTARTED.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/GETSTARTED.md b/doc/GETSTARTED.md index 9b71ddc8b23..11de8f60e3b 100644 --- a/doc/GETSTARTED.md +++ b/doc/GETSTARTED.md @@ -62,6 +62,7 @@ ``` Optional you can also run tests, see [here for more information](/doc/TESTING.md): + ```sh cmake --build build --target run_nokdbtests ``` @@ -76,6 +77,7 @@ [Installation documentation](/doc/INSTALL.md) contains further information about available packages. Optional you can also run tests to verify the installed Elektra, see [here for more information](/doc/TESTING.md): + ```sh kdb run_nokdbtests ``` From 49a324eef807fd7d08ff60dce4b6346b8b6491da Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Wed, 4 Nov 2020 08:45:35 +0100 Subject: [PATCH 14/50] doc: write rationale for semantics_name.md --- doc/decisions/semantics_name.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index 6172732d335..948370aef34 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -31,10 +31,17 @@ The `spec` plugin guards these rules. ## Rationale +- for consistency, whenever possible, meta-data should be preferred +- no escaping of key base names necessary +- it is very unlikely that the 9-byte UTF-8 sequence `®elektra` collides with a real key base name + a user wanted to have +- `®elektra` makes very clear that there is a special reserved meaning + ## Implications ## Related Decisions - [Arrays](array.md) +- [Base Names](base_name.md) ## Notes From dfe3d648adb06984d53eba3254e41932df1fe6fb Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Wed, 4 Nov 2020 12:11:25 +0100 Subject: [PATCH 15/50] doc: add open points from #2698 --- doc/decisions/array.md | 5 +++++ doc/decisions/escaped_name.md | 13 ++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/doc/decisions/array.md b/doc/decisions/array.md index 0d26fba0909..df0d8baf2a9 100644 --- a/doc/decisions/array.md +++ b/doc/decisions/array.md @@ -24,6 +24,10 @@ an array or not. ## Decision Store length in metadata `array` of key, or keep metadata `array` empty if empty array. +Only children that have `#` syntax are allowed. +The index start with `#0`. +Both `keyAddName("#12")` or `keyAddBaseName("#_12")` is allowed to add the 13th index. + For example (`ni syntax`, sections used for metadata): ``` @@ -103,6 +107,7 @@ Guarantees we want from the spec plugin: - [Global Plugins](global_plugins.md) - [Global Validation](global_validation.md) +- [Base Names](base_name.md) ## Notes diff --git a/doc/decisions/escaped_name.md b/doc/decisions/escaped_name.md index c1949b50ca9..19d1dbe6d2a 100644 --- a/doc/decisions/escaped_name.md +++ b/doc/decisions/escaped_name.md @@ -19,6 +19,16 @@ functions handling out the pointer). - store it twice - store it once but in one memory block +## Open Points + +Clarify and reduce [terminology](/doc/help/elektra-glossary.md): + +- escaped name +- full name +- part name +- base name +- dir name + ## Decision - remove the escaped name from `_Key` @@ -28,7 +38,8 @@ functions handling out the pointer). the escaped name will be explicitly called "escaped key name". "full name" (a variant of the escaped name) will not be used anymore. - escaped name will be only present in `keyNew` and `keyGetEscapedName` -- remove `keyAddName` +- rename `keyAddName`, e.g. to `keyAddEscapedName` + (or `keyAddFullName` if we decide to call the escaped name full name) - `keyDup` with arguments to filter which parts are copied (to allow copy of keys where only the key name is copied) From 25b96fad9d6d350cf3888dd2c1a6cf95ee798e8c Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 5 Nov 2020 10:23:05 +0100 Subject: [PATCH 16/50] doc: tip for #3491 --- .github/PULL_REQUEST_TEMPLATE.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 18f46cd998e..163fca7033e 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -9,7 +9,11 @@ These points need to be fulfilled for every PR: - [ ] Details of what you changed are in commit messages (first line should have `module: short statement` syntax) - [ ] References to issues, e.g. `close #X`, are in the commit messages. -- [ ] The buildservers are happy. +- [ ] The buildservers are happy. If not, fix **in this order**: + - [ ] add a line in `doc/news/_preparation_next_release.md` + - [ ] reformat the code with `scripts/dev/reformat-all` + - [ ] make all unit tests pass + - [ ] fix all memleaks - [ ] The PR is rebased with current master. If you have any troubles fulfilling these criteria, please write From d08e265cfc5c8284a5103db1ff2b944d4b3f4d6e Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 09:32:32 +0100 Subject: [PATCH 17/50] doc: clarify valid array --- doc/decisions/array.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/decisions/array.md b/doc/decisions/array.md index df0d8baf2a9..f881ac07c05 100644 --- a/doc/decisions/array.md +++ b/doc/decisions/array.md @@ -24,7 +24,7 @@ an array or not. ## Decision Store length in metadata `array` of key, or keep metadata `array` empty if empty array. -Only children that have `#` syntax are allowed. +Only children that have `#` syntax are allowed in a valid array. The index start with `#0`. Both `keyAddName("#12")` or `keyAddBaseName("#_12")` is allowed to add the 13th index. @@ -80,12 +80,12 @@ user:/myarray/#0 system:/myarray # <- not found in cascading lookup, as user:/myarray exists ``` -Guarantees we want from the spec plugin: +The `spec` plugin should check if it is a valid array, i.e.: -- that the parent key always contain `array`. -- that the correct length is in `array` -- that the array only contains `#` children -- that the children are numbered from 0 to n, without holes +- that the parent key always contain the metadata `array`, +- that the correct length is in `array`, +- that the array only contains `#` children, and +- that the children are numbered from `#0` to `#n`, without holes. ## Rationale From b3728348754d6e9fa24802adb4ea940f3b4738b3 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 09:48:12 +0100 Subject: [PATCH 18/50] doc: make boolean definition stricter --- doc/decisions/boolean.md | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/doc/decisions/boolean.md b/doc/decisions/boolean.md index 2e2db730959..557ee339e74 100644 --- a/doc/decisions/boolean.md +++ b/doc/decisions/boolean.md @@ -22,28 +22,20 @@ Inconsistent use of bool in various parts of Elektra. Only the strings `0` and `1` are allowed in the `KeySet` for `type = boolean`. Everything else should lead to errors in checkers (in `kdbSet`). -Storage plugins are allowed any representation as suitable. +The spec/docu should mention that a `boolean` is used and may specify the default. +The absence of a non-required key without a specified default should be interpreted as false. -In the absence of the key, the default can be either: - -- default is true: - `0` is false, everything else is true, or -- default is false: - `1` is true, everything else is false. - -Example for implementation in C: +Example for an implementation in C in an application: ```c -if ( strcmp(keyString(k), "0")) {/*true*/} else {/*false*/} if (!strcmp(keyString(k), "1")) {/*true*/} else {/*false*/} ``` -In the spec/docu it should mention that a bool is used -and which is the default. +Storage plugins are allowed any representation as suitable, e.g., a JSON plugin might render `1` as `true`. The type checker plugin should allow -- non-presence (for default), if not required +- non-presence, if not required - the string "0" - the string "1" From df3a509f13dc22745e5e89771cf57dfebcffdf1b Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 10:33:25 +0100 Subject: [PATCH 19/50] decisions: ensure --- doc/decisions/ensure.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index f4101ff93a1..b0cd2e9bf72 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -13,11 +13,17 @@ is present in Elektra. ## Considered Alternatives -- integrate in kdbOpen +- integrate in kdbOpen: + + can immediately build up correct plugin positioning + - cannot be called later easily (needs `kdbClose` and `kdbOpen` again) + - cannot be integrated with some APIs, e.g. `kdbDup` or current notification API +- specific APIs per plugin + - difficult for application developers + - every plugin would need to design new APIs ## Decision -Keep kdbEnsure but: +Keep `kdbEnsure` but: - reduce for only global plugins, as the partial other functionality is confusing and not needed @@ -26,12 +32,16 @@ Keep kdbEnsure but: ## Rationale -- allows interfaces that let the user do `kdbOpen` +- allows APIs (that call `kdbEnsure`) while the user still calls `kdbOpen`, like done in the notification API: + `kdb = kdbOpen(); APIcallDoingKdbEnsure (kdb)` + (`APIcallDoingKdbEnsure` would not be possible if the functionality would be integrated in `kdbOpen`) +- allows applications to continue even if `kdbEnsure` failed +- is already implemented +- is already used by notification and high-level library +- gopts would continue to work as is but also a nice wrapper library could be added (low-prior) ## Implications -- gopts needs a nice wrapper library (low-prior) - ## Related Decisions - [Global Plugins](global_plugins.md) From 1dd7fb12d2e19256f55d4c0ce416d5d7c4f6c69c Mon Sep 17 00:00:00 2001 From: "Restyled.io" Date: Mon, 9 Nov 2020 09:28:55 +0000 Subject: [PATCH 20/50] Restyled by prettier-markdown --- doc/decisions/ensure.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index b0cd2e9bf72..3b072c148d4 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -14,9 +14,9 @@ is present in Elektra. ## Considered Alternatives - integrate in kdbOpen: - + can immediately build up correct plugin positioning - - cannot be called later easily (needs `kdbClose` and `kdbOpen` again) - - cannot be integrated with some APIs, e.g. `kdbDup` or current notification API + - can immediately build up correct plugin positioning + * cannot be called later easily (needs `kdbClose` and `kdbOpen` again) + * cannot be integrated with some APIs, e.g. `kdbDup` or current notification API - specific APIs per plugin - difficult for application developers - every plugin would need to design new APIs From 2e3cf5e6b10217a58e2b16fcd9c7cb3a445ec9a0 Mon Sep 17 00:00:00 2001 From: markus2330 Date: Mon, 9 Nov 2020 11:05:43 +0100 Subject: [PATCH 21/50] Update doc/decisions/semantics_name.md Co-authored-by: kodebach <23529132+kodebach@users.noreply.github.com> --- doc/decisions/semantics_name.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index 948370aef34..f7008860352 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -33,7 +33,7 @@ The `spec` plugin guards these rules. - for consistency, whenever possible, meta-data should be preferred - no escaping of key base names necessary -- it is very unlikely that the 9-byte UTF-8 sequence `®elektra` collides with a real key base name +- it is very unlikely that the UTF-8 sequence `®elektra` (i.e. the 9-byte sequence `C2 AE 65 6C 65 6B 74 72 61`) collides with a real key base name a user wanted to have - `®elektra` makes very clear that there is a special reserved meaning From 7b35fcfff0a429ab8ffac8823fb5d41fc9e22086 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 11:16:01 +0100 Subject: [PATCH 22/50] =?UTF-8?q?decisions:=20add=20advantage=20of=20?= =?UTF-8?q?=C2=AE?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- doc/decisions/semantics_name.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index f7008860352..6bb9854f56f 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -24,7 +24,8 @@ Nevertheless, there are guidelines (without any checks in `keySetBaseName`): - `#` is used to indicate that array numbers follow. - `®` is used to indicate that some information was encoded in the key name. This is usually only needed internally in storage plugins. -- `®elektra` is reserved, key names should not start with that sequence. +- the UTF-8 sequence `®elektra` (i.e. the 9-byte sequence `C2 AE 65 6C 65 6B 74 72 61`) is reserved, + key names should not start with that sequence. There are, however, rules and conventions which syntax to use for specific semantics. The `spec` plugin guards these rules. @@ -33,9 +34,12 @@ The `spec` plugin guards these rules. - for consistency, whenever possible, meta-data should be preferred - no escaping of key base names necessary -- it is very unlikely that the UTF-8 sequence `®elektra` (i.e. the 9-byte sequence `C2 AE 65 6C 65 6B 74 72 61`) collides with a real key base name +- it is very unlikely that `®elektra` collides with a real key base name a user wanted to have - `®elektra` makes very clear that there is a special reserved meaning +- `®elektra` UTF-8 encoding decodes to "some character" + ® in many 8-bit encodings + (including ISO 8859-1 aka Latin1 and Windows (Codepage) 1252, + in the encoding `C`, however, you get `''$'\302\256''elektra'`) ## Implications From fbe13d369ad61e6216111caf3d6057549a8839f2 Mon Sep 17 00:00:00 2001 From: "Restyled.io" Date: Mon, 9 Nov 2020 10:09:49 +0000 Subject: [PATCH 23/50] Restyled by prettier-markdown --- doc/decisions/semantics_name.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index 6bb9854f56f..ad3fbf8224e 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -34,12 +34,12 @@ The `spec` plugin guards these rules. - for consistency, whenever possible, meta-data should be preferred - no escaping of key base names necessary -- it is very unlikely that `®elektra` collides with a real key base name +- it is very unlikely that `®elektra` collides with a real key base name a user wanted to have - `®elektra` makes very clear that there is a special reserved meaning - `®elektra` UTF-8 encoding decodes to "some character" + ® in many 8-bit encodings (including ISO 8859-1 aka Latin1 and Windows (Codepage) 1252, - in the encoding `C`, however, you get `''$'\302\256''elektra'`) + in the encoding `C`, however, you get `''$'\302\256''elektra'`) ## Implications From bf901a06bb37224c70118e0571bb72055157ce6e Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 11:18:49 +0100 Subject: [PATCH 24/50] decisions clarify sentence --- doc/decisions/arbitrary_metadata.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/decisions/arbitrary_metadata.md b/doc/decisions/arbitrary_metadata.md index d541142d720..337665357cb 100644 --- a/doc/decisions/arbitrary_metadata.md +++ b/doc/decisions/arbitrary_metadata.md @@ -6,7 +6,9 @@ To make storage-plugins suitable for `spec` they need to be able to store all the metadata as specified in [METADATA.ini](/doc/METADATA.ini). Most file formats do not have support for that. -If metadata is merged from different sources, metadata from spec might end up somewhere else. +If metadata is merged from different namespaces, e.g., `spec:` and `user:`, +metadata from one namespace might end up in keys of other namespaces, e.g., +metadata from `spec:` might end up in `user:`. ## Constraints From 1ff7c6741ba83965ad3262ab426a3024fa55b458 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 11:24:51 +0100 Subject: [PATCH 25/50] decisions: clarify arbitrary metadata --- doc/decisions/arbitrary_metadata.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/doc/decisions/arbitrary_metadata.md b/doc/decisions/arbitrary_metadata.md index 337665357cb..87706193ba7 100644 --- a/doc/decisions/arbitrary_metadata.md +++ b/doc/decisions/arbitrary_metadata.md @@ -23,7 +23,15 @@ metadata from `spec:` might end up in `user:`. ## Decision -Do not store metadata in any namespace but spec. +Do not store arbitrary metadata in any namespace except spec. +In other namespaces only store metadata that is handled by the `spec` plugin. +E.g. for `array`, metadata might be stored in any namespace: + +- either the storage plugin does not support arrays, then the metadata will be discarded + on `kdbSet` but `spec` will keep on adding it for every `kdbGet` +- if the storage plugin supports arrays, the data will be serialized as array + (even if the metadata comes from `spec`) and as such available in the next `kdbGet` + from the storage plugin to be validated by `spec` Use different storage plugins, or plugins with different configurations, for the `spec` namespace: From 8ad525ff4316d45b147204d99c7113c19d089a3a Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Mon, 9 Nov 2020 13:00:39 +0100 Subject: [PATCH 26/50] decisions: fix spelling --- doc/decisions/semantics_name.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index ad3fbf8224e..6e5d897faf7 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -32,7 +32,7 @@ The `spec` plugin guards these rules. ## Rationale -- for consistency, whenever possible, meta-data should be preferred +- for consistency, whenever possible, metadata should be preferred - no escaping of key base names necessary - it is very unlikely that `®elektra` collides with a real key base name a user wanted to have From a64449001073770e88859c77d32d6c918312ff7c Mon Sep 17 00:00:00 2001 From: markus2330 Date: Tue, 10 Nov 2020 09:21:37 +0100 Subject: [PATCH 27/50] Update doc/decisions/boolean.md Co-authored-by: kodebach <23529132+kodebach@users.noreply.github.com> --- doc/decisions/boolean.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/boolean.md b/doc/decisions/boolean.md index 557ee339e74..7ff7e70c11c 100644 --- a/doc/decisions/boolean.md +++ b/doc/decisions/boolean.md @@ -28,7 +28,7 @@ The absence of a non-required key without a specified default should be interpre Example for an implementation in C in an application: ```c -if (!strcmp(keyString(k), "1")) {/*true*/} else {/*false*/} +if (k != NULL && strcmp(keyString(k), "1") == 0) {/*true*/} else {/*false*/} ``` Storage plugins are allowed any representation as suitable, e.g., a JSON plugin might render `1` as `true`. From 60383f72103709c07460beabb6e0ea27efb83fda Mon Sep 17 00:00:00 2001 From: markus2330 Date: Tue, 10 Nov 2020 09:22:16 +0100 Subject: [PATCH 28/50] Update doc/decisions/semantics_name.md Co-authored-by: kodebach <23529132+kodebach@users.noreply.github.com> --- doc/decisions/semantics_name.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index 6e5d897faf7..0c6b2bdc972 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -25,7 +25,7 @@ Nevertheless, there are guidelines (without any checks in `keySetBaseName`): - `®` is used to indicate that some information was encoded in the key name. This is usually only needed internally in storage plugins. - the UTF-8 sequence `®elektra` (i.e. the 9-byte sequence `C2 AE 65 6C 65 6B 74 72 61`) is reserved, - key names should not start with that sequence. + key name parts should not start with that sequence. There are, however, rules and conventions which syntax to use for specific semantics. The `spec` plugin guards these rules. From 94a33e9bef101543f8aecb8ded0e15eedbdd46f0 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 10 Nov 2020 13:21:00 +0100 Subject: [PATCH 29/50] decisions: rework spec metadata --- doc/decisions/README.md | 2 +- doc/decisions/arbitrary_metadata.md | 53 ---------------- doc/decisions/array.md | 6 +- doc/decisions/spec_metadata.md | 94 +++++++++++++++++++++++++++++ 4 files changed, 99 insertions(+), 56 deletions(-) delete mode 100644 doc/decisions/arbitrary_metadata.md create mode 100644 doc/decisions/spec_metadata.md diff --git a/doc/decisions/README.md b/doc/decisions/README.md index e34ce4f9089..89899823a34 100644 --- a/doc/decisions/README.md +++ b/doc/decisions/README.md @@ -51,7 +51,7 @@ section here. - [Store the escaped and/or unescaped key name](store_name.md) - [Lookup every key](lookup_every_key.md) - [Holes and Non-leaf values](holes.md) -- [Arbitrary Meta-Data](arbitrary_metadata.md) +- [Metadata in Spec Namespace](spec_metadata.md) - [Multiple File Backends](multiple_file_backends.md) - [Capabilities](capabilities.md) - [Array](array.md) diff --git a/doc/decisions/arbitrary_metadata.md b/doc/decisions/arbitrary_metadata.md deleted file mode 100644 index 87706193ba7..00000000000 --- a/doc/decisions/arbitrary_metadata.md +++ /dev/null @@ -1,53 +0,0 @@ -# Arbitrary Metadata - -## Problem - -To make storage-plugins suitable for `spec` they need to be able to store -all the metadata as specified in [METADATA.ini](/doc/METADATA.ini). -Most file formats do not have support for that. - -If metadata is merged from different namespaces, e.g., `spec:` and `user:`, -metadata from one namespace might end up in keys of other namespaces, e.g., -metadata from `spec:` might end up in `user:`. - -## Constraints - -## Assumptions - -## Considered Alternatives - -- store metadata in the comments like the `ini` plugin: - this exposes internal metadata into the comments and - can drastically affect the readability of a storage file. - Comments should never be touched by a parser. - -## Decision - -Do not store arbitrary metadata in any namespace except spec. -In other namespaces only store metadata that is handled by the `spec` plugin. -E.g. for `array`, metadata might be stored in any namespace: - -- either the storage plugin does not support arrays, then the metadata will be discarded - on `kdbSet` but `spec` will keep on adding it for every `kdbGet` -- if the storage plugin supports arrays, the data will be serialized as array - (even if the metadata comes from `spec`) and as such available in the next `kdbGet` - from the storage plugin to be validated by `spec` - -Use different storage plugins, or plugins with different configurations, -for the `spec` namespace: - -- `ni` -- TOML with `meta` configuration - -## Rationale - -- We do not need a storage plugin suitable for everything. -- The problems that internal metadata ends up in configuration files disappears. - -## Implications - -We need to have different default plugins in `spec` than in the other namespaces. - -## Related Decisions - -## Notes diff --git a/doc/decisions/array.md b/doc/decisions/array.md index f881ac07c05..a12195cbb3f 100644 --- a/doc/decisions/array.md +++ b/doc/decisions/array.md @@ -44,8 +44,8 @@ myarray/#5 = value5 It is not allowed to have anything else than `#5` in the metadata `array`, e.g. even `#4` would be a malformed array. -The metadata `array` preferable should be in `spec` (specified configuration). -Then the `spec` plugin will add the `array` marker with the correct length. +With the metadata `array` in `spec:/` the `spec` plugin will add the +`array` marker with the correct length. This needs to be happen early, so that plugins can rely on having correct arrays. @@ -108,6 +108,8 @@ The `spec` plugin should check if it is a valid array, i.e.: - [Global Plugins](global_plugins.md) - [Global Validation](global_validation.md) - [Base Names](base_name.md) +- [Metadata in Spec Namespace](spec_metadata.md) +- [Spec Expressiveness](spec_expressiveness.md) ## Notes diff --git a/doc/decisions/spec_metadata.md b/doc/decisions/spec_metadata.md new file mode 100644 index 00000000000..06b624390c4 --- /dev/null +++ b/doc/decisions/spec_metadata.md @@ -0,0 +1,94 @@ +# Metadata in Spec Namespace + +## Problem + +To make storage-plugins suitable for `spec` they need to be able to store +all the metadata as specified in [METADATA.ini](/doc/METADATA.ini). +Most file formats do not have support for that. + +If metadata is merged from different namespaces, e.g., `spec:` and `user:`, +metadata from one namespace might end up in keys of other namespaces, e.g., +metadata from `spec:` might end up in `user:`. + +## Constraints + +- We do not touch what is already described in the + [storage plugin tutorial](/doc/tutorials/storage-plugins.md). + +## Assumptions + +- Metadata can always safely merged from `spec:/` to other + namespace. + +## Considered Alternatives + +- Store metadata in the comments like the `ini` plugin: + this exposes internal metadata into the comments and + can drastically affect the readability of a storage file. + Comments should never be touched by a parser. +- Designate metadata to be only for `spec:/` or to only + for restoring configuration file formats. + Rename metadata `` in `spec:/` to `check/` + or `make/array`. + This way no merging of meta-data would be needed and + by the name alone it would be clear to which namespace + it belongs. +- Do not support `array` or `type` if the underlying + configuration file format does not support it. + `spec-mount` could make sure that the storage plugin + supports everything the underlying format needs. + +## Decision + +Do not store metadata unrelated to the configuration file structure in any namespace except in `spec:/`. + +- Trying to store any other metadata in any other namespace leads to an error. + E.g. `kdb set-meta user:/data metadata_not_suitable_for_storage_plugins something` would fail + (validated by `spec` plugin). +- Metadata that is designed to be stored by storage plugins to preserve configuration file structure. + E.g. `comment` or `order`, might be stored in any namespace. + +Sometimes, the same metadata can be used in several namespaces but with different meanings +and ways of serialization, e.g. `type` and `array`: + +- In `spec:/` the metadata `array=` (empty value) means "this is an array". + If you give it a value e.g. `array=#4` it means "this is an array with default size X" (e.g. `#4` = size 5). +- In any other namespace `array=` means "this is an empty array" and e.g. `array=#4` means "this is an array with max index #4". + `array=#4` is not stored literally but inferred. +- Either the storage plugin does not support arrays, then the metadata will be discarded + on `kdbSet` but `spec` will keep on adding it for every `kdbGet`. +- Or, if the storage plugin supports arrays, the data will be serialized as array + (even if the metadata comes from `spec`) and as such available in the next `kdbGet` + from the storage plugin to be validated by `spec`. + +Use different storage plugins, or plugins with different configurations, +for the `spec:/` namespace: + +- `ni` +- TOML with `meta` configuration + +The `kdb mount` tool will add the `meta` plugin configuration when mounting a storage plugin to `spec:/`. + +## Rationale + +- We do not need a storage plugin suitable for everything. +- The problems that internal metadata ends up in configuration files disappears. +- The short names `type` and `array` can be used in specs. +- Also simple storage plugins (like properties) can still be used with realistic + specifications (that include array and/or type). + +## Implications + +- We need to have different (default) plugins in `spec:/` than in the other namespaces. +- It can be confusing about which metadata we are speaking, e.g., `array` in + `spec:/` has a quite different meaning to `array` in other namespaces. +- The spec:/ meaning of array= must obviously be stored. The other meaning should never + be stored explicitly and instead should be inferred from the stored data. Either by + the storage plugin (e.g. JSON array) or by the spec plugin. + +## Related Decisions + +- [Spec Expressiveness](spec_expressiveness.md) +- [Arrays](array.md) + +## Notes From b918c03bd8228fff6c2789f6328552547ca79dfd Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 10 Nov 2020 13:44:56 +0100 Subject: [PATCH 30/50] decisions: rework boolean --- doc/decisions/boolean.md | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/doc/decisions/boolean.md b/doc/decisions/boolean.md index 7ff7e70c11c..17168793e6b 100644 --- a/doc/decisions/boolean.md +++ b/doc/decisions/boolean.md @@ -2,7 +2,7 @@ ## Problem -Inconsistent use of bool in various parts of Elektra. +Inconsistent use of booleans in various parts of Elektra. ## Constraints @@ -10,20 +10,18 @@ Inconsistent use of bool in various parts of Elektra. ## Assumptions -- type checker plugins can reject everything not 0 or 1 - ## Considered Alternatives - only check presence or absence (no cascading override of already present key possible) - use booleans as in CMake, which allows on/off, true/false, ... (would need convenience across the code) +- do not accept a specification with `type = boolean` without a default ## Decision -Only the strings `0` and `1` are allowed in the `KeySet` for `type = boolean`. +Only the strings `0` and `1` are allowed in the `KeySet` for `type = boolean`, for both values and defaults. Everything else should lead to errors in checkers (in `kdbSet`). -The spec/docu should mention that a `boolean` is used and may specify the default. -The absence of a non-required key without a specified default should be interpreted as false. +A spec with `type = boolean` without a specified default should be interpreted as `default = 0`. Example for an implementation in C in an application: @@ -35,22 +33,22 @@ Storage plugins are allowed any representation as suitable, e.g., a JSON plugin The type checker plugin should allow -- non-presence, if not required +- non-presence - the string "0" - the string "1" ## Rationale - most easy to implement -- allows presence to be true +- allows non-presence to be false - plugins allow us to convert to any other behavior ## Implications -- change code with different behavior +- Storage plugins are only allowed to emit `0` or `1` +- Applications either get `0` or `1`, or (without a key) + can safely assume that false is meant ## Related Decisions ## Notes - -See [here](https://github.com/ElektraInitiative/libelektra/issues/308) From 1464dfd44fa527d5bb8f2cffeaa184ebd3844bd4 Mon Sep 17 00:00:00 2001 From: "Restyled.io" Date: Tue, 10 Nov 2020 12:38:55 +0000 Subject: [PATCH 31/50] Restyled by prettier-markdown --- doc/decisions/boolean.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/boolean.md b/doc/decisions/boolean.md index 17168793e6b..db47f06292d 100644 --- a/doc/decisions/boolean.md +++ b/doc/decisions/boolean.md @@ -21,7 +21,7 @@ Inconsistent use of booleans in various parts of Elektra. Only the strings `0` and `1` are allowed in the `KeySet` for `type = boolean`, for both values and defaults. Everything else should lead to errors in checkers (in `kdbSet`). -A spec with `type = boolean` without a specified default should be interpreted as `default = 0`. +A spec with `type = boolean` without a specified default should be interpreted as `default = 0`. Example for an implementation in C in an application: From b60f38288214a3207f942ae262a46c07fda57878 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 10 Nov 2020 13:48:27 +0100 Subject: [PATCH 32/50] decision: apply suggestion --- doc/decisions/array.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/decisions/array.md b/doc/decisions/array.md index a12195cbb3f..a8b5b47903c 100644 --- a/doc/decisions/array.md +++ b/doc/decisions/array.md @@ -102,6 +102,9 @@ The `spec` plugin should check if it is a valid array, i.e.: which is a possibility also in all the alternatives of this decision. - A `user:/` or `dir:/` key can change the semantics of a `system:/` array, if not avoided by `spec`. +- user-facing documentation should contain a note like: + "Mixing array and non-array keys in arrays is not supported. + In many cases the trivial solution is to move the array part into a separate child-key." ## Related Decisions From 5a89d215255be350dc2288d916910106ea68ded9 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 10 Nov 2020 14:46:41 +0100 Subject: [PATCH 33/50] decision: kdbOpen contract --- doc/decisions/ensure.md | 60 ++++++++++++++++++++++++++-------------- src/libs/invoke/invoke.c | 2 +- 2 files changed, 41 insertions(+), 21 deletions(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index 3b072c148d4..2661a06c988 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -7,41 +7,61 @@ is present in Elektra. ## Constraints -- keep notifications and highlevel API as-is - ## Assumptions ## Considered Alternatives -- integrate in kdbOpen: - - can immediately build up correct plugin positioning - * cannot be called later easily (needs `kdbClose` and `kdbOpen` again) - * cannot be integrated with some APIs, e.g. `kdbDup` or current notification API -- specific APIs per plugin +- Keep `kdbEnsure` (Rejected because it is too flexible and can be called many times. Furthermore, + `kdbOpen` would build up configurations that get removed afterwards.) + - reduce for only global plugins, as the partial other functionality is confusing + and not needed + - find solution that list plugin is not needed + - only as implementation detail below libraries (e.g. like done for notification) +- Specific APIs per plugin, Rejected because: - difficult for application developers - every plugin would need to design new APIs +- Have a new API: `KDB * kdbOpen (Key * parent);` + `int kdbConfigure (KDB * handle, KeySet * contract, Key * parentKey);` + `KDB * kdbOpenDefault (Key * parent);` + The new kdbOpen only does the absolute minimum work, in particular it doesn't set up any global plugins. + If you use kdbOpen you must call kdbConfigure otherwise kdbGet will fail. kdbConfigure configures global + plugins (basically just a renamed kdbEnsure). Lastly, kdbOpenDefault does more or less what the old kdbOpen + does. It sets up the default case and you can call kdbGet immediately. But you cannot call kdbConfigure after + kdbOpenDefault. + Rejected because of API bloat and introduction of further state in `kdb`. ## Decision -Keep `kdbEnsure` but: - -- reduce for only global plugins, as the partial other functionality is confusing - and not needed -- find solution that list plugin is not needed -- only as implementation detail below libraries (e.g. like done for notification) +Integrate `kdbEnsure` in `kdbOpen(Key *errorKey, KeySet *contract)` but only allow global plugins. ## Rationale -- allows APIs (that call `kdbEnsure`) while the user still calls `kdbOpen`, like done in the notification API: - `kdb = kdbOpen(); APIcallDoingKdbEnsure (kdb)` - (`APIcallDoingKdbEnsure` would not be possible if the functionality would be integrated in `kdbOpen`) -- allows applications to continue even if `kdbEnsure` failed -- is already implemented -- is already used by notification and high-level library -- gopts would continue to work as is but also a nice wrapper library could be added (low-prior) +- can immediately build up correct plugin positioning +- does not allow to start applications if the contract cannot be fulfilled +- simplest and minimalistic solution ## Implications +`elektraNotificationOpen` needs to be removed and instead we need a new API: + +```c +KeySet * contract = ksNew (0, KS_END); +elektraNotificationGetContract (contract); +KDB * kdb = kdbOpen (key, contract); // contract contains notification config +elektraIoSetBinding (kdb, binding); +``` + +Similar contract getters need to be introduced for other functionality, like gopts: + +```c +KeySet * contract = ksNew (0, KS_END); +elektraGOptsGetContract (contract, argc, argv, environ); +KDB * kdb = kdbOpen (key, contract); +``` + + +The high-level API can make this more pretty, though. + ## Related Decisions - [Global Plugins](global_plugins.md) diff --git a/src/libs/invoke/invoke.c b/src/libs/invoke/invoke.c index ee3015fef96..820d6c64033 100644 --- a/src/libs/invoke/invoke.c +++ b/src/libs/invoke/invoke.c @@ -377,7 +377,7 @@ void elektraInvokeExecuteDeferredCalls (ElektraInvokeHandle * handle, ElektraDef } /** - * Call a deferable function on a plugin handle. + * Call a deferrable function on a plugin handle. * If the function is exported by the plugin it is directly invoked, * if the plugin supports deferring calls, the call is deferred. * If both is possible (function is exported and deferred calls are supported), From 90dd882c0ffd318862580987db5065e092f94795 Mon Sep 17 00:00:00 2001 From: "Restyled.io" Date: Tue, 10 Nov 2020 13:40:40 +0000 Subject: [PATCH 34/50] Restyled by prettier-markdown --- doc/decisions/ensure.md | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index 2661a06c988..907ffc3eebc 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -59,7 +59,6 @@ elektraGOptsGetContract (contract, argc, argv, environ); KDB * kdb = kdbOpen (key, contract); ``` - The high-level API can make this more pretty, though. ## Related Decisions From ee72c8f8d1e68b6d68cac0bd72a1e2abb44edd6b Mon Sep 17 00:00:00 2001 From: markus2330 Date: Tue, 10 Nov 2020 17:49:26 +0100 Subject: [PATCH 35/50] Update doc/decisions/boolean.md Co-authored-by: kodebach <23529132+kodebach@users.noreply.github.com> --- doc/decisions/boolean.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/boolean.md b/doc/decisions/boolean.md index db47f06292d..4c96c24431b 100644 --- a/doc/decisions/boolean.md +++ b/doc/decisions/boolean.md @@ -45,7 +45,7 @@ The type checker plugin should allow ## Implications -- Storage plugins are only allowed to emit `0` or `1` +- Storage plugins are only allowed to emit `0` or `1` as key values - Applications either get `0` or `1`, or (without a key) can safely assume that false is meant From 80a2548564390a21caa85687493b17e22283965d Mon Sep 17 00:00:00 2001 From: markus2330 Date: Tue, 10 Nov 2020 17:49:46 +0100 Subject: [PATCH 36/50] Update doc/decisions/spec_expressiveness.md Co-authored-by: kodebach <23529132+kodebach@users.noreply.github.com> --- doc/decisions/spec_expressiveness.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/spec_expressiveness.md b/doc/decisions/spec_expressiveness.md index f7a071931a1..b9307a66d66 100644 --- a/doc/decisions/spec_expressiveness.md +++ b/doc/decisions/spec_expressiveness.md @@ -17,7 +17,7 @@ We need to find minimal requirements to implement a sane spec plugin. - no defaults for `sw/_/key` specifications (default will not work for `ksLookup(/sw/sthg/key)`) -- plugins are not allowed to create keys +- plugins are not allowed to create keys (may change in future; depends on plugin positions) The spec plugin should yield errors when it detects such situations. From aabb2e57e567829ba1e87c3b8bb7989aee470d02 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Wed, 11 Nov 2020 18:36:50 +0100 Subject: [PATCH 37/50] metadata: move internal up --- doc/METADATA.ini | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/METADATA.ini b/doc/METADATA.ini index 71852fb5707..6cf06767495 100644 --- a/doc/METADATA.ini +++ b/doc/METADATA.ini @@ -153,6 +153,12 @@ description= The spaces/tabs used between a comment and the beginning of the example= +[internal//*] +status= implemented +usedby/plugin= (ini) +description= Internal metadata to be ignored by other plugins. +example=internal/ini/parent could be the name for internal parent + information of the ini plugin. [line] @@ -1069,10 +1075,6 @@ status= idea description= Flag keys that will be removed in further versions note= thanks to Config::Model for that idea -[internal//*] -status= implemented -description= Internal metadata to be ignored by other plugins. - [source] status= idea description= states where a key comes from if it is not from an ordinary From f5b31736d4c56368aef4325cc3b6cf182e55d3ee Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Wed, 11 Nov 2020 18:41:45 +0100 Subject: [PATCH 38/50] doc: clarify about name in toml --- doc/METADATA.ini | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/doc/METADATA.ini b/doc/METADATA.ini index 6cf06767495..eb6c55892fc 100644 --- a/doc/METADATA.ini +++ b/doc/METADATA.ini @@ -155,11 +155,16 @@ example= [internal//*] status= implemented -usedby/plugin= (ini) -description= Internal metadata to be ignored by other plugins. -example=internal/ini/parent could be the name for internal parent - information of the ini plugin. - +usedby/plugin= +description= Internal metadata that must be ignored by other plugins. + It is useful for reconstruction of information about keys between + kdbGet and kdbSet. + Always avoid to use it if there is any other metadata that + can be used instead. + For example, if a type can be represented with the CORBA + type system, metadata `type` should be used. + Only if a type cannot be represented, e.g., the `toml` + plugin would use `internal/toml/type`. [line] type= int From 8462af0ce41a7d6f0f2467d1ae2b730fefb2dd25 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Wed, 11 Nov 2020 18:49:07 +0100 Subject: [PATCH 39/50] metadata: a bit cleanup --- doc/METADATA.ini | 39 ++++----------------------------------- 1 file changed, 4 insertions(+), 35 deletions(-) diff --git a/doc/METADATA.ini b/doc/METADATA.ini index eb6c55892fc..f5514658bf1 100644 --- a/doc/METADATA.ini +++ b/doc/METADATA.ini @@ -59,7 +59,7 @@ version/minor=6 # # Important metadata often used in configuration files -# (and seldom in specifications) +# and not used in specifications. # [order] @@ -166,13 +166,6 @@ description= Internal metadata that must be ignored by other plugins. Only if a type cannot be represented, e.g., the `toml` plugin would use `internal/toml/type`. -[line] -type= int -status= proposal -description= contains the line number from which a key was read. -note= is useful for validation problems when someone directly - edits files - # @@ -266,7 +259,7 @@ description= For this metadata only the presence is significant, not its value. [array] type= array-index -status= proposed +status= implemented description= Such keys have only keys conforming to Elektra’s arrays convention as subkeys. If the metavalue is empty, the array is empty (no subkeys). Otherwise the metavalue contains the last element. @@ -311,9 +304,9 @@ note= It is generally preferred that other metadata will be used to deduce the needs of the specification, e.g. use check/validation metadata instead of as given above. -[config] +[config/needs] type= string -status= proposed +status= implemented description= indicate that the mount point needs some configuration for plugins in order to work. @@ -764,18 +757,6 @@ description = a number of warnings, see src/error/specification -# -# Struct validation -# - - -[struct] -type = vector -status = proposed -description = checks if the subkeys form a valid struct with the given - names. - The validator makes sure that there are no keys for the struct missing. -example = device mpoint type options [dumpfreq] [passno] # @@ -973,12 +954,6 @@ type= string description= A ternary conditional style operator to define relations between keys -[condition/validsuffix] -status= proposal -usedby/plugin= conditionals -type= string -description= A unit of measure style prefix - [check/condition/any/#] status= implemented usedby/plugin= conditionals @@ -1059,12 +1034,6 @@ type= long status= implemented description= Trigger an error with given error number from src/error/specifications -[trigger/error/nofail] -usedby/plugin= error -type= long -status= proposal -description= Add the error but do not fail (to test robustness of framework). - [check/macaddr] status= implemented type= empty From f290b8e5a863ee1474ed1482952f9e18d48de2bf Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Tue, 17 Nov 2020 10:59:37 +0100 Subject: [PATCH 40/50] decisions: improve ensure API --- doc/decisions/ensure.md | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index 907ffc3eebc..5d2b65a8ebf 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -42,24 +42,36 @@ Integrate `kdbEnsure` in `kdbOpen(Key *errorKey, KeySet *contract)` but only all ## Implications -`elektraNotificationOpen` needs to be removed and instead we need a new API: +`elektraNotificationOpen` will only return a contract KeySet: ```c KeySet * contract = ksNew (0, KS_END); -elektraNotificationGetContract (contract); -KDB * kdb = kdbOpen (key, contract); // contract contains notification config -elektraIoSetBinding (kdb, binding); +elektraNotificationContract (contract, iobinding); ``` -Similar contract getters need to be introduced for other functionality, like gopts: +The same for gopts: + +```c +elektraGOptsContract (contract, argc, argv, environ)); +``` + +Finally, we create `KDB` with the contracts we got before: ```c -KeySet * contract = ksNew (0, KS_END); -elektraGOptsGetContract (contract, argc, argv, environ); KDB * kdb = kdbOpen (key, contract); ``` -The high-level API can make this more pretty, though. +Opening `KDB` will fail if any of the contracts cannot be ensured. + +The cleanup happens within: + +```c +kdbClose (kdb, errorKey); +ksDel (contract); +``` + +It is save to use the contract `KeySet` also for `kdbGet` and `kdbSet` +invocations. ## Related Decisions From 9fd8ae4aa5107c6a772499596eff4b429e4e50ae Mon Sep 17 00:00:00 2001 From: markus2330 Date: Tue, 17 Nov 2020 11:51:05 +0100 Subject: [PATCH 41/50] Update doc/decisions/ensure.md Co-authored-by: kodebach <23529132+kodebach@users.noreply.github.com> --- doc/decisions/ensure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index 5d2b65a8ebf..9e6841f6bec 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -52,7 +52,7 @@ elektraNotificationContract (contract, iobinding); The same for gopts: ```c -elektraGOptsContract (contract, argc, argv, environ)); +elektraGOptsContract (contract, argc, argv, environ); ``` Finally, we create `KDB` with the contracts we got before: From 601ad52150e72df124c7dbbdc8fdb1b941d0dec7 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 19 Nov 2020 08:23:41 +0100 Subject: [PATCH 42/50] decisions: shorten links --- doc/decisions/error_code_implementation.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/doc/decisions/error_code_implementation.md b/doc/decisions/error_code_implementation.md index 6012adbca1c..c53da866183 100644 --- a/doc/decisions/error_code_implementation.md +++ b/doc/decisions/error_code_implementation.md @@ -2,7 +2,8 @@ ## Problem -In the previous error concept it was very useful to generate macros as we often added new errors. The code generation itself, however, is quite complicated (C++ code that prints the code; which is also not ideal for cross compilation, see also https://github.com/ElektraInitiative/libelektra/issues/2814). +In the previous error concept it was very useful to generate macros as we often added new errors. +The code generation itself, however, was not ideal for cross compilation. ## Constraints @@ -15,9 +16,9 @@ In the previous error concept it was very useful to generate macros as we often - Migrate to a more modern and easier way to generate code with our mustache system, and let this generate the (mapping) code for all compiled languages (C, C++, Java, Rust, Go). All we get out of this is the removal of `std::cout << ...` code from C++ but not much more. -- Migrate to CMake code that generates such macros/classes (see also https://github.com/ElektraInitiative/libelektra/issues/2814) +- Migrate to CMake code that generates such macros/classes. Mustache templates have to be supplied with the input data somehow. Either we have to use a custom executable that is compiled at build time. - In that case we would just get rid of the std::cout << ... in the C++ code, but not much else would change. + In that case we would just get rid of the `std::cout << ...` in the C++ code, but not much else would change. The other option is to use the default mustache executable, which is a Ruby script and therefore requires Ruby to be installed. Also kdb gen cannot be reused, since that would require compiling kdb first, which needs kdberrors.h. @@ -44,5 +45,5 @@ When adding a new error code, language bindings have to be adapted accordingly s ## Notes -- https://github.com/ElektraInitiative/libelektra/issues/2871 -- https://github.com/ElektraInitiative/libelektra/issues/2814 +- See also [Issue #2814](https://issues.libelektra.org/2814) +- See also [Issue #2871](https://issues.libelektra.org/2871) From c6a4c9b9fb5237537a1410115ce35fe2084fd452 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 19 Nov 2020 08:59:39 +0100 Subject: [PATCH 43/50] decisions: added error_semantics --- doc/decisions/README.md | 1 + doc/decisions/error_semantics.md | 56 ++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 doc/decisions/error_semantics.md diff --git a/doc/decisions/README.md b/doc/decisions/README.md index 89899823a34..ff10943c365 100644 --- a/doc/decisions/README.md +++ b/doc/decisions/README.md @@ -61,6 +61,7 @@ section here. - [Plugin Variants](plugin_variants.md) - [Global Validation](global_validation.md) - [Spec Expressiveness](spec_expressiveness.md) +- [Storage Plugin Semantics](error_semantics.md) ## Rejected diff --git a/doc/decisions/error_semantics.md b/doc/decisions/error_semantics.md new file mode 100644 index 00000000000..57090790757 --- /dev/null +++ b/doc/decisions/error_semantics.md @@ -0,0 +1,56 @@ +# Error Semantics + +## Problem + +While we have a classification of errors and warnings, it remains +unclear when plugins actually should emit errors and warnings. + +## Constraints + +- Should not be contradicting to specified behavior in [storage plugin tutorial](/doc/tutorials/storage-plugins.md). + +## Assumptions + +- Users want a uniform behavior within Elektra, so plugins must behave uniformly. + +## Considered Alternatives + +- freedom to plugin writers +- strict rules and conformance tests for plugins + +## Decision + +Provide guidelines in the form as tutorials, covering: + +- that any not understood metadata (e.g. types), should lead to an error +- if the value does not confirm **exactly** to the specified type, an error should be emitted + (e.g. only `0` or `1` as boolean) +- anything else that is beyond the capabilities of a plugin, should lead to an error, e.g.: + - not supported structure + - not supported (meta) key names + +Violations against these guidelines can be reported as bug and then either: + +- the bug gets fixed +- the plugin get a worse `infos/status` but still get shipped with 1.0 +- the plugin gets removed + +## Rationale + +It is easier for developers if there are clear expectations on how a plugin +should behave. And it is much easier for overall Elektra if there is more +consistency. + +## Implications + +- more checks&errors in storage plugins are needed + +## Related Decisions + +- [Metadata in Spec Namespace](spec_metadata.md) +- [Capabilities](capabilities.md) +- [Boolean](boolean.md) + +## Notes + +- [Issue #1511](https://issues.libelektra.org/1511) From dabdae0fc5e81ed3bae7596ec4b54026a567a975 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 19 Nov 2020 09:08:31 +0100 Subject: [PATCH 44/50] decision: make clear that API is an open point --- doc/decisions/escaped_name.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/decisions/escaped_name.md b/doc/decisions/escaped_name.md index 19d1dbe6d2a..fe2ad3a43cc 100644 --- a/doc/decisions/escaped_name.md +++ b/doc/decisions/escaped_name.md @@ -19,6 +19,10 @@ functions handling out the pointer). - store it twice - store it once but in one memory block +## Decision + +Remove the escaped name from `struct _Key` and use it only when necessary. + ## Open Points Clarify and reduce [terminology](/doc/help/elektra-glossary.md): @@ -29,9 +33,8 @@ Clarify and reduce [terminology](/doc/help/elektra-glossary.md): - base name - dir name -## Decision +API Changes: -- remove the escaped name from `_Key` - `keyName` returns the unescaped name - remove `keyUnescapedName`, `keyGetUnescapedNameSize`, `keyGetFullName`. - reverse terminology: with "key name" we will refer to the unescaped (base) name, @@ -39,7 +42,6 @@ Clarify and reduce [terminology](/doc/help/elektra-glossary.md): "full name" (a variant of the escaped name) will not be used anymore. - escaped name will be only present in `keyNew` and `keyGetEscapedName` - rename `keyAddName`, e.g. to `keyAddEscapedName` - (or `keyAddFullName` if we decide to call the escaped name full name) - `keyDup` with arguments to filter which parts are copied (to allow copy of keys where only the key name is copied) From 289a9aea7f95400aff2fe0101c595444d49204d0 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 19 Nov 2020 09:12:46 +0100 Subject: [PATCH 45/50] decisions: clarify API --- doc/decisions/ensure.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index 9e6841f6bec..3f864f33d96 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -63,11 +63,17 @@ KDB * kdb = kdbOpen (key, contract); Opening `KDB` will fail if any of the contracts cannot be ensured. -The cleanup happens within: +As the `contract` gets copied, at any point after `kdbOpen` the +contract can be safely deleted: + +``` +ksDel (contract); +``` + +The cleanup of the global plugins happens within: ```c kdbClose (kdb, errorKey); -ksDel (contract); ``` It is save to use the contract `KeySet` also for `kdbGet` and `kdbSet` From a13a3621d834aa1945cb7f6d48fabe0bb10607e3 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 19 Nov 2020 11:09:14 +0100 Subject: [PATCH 46/50] spelling: fix+comment how to apply --- doc/decisions/spec_metadata.md | 2 +- scripts/sed | 2 ++ tests/shell/check_spelling.sh | 1 + 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/decisions/spec_metadata.md b/doc/decisions/spec_metadata.md index 06b624390c4..129122387d3 100644 --- a/doc/decisions/spec_metadata.md +++ b/doc/decisions/spec_metadata.md @@ -30,7 +30,7 @@ metadata from `spec:` might end up in `user:`. for restoring configuration file formats. Rename metadata `` in `spec:/` to `check/` or `make/array`. - This way no merging of meta-data would be needed and + This way no merging of metadata would be needed and by the name alone it would be clear to which namespace it belongs. - Do not support `array` or `type` if the underlying diff --git a/scripts/sed b/scripts/sed index 4861bf5f91a..65ab51d540a 100644 --- a/scripts/sed +++ b/scripts/sed @@ -1,3 +1,5 @@ +# use scripts/dev/fix-spelling to apply these corrections + # ================= # = General Terms = # ================= diff --git a/tests/shell/check_spelling.sh b/tests/shell/check_spelling.sh index c842ac1b0dc..7a4ccd94bfd 100755 --- a/tests/shell/check_spelling.sh +++ b/tests/shell/check_spelling.sh @@ -15,6 +15,7 @@ scripts/dev/fix-spelling git diff --exit-code succeed_if 'Please commit the spelling fixes before pushing. +Use scripts/dev/fix-spelling to make the changes, or apply the patch shown above. If one of the spelling fixes was incorrect, then please update `/scripts/sed` accordingly.' printf '\n' From 3b7bfd66227666fdb4872cf815c6ffcccf36e024 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 19 Nov 2020 12:42:27 +0100 Subject: [PATCH 47/50] decisions: also be strict about specifications --- doc/decisions/error_semantics.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/decisions/error_semantics.md b/doc/decisions/error_semantics.md index 57090790757..2640382a374 100644 --- a/doc/decisions/error_semantics.md +++ b/doc/decisions/error_semantics.md @@ -23,6 +23,7 @@ unclear when plugins actually should emit errors and warnings. Provide guidelines in the form as tutorials, covering: - that any not understood metadata (e.g. types), should lead to an error +- that wrong specifications, like `kdb meta-set /tests/ipaddr/ipv4 check/ipaddr ipv8` should be rejected - if the value does not confirm **exactly** to the specified type, an error should be emitted (e.g. only `0` or `1` as boolean) - anything else that is beyond the capabilities of a plugin, should lead to an error, e.g.: From 755a7629011033bc2ca70701806feebbe0470a0e Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Wed, 2 Dec 2020 18:14:08 +0100 Subject: [PATCH 48/50] decisions: as discussed during meeting --- doc/decisions/README.md | 34 +++++++++++++++++++------------- doc/decisions/error_semantics.md | 6 +++--- doc/decisions/escaped_name.md | 22 ++++++++------------- doc/decisions/global_plugins.md | 4 ++++ doc/decisions/semantics_name.md | 4 ++-- doc/decisions/vendor_spec.md | 1 - 6 files changed, 37 insertions(+), 34 deletions(-) diff --git a/doc/decisions/README.md b/doc/decisions/README.md index ff10943c365..fa5582dbb29 100644 --- a/doc/decisions/README.md +++ b/doc/decisions/README.md @@ -37,31 +37,36 @@ section here. - [High Level API](high_level_api.md) - [Error codes](error_codes.md) - [Error code implementation](error_code_implementation.md) +- [Semantics for Name](semantics_name.md) +- [Base Name](base_name.md) +- [Lookup every key](lookup_every_key.md) +- [Holes and Non-leaf values](holes.md) +- [Multiple File Backends](multiple_file_backends.md) +- [Boolean](boolean.md) + +## In Progress + +- [Global Plugins](global_plugins.md) (@mpranj) +- [Ensure](ensure.md) (@kodebach) +- [Capabilities](capabilities.md) (@markus2330) +- [Error Semantics](error_semantics.md) (API) ## Decided - [Array for Warnings](warning_array.md) -- [Vendor Spec](vendor_spec.md) +- [Array](array.md) ## In Discussion -- [Semantics for Name](semantics_name.md) -- [Base Name](base_name.md) -- [Escaped Name](escaped_name.md) +- [Escaped Name](escaped_name.md) merge with: - [Store the escaped and/or unescaped key name](store_name.md) -- [Lookup every key](lookup_every_key.md) -- [Holes and Non-leaf values](holes.md) +- [Spec Expressiveness](spec_expressiveness.md) - [Metadata in Spec Namespace](spec_metadata.md) -- [Multiple File Backends](multiple_file_backends.md) -- [Capabilities](capabilities.md) -- [Array](array.md) -- [Boolean](boolean.md) -- [Global Plugins](global_plugins.md) -- [Ensure](ensure.md) + +## Delayed + - [Plugin Variants](plugin_variants.md) - [Global Validation](global_validation.md) -- [Spec Expressiveness](spec_expressiveness.md) -- [Storage Plugin Semantics](error_semantics.md) ## Rejected @@ -69,3 +74,4 @@ section here. - [Null Pointer Checks](null_pointer_checks.md) - [Elektra Web Publish Subscribe](elektra_web_pubsub.md) - [Internal Cache](internal_cache.md) +- [Vendor Spec](vendor_spec.md) diff --git a/doc/decisions/error_semantics.md b/doc/decisions/error_semantics.md index 2640382a374..cbc990484b0 100644 --- a/doc/decisions/error_semantics.md +++ b/doc/decisions/error_semantics.md @@ -22,13 +22,13 @@ unclear when plugins actually should emit errors and warnings. Provide guidelines in the form as tutorials, covering: +- prefer errors to warnings - that any not understood metadata (e.g. types), should lead to an error - that wrong specifications, like `kdb meta-set /tests/ipaddr/ipv4 check/ipaddr ipv8` should be rejected - if the value does not confirm **exactly** to the specified type, an error should be emitted (e.g. only `0` or `1` as boolean) -- anything else that is beyond the capabilities of a plugin, should lead to an error, e.g.: - - not supported structure - - not supported (meta) key names +- anything else that is beyond the capabilities of a plugin (not implemented), + should lead to an error Violations against these guidelines can be reported as bug and then either: diff --git a/doc/decisions/escaped_name.md b/doc/decisions/escaped_name.md index fe2ad3a43cc..12d7327947d 100644 --- a/doc/decisions/escaped_name.md +++ b/doc/decisions/escaped_name.md @@ -23,26 +23,20 @@ functions handling out the pointer). Remove the escaped name from `struct _Key` and use it only when necessary. -## Open Points - -Clarify and reduce [terminology](/doc/help/elektra-glossary.md): - -- escaped name -- full name -- part name -- base name -- dir name +Clarify and reduce [terminology](/doc/help/elektra-glossary.md). API Changes: - `keyName` returns the unescaped name -- remove `keyUnescapedName`, `keyGetUnescapedNameSize`, `keyGetFullName`. + (temporary some other name for PR: `keyUnescapedName(Size)`) +- remove `keyUnescapedName`, `keyGetUnescapedNameSize`. - reverse terminology: with "key name" we will refer to the unescaped (base) name, the escaped name will be explicitly called "escaped key name". - "full name" (a variant of the escaped name) will not be used anymore. -- escaped name will be only present in `keyNew` and `keyGetEscapedName` -- rename `keyAddName`, e.g. to `keyAddEscapedName` -- `keyDup` with arguments to filter which parts are copied +- escaped name will be only present in + - `keyNew` (+ arguments for adding key names) [unclear: maybe not needed] + - `elektraEscapeName` (operating on chars) + - rename `keyAddName`, e.g. to `keyAddEscapedName` +- `keyDup(.., int)` with options to filter which parts are copied (to allow copy of keys where only the key name is copied) ## Rationale diff --git a/doc/decisions/global_plugins.md b/doc/decisions/global_plugins.md index e4dcd38972d..e4d7186ae99 100644 --- a/doc/decisions/global_plugins.md +++ b/doc/decisions/global_plugins.md @@ -42,6 +42,8 @@ We need to cleanup and simplify the placement. Have hooks and API specific to the list of global plugins in assumptions. These hooks are not shared, so no `list` plugin is needed. +Installed plugins will be used. + ## Rationale - allows adding more types of plugins later, also post-1.0 @@ -50,6 +52,8 @@ These hooks are not shared, so no `list` plugin is needed. ## Implications +- remove `global-mount` command +- command for mmap/notification/... enable disable (like current `kdb cache` tool) - remove `list` plugin - remove plugins that stop working or disallow global positioning for them - call `spec` as needed several times diff --git a/doc/decisions/semantics_name.md b/doc/decisions/semantics_name.md index 0c6b2bdc972..9f44fa2a9aa 100644 --- a/doc/decisions/semantics_name.md +++ b/doc/decisions/semantics_name.md @@ -24,8 +24,8 @@ Nevertheless, there are guidelines (without any checks in `keySetBaseName`): - `#` is used to indicate that array numbers follow. - `®` is used to indicate that some information was encoded in the key name. This is usually only needed internally in storage plugins. -- the UTF-8 sequence `®elektra` (i.e. the 9-byte sequence `C2 AE 65 6C 65 6B 74 72 61`) is reserved, - key name parts should not start with that sequence. +- The UTF-8 sequence `®elektra` (i.e. the 9-byte sequence `C2 AE 65 6C 65 6B 74 72 61`) is reserved, + see key name docu. There are, however, rules and conventions which syntax to use for specific semantics. The `spec` plugin guards these rules. diff --git a/doc/decisions/vendor_spec.md b/doc/decisions/vendor_spec.md index 3336fae9130..010b44d7627 100644 --- a/doc/decisions/vendor_spec.md +++ b/doc/decisions/vendor_spec.md @@ -25,7 +25,6 @@ and being administer friendly. As found out during implementation of [specload](/src/plugins/specload), only a very limited subset can be modified safely, e.g.: - add/edit/remove `description`, `opt/help` and `comment` -- add/edit `default` ## Rationale From 400bdc09118544dc8e0752c95b3157278ad49404 Mon Sep 17 00:00:00 2001 From: Markus Raab Date: Thu, 3 Dec 2020 10:48:57 +0100 Subject: [PATCH 49/50] metadata: readd as it is actually implemented --- doc/METADATA.ini | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/doc/METADATA.ini b/doc/METADATA.ini index f5514658bf1..1d1c7161e51 100644 --- a/doc/METADATA.ini +++ b/doc/METADATA.ini @@ -972,6 +972,12 @@ usedby/plugin= conditionals type= string description= An array of multiple condition statements +[condition/validsuffix] +status= implemented +usedby/plugin= conditionals +type= string +description= A unit of measure style prefix + [assign/condition] status= implemented usedby/plugin= conditionals @@ -1034,6 +1040,12 @@ type= long status= implemented description= Trigger an error with given error number from src/error/specifications +[trigger/error/nofail] +usedby/plugin= error +type= long +status= implemented +description= Add the error but do not fail (to test robustness of framework). + [check/macaddr] status= implemented type= empty From b043ef85604cdb0c6eb15eb82421a9a960606571 Mon Sep 17 00:00:00 2001 From: markus2330 Date: Thu, 3 Dec 2020 14:26:39 +0100 Subject: [PATCH 50/50] decisions: fix typos MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Mihael Pranjić --- doc/GETSTARTED.md | 6 +++--- doc/decisions/array.md | 2 +- doc/decisions/ensure.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/GETSTARTED.md b/doc/GETSTARTED.md index 11de8f60e3b..7c0e2f7535e 100644 --- a/doc/GETSTARTED.md +++ b/doc/GETSTARTED.md @@ -61,13 +61,13 @@ cmake --build build -- -j5 ``` - Optional you can also run tests, see [here for more information](/doc/TESTING.md): + Optionally you can also run tests, see [here for more information](/doc/TESTING.md): ```sh cmake --build build --target run_nokdbtests ``` - With these commands you will be able to run the "Hello World!" example but usually you will need to use some of the [plugins](/src/plugins/README.md), tools and bindings of Elektra. Please take a look to the more detailed [compiling documentation](/doc/COMPILE.md). After you completed building Elektra on your own, you can execute these commands to install Elektra (please check the [installation documentation](/doc/INSTALL.md) for the many available packages): + With these commands you will be able to run the "Hello World!" example but usually you will need to use some of the [plugins](/src/plugins/README.md), tools and bindings of Elektra. Please take a look at the more detailed [compiling documentation](/doc/COMPILE.md). After you completed building Elektra on your own, you can execute these commands to install Elektra (please check the [installation documentation](/doc/INSTALL.md) for the many available packages): ```sh sudo make install @@ -76,7 +76,7 @@ [Installation documentation](/doc/INSTALL.md) contains further information about available packages. - Optional you can also run tests to verify the installed Elektra, see [here for more information](/doc/TESTING.md): + Optionally you can also run tests to verify the installed Elektra, see [here for more information](/doc/TESTING.md): ```sh kdb run_nokdbtests diff --git a/doc/decisions/array.md b/doc/decisions/array.md index a8b5b47903c..2ad925c5160 100644 --- a/doc/decisions/array.md +++ b/doc/decisions/array.md @@ -82,7 +82,7 @@ system:/myarray # <- not found in cascading lookup, as user:/myarray exists The `spec` plugin should check if it is a valid array, i.e.: -- that the parent key always contain the metadata `array`, +- that the parent key always contains the metadata `array`, - that the correct length is in `array`, - that the array only contains `#` children, and - that the children are numbered from `#0` to `#n`, without holes. diff --git a/doc/decisions/ensure.md b/doc/decisions/ensure.md index 3f864f33d96..eaf9da0ed5f 100644 --- a/doc/decisions/ensure.md +++ b/doc/decisions/ensure.md @@ -76,7 +76,7 @@ The cleanup of the global plugins happens within: kdbClose (kdb, errorKey); ``` -It is save to use the contract `KeySet` also for `kdbGet` and `kdbSet` +It is safe to use the contract `KeySet` also for `kdbGet` and `kdbSet` invocations. ## Related Decisions