Merge pull request #2944 from kodebach/codegen

Codegen: Ignore missing required keys in help mode

doc/decisions/README.md

@@ -21,6 +21,7 @@ section here.
 - [Elektra Web Structure](elektra_web.md)
 - [Elektra Web Recursive Structure](elektra_web_recursive.md)
 - [Cryptographic Key Handling](cryptograhic_key_handling.md)
+- [High-level API Help Message](highlevel_help_message.md)
 
 ## Decided
 

doc/decisions/highlevel_help_message.md

@@ -0,0 +1,54 @@
+# High-level API Help Message
+
+This decision _does not_ assume code-generation is used. For the case of code-generation see the [Notes](#notes) section.
+
+## Problem
+
+We want to allow to print the help message no matter what errors happened in `kdbOpen` or `kdbGet`.
+
+## Constraints
+
+- `elektraOpen` should not return a broken `Elektra` instance.
+- The help message can only be printed, if `elektraOpen` returns an `Elektra` instance and no `ElektraError`.
+
+## Assumptions
+
+- We assume that the application in question was correctly installed.
+- We assume `gopts` was mounted.
+- We assume the application was called in _help mode_, i.e. with `-h` or `--help`.
+  Otherwise printing the help message is not possible, anyway.
+
+## Considered Alternatives
+
+- Ignore all errors (in help mode):
+  Not a feasible solution, because there may have been problems when reading the storage file and therefore,
+  the help message may be broken or incomplete.
+- Ignore all errors (in help mode), which occurred after the `gopts` plugin ran:
+  Complicated to implement (we need to know about plugin order, etc.).
+  Not actually necessary (see [Rationale](#rationale)).
+
+## Decision
+
+Ignore missing `require`d keys (in help mode), but fail for every other error.
+
+## Rationale
+
+Required keys **must** be provided by the user/admin and cannot come from another source (Elektra, app developer, etc.).
+Therefore they will be missing until the user makes changes to the KDB. Before that, no other error should occur (we
+assumed a correct installation). If a user runs `app` for the first time and receives an error about a missing required
+key, they will:
+
+- know what to do and add the key, thereby fixing the problem.
+- try `app -h`/`app --help` to find out more. The help message may or may not contain useful information. If not they may try 3.
+- read some other documentation to find out more. Ideally this leads them to 1.
+
+In any case after this the user definitely know how to interact with the KDB. Since we assumed that there won't be any
+errors before the KDB was changed, we can assume that the user caused other errors by changing the KDB.
+
+## Notes
+
+If code-generation is used, the situation is a little different. If the parameter `embedHelpFallback` is set to `1`, a
+fallback help message will be created from the specification originally passed to the code-generator and embedded into
+the application. The parameter also changes, how help mode is detected and ultimately allows the help message function
+(`printHelpMessage` by default) to always print a help message. Although it may not reflect changes the user made to the
+specification.

doc/help/kdb-gen-highlevel.md

@@ -60,6 +60,10 @@ For detailed information about the contents of the header file see [elektra-high
   Changes the name of the function that checks for "specload mode" (default: `exitForSpecload`)
 - `tagPrefix`:
   Changes the prefix of the generated tags (default: `ELEKTRA_TAG_`)
+- `embedHelpFallback`:
+  Switches whether a fallback help message should be embedded; allowed values: `1` (default), `0`
+  If enabled (`1`), a help message will be generated from the specification passed to the code-generator and embedded
+  into the application. This message will be used, if the normal help message could not be generated at runtime.
 - `enumConv`:
   Switches how enum conversion should be done; allowed values: `default` (default), `switch`, `strcmp`
   - `strcmp`: uses a simple series of `if (strcmp(*, *) == 0)` to convert strings into enums
@@ -69,7 +73,7 @@ For detailed information about the contents of the header file see [elektra-high
   Comma-separated (`,`) list of additional header files to include. For each of the listed headers we will generate an `#include "*"`
   statement
 - `genSetters`:
-  Switches whether setters should be generated at all; allowed values: `true` (default), `false`
+  Switches whether setters should be generated at all; allowed values: `1` (default), `0`
 - `embeddedSpec`:
   Changes how much of the specification is embedded into the application; allowed values: `full` (default), `defaults`, `none`.
   see [elektra-highlevel-gen(7)](elektra-highlevel-gen.md)

doc/man/man1/kdb-gen-highlevel.1

@@ -80,6 +80,9 @@ For detailed information about the contents of the header file see elektra\-high
 \fBtagPrefix\fR: Changes the prefix of the generated tags (default: \fBELEKTRA_TAG_\fR)
 .
 .IP "\(bu" 4
+\fBembedHelpFallback\fR: Switches whether a fallback help message should be embedded; allowed values: \fB1\fR (default), \fB0\fR If enabled (\fB1\fR), a help message will be generated from the specification passed to the code\-generator and embedded into the application\. This message will be used, if the normal help message could not be generated at runtime\.
+.
+.IP "\(bu" 4
 .
 .IP "\(bu" 4
 \fBstrcmp\fR: uses a simple series of \fBif (strcmp(*, *) == 0)\fR to convert strings into enums
@@ -97,7 +100,7 @@ For detailed information about the contents of the header file see elektra\-high
 \fBheaders\fR: Comma\-separated (\fB,\fR) list of additional header files to include\. For each of the listed headers we will generate an \fB#include "*"\fR statement
 .
 .IP "\(bu" 4
-\fBgenSetters\fR: Switches whether setters should be generated at all; allowed values: \fBtrue\fR (default), \fBfalse\fR
+\fBgenSetters\fR: Switches whether setters should be generated at all; allowed values: \fB1\fR (default), \fB0\fR
 .
 .IP "\(bu" 4
 \fBembeddedSpec\fR: Changes how much of the specification is embedded into the application; allowed values: \fBfull\fR (default), \fBdefaults\fR, \fBnone\fR\. see elektra\-highlevel\-gen(7) \fIelektra\-highlevel\-gen\.md\fR

doc/news/_preparation_next_release.md

@@ -38,7 +38,7 @@ If you specifically want to use it with the High-Level API take a look at [this
 
 We also created a new CMake function that will be available, if you include Elektra via CMake's
 `find_package`. The function is called `elektra_kdb_gen` and can be used to tell CMake about files
-that are generated with `kdb gen`. _(Klemens Böswirth)_
+that are generated via `kdb gen`. _(Klemens Böswirth)_
 
 ### <<HIGHLIGHT2>>
 
@@ -89,6 +89,10 @@ plugins. _(René Schwaiger)_
 
 - We now treat relative paths as relative to `KDB_DB_SPEC` instead of the current working directory. _(Klemens Böswirth)_
 
+### Spec
+
+- There is now the config key `missing/log` that allows logging of all missing `require`d keys. _(Klemens Böswirth)_
+
 ## Libraries
 
 The text below summarizes updates to the [C (and C++)-based libraries](https://www.libelektra.org/libraries/readme) of Elektra.

src/include/kdbopts.h

@@ -12,7 +12,18 @@
 
 #include <kdb.h>
 
+#ifdef __cplusplus
+namespace ckdb
+{
+extern "C" {
+#endif
+
 int elektraGetOpts (KeySet * ks, int argc, const char ** argv, const char ** envp, Key * parentKey);
 char * elektraGetOptsHelpMessage (Key * errorKey, const char * usage, const char * prefix);
 
+#ifdef __cplusplus
+}
+}
+#endif
+
 #endif // ELEKTRA_KDBOPTS_H

src/libs/ease/symbols.map

@@ -10,7 +10,9 @@ libelektra_0.8 {
 	elektraKeyGetRelativeName;
 	elektraKsFilter;
 	elektraResolveReference;
+};
 
+libelektra_0.9 {
 	## ToString;
 	elektraBooleanToString;
 	elektraCharToString;

src/libs/highlevel/elektra.c

@@ -68,16 +68,24 @@ Elektra * elektraOpen (const char * application, KeySet * defaults, KeySet * con
 	if (kdb == NULL)
 	{
 		*error = elektraErrorFromKey (parentKey);
+		keyDel (parentKey);
 		return NULL;
 	}
 
+	int ignoreRequireInHelpMode = 0;
+
 	if (contract != NULL)
 	{
 		Key * contractCut = keyNew ("system/elektra/highlevel", KEY_END);
 		KeySet * highlevelContract = ksCut (contract, contractCut);
 
 		if (ksGetSize (highlevelContract) > 0)
 		{
+			if (ksLookupByName (highlevelContract, "system/elektra/highlevel/helpmode/ignore/require", 0) != NULL)
+			{
+				ignoreRequireInHelpMode = 1;
+			}
+
 			if (!checkHighlevelContract (application, highlevelContract, error))
 			{
 				keyDel (contractCut);
@@ -99,6 +107,7 @@ Elektra * elektraOpen (const char * application, KeySet * defaults, KeySet * con
 			     keyNew ("system/elektra/ensure/plugins/global/spec/config/conflict/get", KEY_VALUE, "ERROR", KEY_END));
 		ksAppendKey (contract,
 			     keyNew ("system/elektra/ensure/plugins/global/spec/config/conflict/set", KEY_VALUE, "ERROR", KEY_END));
+		ksAppendKey (contract, keyNew ("system/elektra/ensure/plugins/global/spec/config/missing/log", KEY_VALUE, "1", KEY_END));
 
 		const int kdbEnsureResult = kdbEnsure (kdb, contract, parentKey);
 
@@ -131,8 +140,31 @@ Elektra * elektraOpen (const char * application, KeySet * defaults, KeySet * con
 
 	if (kdbGetResult == -1)
 	{
-		*error = elektraErrorFromKey (parentKey);
-		return NULL;
+		Key * helpKey = ksLookupByName (config, "proc/elektra/gopts/help", 0);
+		const Key * missingKeys = keyGetMeta (parentKey, "logs/spec/missing");
+		if (ignoreRequireInHelpMode == 1 && helpKey != NULL && missingKeys != NULL)
+		{
+			// proc/elektra/gopts/help was set -> we are in help mode
+			// logs/spec/missing exists on parentKey -> spec detected missing keys
+			// we ensured that spec uses conflict/get = ERROR -> the error in kdbGet must be from spec
+			// --> we are in the error case that should be ignored
+
+			// BUT: anything other than helpKey may be incorrect
+			// and only helpKey should be used anyway
+			// so create a new config KeySet
+			Key * helpKeyDup = keyDup (helpKey);
+			ksClear (config);
+			ksAppendKey (config, helpKeyDup);
+		}
+		else
+		{
+			*error = elektraErrorFromKey (parentKey);
+
+			ksDel (config);
+			kdbClose (kdb, parentKey);
+			keyDel (parentKey);
+			return NULL;
+		}
 	}
 
 	Elektra * const elektra = elektraCalloc (sizeof (struct _Elektra));

src/plugins/gopts/gopts.c

@@ -34,6 +34,35 @@ static void cleanupEnvp (char ** envp);
 #error "No implementation available"
 #endif
 
+/**
+ * Detects whether we are in help mode or not.
+ * DOES NOT set 'proc/elektra/gopts/help' for use with elektraGetOptsHelpMessage().
+ *
+ * @retval 1 if the -h or --help is part of argv
+ * @retval 0 otherwise
+ * @retval -1 on error (could not load argv)
+ */
+int elektraGOptsIsHelpMode (void)
+{
+	char ** argv = NULL;
+	int argc = loadArgs (&argv);
+
+	if (argv == NULL)
+	{
+		return -1;
+	}
+
+	for (int i = 0; i < argc; ++i)
+	{
+		if (strcmp (argv[i], "-h") == 0 || strcmp (argv[i], "--help") == 0)
+		{
+			return 1;
+		}
+	}
+
+	return 0;
+}
+
 
 int elektraGOptsGet (Plugin * handle ELEKTRA_UNUSED, KeySet * returned, Key * parentKey)
 {
@@ -43,6 +72,7 @@ int elektraGOptsGet (Plugin * handle ELEKTRA_UNUSED, KeySet * returned, Key * pa
 			ksNew (30, keyNew ("system/elektra/modules/gopts", KEY_VALUE, "gopts plugin waits for your orders", KEY_END),
 			       keyNew ("system/elektra/modules/gopts/exports", KEY_END),
 			       keyNew ("system/elektra/modules/gopts/exports/get", KEY_FUNC, elektraGOptsGet, KEY_END),
+			       keyNew ("system/elektra/modules/gopts/exports/ishelpmode", KEY_FUNC, elektraGOptsIsHelpMode, KEY_END),
 #include ELEKTRA_README
 			       keyNew ("system/elektra/modules/gopts/infos/version", KEY_VALUE, PLUGINVERSION, KEY_END), KS_END);
 		ksAppend (returned, contract);

src/plugins/spec/README.md

@@ -133,6 +133,10 @@ The allowed values for the conflict keys are always the same:
   `logs/spec/info` is an array, each element is one conflict.
 - any other value ignores the conflict, this includes if the conflict key is not given (i.e. the default)
 
+There is also the special key `missing/log`. If it is set to `1`, the plugin will create the meta array `logs/spec/missing/#`.
+This array will contain a list of the missing keys. The key `missing/log` can only be part of the main plugin configuration,
+not individual keys' metakeys. It also applies to `kdbGet` and `kdbSet` calls.
+
 ## Examples
 
 Ini files can be found in [/examples/spec](/examples/spec) which should be PWD

src/plugins/spec/spec.c

@@ -54,6 +54,7 @@ typedef struct
 	OnConflict conflict;
 	OnConflict range;
 	OnConflict missing;
+	int logMissing;
 } ConflictHandling;
 
 static void copyMeta (Key * dest, Key * src);
@@ -128,6 +129,9 @@ static void parseConfig (KeySet * config, ConflictHandling * ch, const char base
 	strcpy (nameBufferEnd, "/missing");
 	key = ksLookupByName (config, nameBuffer, 0);
 	ch->missing = key == NULL ? base : parseOnConflictKey (key);
+
+	key = ksLookupByName (config, "/missing/log", 0);
+	ch->logMissing = key != NULL && strcmp (keyString (key), "1") == 0;
 }
 
 static void parseLocalConfig (Key * specKey, ConflictHandling * ch, bool isKdbGet)
@@ -801,13 +805,19 @@ static int processSpecKey (Key * specKey, Key * parentKey, KeySet * ks, const Co
 	{
 		if (require)
 		{
-			char * msg = elektraFormat ("Required key %s is missing.", strchr (keyName (specKey), '/'));
+			const char * missing = strchr (keyName (specKey), '/');
+			char * msg = elektraFormat ("Required key %s is missing.", missing);
 			handleConflict (parentKey, msg, ch->missing);
 			elektraFree (msg);
 			if (ch->missing != IGNORE)
 			{
 				ret = -1;
 			}
+
+			if (ch->logMissing)
+			{
+				elektraMetaArrayAdd (parentKey, "logs/spec/missing", missing);
+			}
 		}
 
 		if (isKdbGet)

src/plugins/spec/testmod_spec.c

@@ -172,6 +172,45 @@ static void test_require (void)
 	ksDel (_conf);
 }
 
+static void test_logMissing (void)
+{
+	printf ("test logMissing\n");
+
+	KeySet * _conf = ksNew (2, keyNew ("user/conflict/get", KEY_VALUE, "ERROR", KEY_END),
+				keyNew ("user/missing/log", KEY_VALUE, "1", KEY_END), KS_END);
+
+	TEST_BEGIN
+	{
+		KeySet * ks = ksNew (10, keyNew ("spec" PARENT_KEY "/a", KEY_META, "require", "", KEY_END), KS_END);
+
+		TEST_CHECK (plugin->kdbGet (plugin, ks, parentKey) == ELEKTRA_PLUGIN_STATUS_ERROR, "kdbGet shouldn't succeed");
+
+		succeed_if (keyGetMeta (parentKey, "logs/spec/missing") != NULL, "missing key should have been logged");
+		succeed_if_same_string (keyString (keyGetMeta (parentKey, "logs/spec/missing")), "#0");
+
+		succeed_if (keyGetMeta (parentKey, "logs/spec/missing/#0") != NULL, "missing key should have been logged");
+		succeed_if_same_string (keyString (keyGetMeta (parentKey, "logs/spec/missing/#0")), PARENT_KEY "/a");
+
+		ksDel (ks);
+	}
+	TEST_END
+
+	TEST_BEGIN
+	{
+		KeySet * ks = ksNew (10, keyNew ("spec" PARENT_KEY "/a", KEY_META, "require", "", KEY_END),
+				     keyNew ("user" PARENT_KEY "/a", KEY_END), KS_END);
+
+		TEST_CHECK (plugin->kdbGet (plugin, ks, parentKey) == ELEKTRA_PLUGIN_STATUS_SUCCESS, "kdbGet failed");
+		TEST_ON_FAIL (output_error (parentKey));
+
+		succeed_if (keyGetMeta (parentKey, "logs/spec/missing") == NULL, "missing key should not have been logged");
+
+		ksDel (ks);
+	}
+	TEST_END
+	ksDel (_conf);
+}
+
 static void test_array (void)
 {
 	printf ("test array\n");
@@ -573,6 +612,7 @@ int main (int argc, char ** argv)
 	test_assign_condition ();
 	test_wildcard ();
 	test_require ();
+	test_logMissing ();
 	test_array ();
 	test_require_array ();
 	test_array_member ();

src/tools/kdb/CMakeLists.txt

@@ -36,6 +36,7 @@ if (BUILD_SHARED)
 			       elektra-core
 			       elektra-kdb
 			       elektratools
+			       elektra-opts
 			       elektra-merge)
 
 	install (TARGETS kdb DESTINATION bin)