Support for complex types (lists, dictionaries, integers) for dynamic config values

[Kami]

Problem Description

Right now dynamic configuration values (ones which are referenced using Jinja variable notation inside the config file and are read from the datastore) can only be strings (this limitation is of course clearly documented and pointed out inside the documentation, blog post, etc.). Non-string types are not supported directly.

In theory, users could handle the serialization of complex types themselves and then de-serialize it inside actions, but that's obviously not ideal (it's not consistent and each user who requires this functionality would need to re-invent the wheel, etc.).

Obviously that's quite limiting and it would make sense to also support other commonly used types such as lists, arrays, dictionaries and integers / numbers.

Background

First prototype of dynamic configuration values actually supported arbitrary types, but per discussion with @lakshmi-kannan and @dzimine we decided to simplify it and only support strings.

Proposed Solution

I propose the same approach as I already mostly implemented in the first prototype. This means we should provide a special API (and related CLI commands) to manage dynamic config values. This API would transparently read value type from the config schema, serialize the user value and store it inside the datastore. Similarly, when action is ran, we would de-serialize the value and pass it to the action (similar story for sensors).

To support that, we would need to implement APIs for managing (list, get, set, delete) the dynamic configuration values and also corresponding CLI commands:

st2 config list [--pack=<pack name>]
st2 config get [--decrypt] <pack name> <config item name>
st2 config set [--encrypt] <pack name> <config item name> <config item value>
st2 config delete <pack name> <config item name>

In short, it would be implemented as a thin-wrapper around datastore API which would handle serialization and de-serialization and some other things.

Backward Compatibility

We can make the change fully backward compatible by treating a value as a string if it's not serialized (those would be existing / old values).

[jjm]

I would really like to see this, especially if we can support the following:

Orion_platforms:
  name:
    user: foo
    password: bar
    host: baz
 name2:
    user: foo
    password: bar
    host: baz

Where each entry should follow a schema that does not allow further nesting. This was common in a number of packs (e.g. vsphere, GitHub, Orion) where multiple servers, platforms, repos. etc needed to be accessed. This time of support has been slowly removed in the migrations to config.schema.

[Kami]

Another thing we need to do is support default values for attributes in a nested object - https://github.com/StackStorm/st2contrib/pull/625/files#r88874069