A way to identify schema of a particular category is needed.

[brettviren]

The setup

Most module-level "conf" schema we have developed so far is an instance of the record schema class with a fully-qualified path and name like:

("dunedaq", <pkgname>, <longunreadablemodulename>, "Conf")

The use of "Conf" as the leaf name is not universal, largely accidental and some developers have been creative and so there is a variation of other names.

The problem

A collection of all known module-level "conf" schema needs to be constructed for use by the web UI command/configuration editor (at least).

Ideally, this collection can be formed in some automated way so as new module-level "conf" schema get invented, the list can grow without human effort/error. Ideally, also, a "schematic" solution is found so that this layer of structure is captured with the same rigor as the rest of the schema.

Possible solutions

There are a number of possible solutions including giving up on ideals and manually curating the list. This will be continually labor intensive and error prone.

Not giving up, one portion of the solution is to require module developers not use a bare record() to define their module-level conf schema but introduce and require a new Jsonnet function, say modconf().

This modconf() function would mostly defer to record() and may even take the same call signature. It would emit the record() output but augmented with something "special" that "marks" the type as being a module-level conf sub-object.

I expect this "meta semantic" marking will be needed in other cases so we should expect DUNE DAQ to ultimately define more of these specific schema constructor functions. They can add value as vehicles to enforce other rules (such as no use of any in some cases, which is an issue in the making).

It is less clear the best way to "mark" the resulting modconf() output type structure.

A naming convention could be used. This could reasonbly apply to either the resulting type name (ie, all named "ModConf") or the parent path name (ie, dunedaq.<pkgname>.<unredablylonglowercasemodulename>.modconf.WhateverCreativeModConfName)

Both conventions require changes in our schema that will reflect into C++ structs and moo.otypes Python classes. While they themselves will be automatically updated, hand-written code will need manual changes. Sweeping search-and-replace will catch much and the C++ compiler will catch any missed. Missed Python changes will have to rely on runtime failures to fully root out.

A second approach is to augment the record type structure. For example, we may add an attribute hint which may hold an array of strings. Much of moo will need updating to then ignore this new hint.

But this change would allow us to write a DUNE DAQ specify Python script that iterates over all known schema to find module-level conf schema by checking:

if "modconf" in type.hint:
    modconfs.append(type)

Such a filter could be provided generally by moo itself.

[brettviren]

We will also need to distinguish other "classes" of schema.

For example, to configure the web UI to know how to construct a "conf" command object we must tell it a schema that contains the type refs for all known "module-conf" schema. This requires generating the form: part of jsonform input in a different manner than we might generate for other schema. The generator of jsonform needs some information to "switch" on.

[brettviren]

Pondering more, modifying the meta-schema to add new interpretation of the schema is not a sustainable conceptual scaling. Tomorrow when we invent some new axis of classification, we would need to further muddy the moo meta schema to accommodate.

Just like the lowly "number" does not know it is a field in a "record" so a "record" should not know it happens to be a thing called a "modconf".

We should provide higher level semantics with yet higher level schema composing lower-level schema. This brings me around back to preferring the hand-crafted solution.