[FEATURE REQUEST] Generate OCClasseSettings documentation automatically #755

mneuwert · 2020-07-21T09:26:21Z

UPDATE: The part regarding the AppConfig spec file has been split out and moved to #842.

Is your feature request related to a problem? Please describe.
In ownCloud app we have a comprehensive set of settings which can be used for branding, MDM, configuration during development. Documentation as well as AppConfig spec files are maintained manually. When new setting is added in the code, I think it is not really ensured that all other artefacts are updated accordingly.

Describe the solution you'd like
Would be nice to be able to extract OCClassSetting information from the code for example using Apple's SourceKit framework. There is also interesting framework / command line tool which abstracts SourceKit called SourceKitten - https://github.com/jpsim/SourceKitten which e.g. can create an XML file describing structure from an Objective-C source file.

So, we could create a small tool run by a custom build phase in Xcode which could generate the artefacts we need.

May be we would need to augment the definition of our settings by adding special comments describing the setting and e.g. also adding tags: mandatory, nice-to-have, supported, development etc.

TASKS

PR

App
Library (if needed)

The text was updated successfully, but these errors were encountered:

michaelstingl · 2020-10-17T19:43:53Z

For every config key/parameter, we'd need some metadata:

Configuration Key
Type (Boolean, Integer, String, String Array, …)
Description
Possible values
Default value
Category
Status (Recommended, Approved for production, Debugging-only…)

@felix-schwarz @mneuwert would it be feasible to make every parameter like an "array" or JSON like structure? It hould already contain the information for the AppConfig, maybe some more metadata to render in our docs. How could the syntax look like? (for example for the passcode.enforced parameter)

ios-app/docs/modules/ROOT/pages/ios_mdm.adoc

Lines 177 to 191 in 0808ad9

    
           ==== Passcode Enforcement 
        
           If your organization policies require users to use a passcode as an additional security barrier for managed apps, the below setting will allow to enforce this requirement. 
        
           [cols=4*,options=header] 
        
           |=== 
        
           |Key 
        
           |Type 
        
           |Default 
        
           |Description 
        
           |passcode.enforced 
        
           |Boolean 
        
           |`false` 
        
           |If set to true, the app will require the user to set up a passcode upon a first launch. 
        
           |===

Then it should be easier to parse nice docs and XML files?

"Category" could be used for the grouping parameters of similar topic.

Only config parameters with "Recommended" status should be included in an XML file that gets submitted to EMM vendors…

michaelstingl · 2020-11-02T09:53:24Z

MDM options can be autogenerated as JSON now:
https://github.com/owncloud/ios-sdk/blob/feature/settings-metadata/doc/CONFIGURATION.json
(will replaced by bigger JSON in repo ios-app later)

@dschmidt will help with gomplate to create nice AsciiDoc tables for docs, similar to what we have today:
https://github.com/owncloud/ios-app/blob/feature/mdm_enhancements/docs/modules/ROOT/pages/ios_mdm.adoc

I was thinking, individual AsciiDoc tables could be generated per category ("category" : "Logging",), then embedded in the big AsciiDoc page…

Entries should be sorted by status ("status" : "supported",):

"status" : "supported",
"status" : "advanced",
"status" : "debugOnly", (maybe hidden)

Table columns needed:

Key
Type
Default value
Description
Status

gomplate could run on local developers machine, and updated AsciiDoc tables will be part of commit/PR.

dschmidt · 2020-11-02T20:59:59Z

I have prepared the gomplate template, but the format of CONFIGURATION.json is incompatible, because gomplate does not like a single \ for escaping forward slashes. Replacing all occurrences of \/ with / makes it work.

I noticed some configuration items are present multiple times, e.g. vacuum-sync-anchor-ttl. IMHO that should be fixed in CONFIGURATION.json.

The template looks as follows:


{{- $config := .config }}
{{- $categories := .config | jsonpath `$[*].category` | uniq | sort }}
{{- range $index, $category := $categories }}
tag::{{$category | strings.ToLower }}[]
[cols=4*,options=header] 
|=== 
|Key 
|Type 
|Default 
|Description 

{{$options := $config | jsonpath (print "$[?(@.category=='" $category "')]") }}
{{- if eq (printf "%T" $options) "map[string]interface {}"}}
{{- $option := $options }}
|{{- $option.key}}
|{{- $option.type}} 
|{{- if has $option "defaultValue"}}`{{$option.defaultValue}}`{{end}} 
|{{- if has $option "description"}}{{$option.description}}{{end}}
{{- else}}
{{- range $statusIndex, $status := slice "supported" "advanced" "debugOnly"}}
{{- range $optionIndex, $option := $options | jsonpath (print "$[?(@.status=='" $status "')]")}}
|{{- $option.key}}
|{{- $option.type}} 
|{{- if has $option "defaultValue"}}`{{$option.defaultValue}}`{{end}} 
|{{- if has $option "description"}}{{$option.description}}{{end}}
{{end}}
{{- end}}
{{- end}}
|=== 
end::{{$category | strings.ToLower }}[]

{{end}}

The actual .adoc file can be generated via

../ios-app/docs/modules/ROOT/pages $ gomplate -f ios_mdm_tables.adoc.tmpl --context config=../../../../CONFIGURATION.json -o ios_mdm_tables.adoc

... and afterwards tables can be referenced in the main .adoc file via:

== Authentication
include::ios_mdm_tables.adoc[tag=authentication] 

== Connection
include::ios_mdm_tables.adoc[tag=connection]

== Endpoints
include::ios_mdm_tables.adoc[tag=endpoints]

== Logging
include::ios_mdm_tables.adoc[tag=logging]

== OAuth2
include::ios_mdm_tables.adoc[tag=oauth2]

== OIDC
include::ios_mdm_tables.adoc[tag=oidc]

== Policies
include::ios_mdm_tables.adoc[tag=policies]

== Privacy
include::ios_mdm_tables.adoc[tag=privacy]

== Security
include::ios_mdm_tables.adoc[tag=security]

I can prepare a PR for adding it to the repo, as soon as a compatible CONFIGURATION.json is added to this repo.
A prepared place to hook gomplate into would be appreciated.

P.S.: this needs gomplate >= 3.8.0, otherwise it won't work.

See #755 (comment)

michaelstingl · 2020-11-09T10:01:37Z

@dschmidt very cool! 💪

I was able to generate a table from the new complete JSON:
https://github.com/owncloud/ios-app/blob/feature/settings-metadata/docs/modules/ROOT/pages/ios_mdm_tables.adoc

Please add 2 more columns:

Possible values
Status

felix-schwarz · 2020-11-09T15:28:02Z

@dschmidt Thanks, that's awesome already!

I have prepared the gomplate template, but the format of CONFIGURATION.json is incompatible, because gomplate does not like a single \ for escaping forward slashes. Replacing all occurrences of \/ with / makes it work.

I have changed the generator code to avoid these. The CONFIGURATION.json no longer escapes the slashes.

I noticed some configuration items are present multiple times, e.g. vacuum-sync-anchor-ttl. IMHO that should be fixed in CONFIGURATION.json.

Good catch. I fixed that as well.

I can prepare a PR for adding it to the repo, as soon as a compatible CONFIGURATION.json is added to this repo.
A prepared place to hook gomplate into would be appreciated.

I'll put together a bash script that generates the JSON documentation, to which gomplate could also be added then.

Regarding possible values, I also made a change to the JSON format that makes it easier to convert. Previously, possible values could come in either of these formats:

    "possibleValues" : {
      "-1" : "verbose",
      "0" : "debug",
      "1" : "info",
      "2" : "warning",
      "3" : "error",
      "4" : "off"
    }

… or more verbose:

    "possibleValues" : [
      {
        "description" : "debug",
        "value" : 0
      },
      {
        "description" : "error",
        "value" : 3
      },
      {
        "description" : "off",
        "value" : 4
      },
      {
        "description" : "warning",
        "value" : 2
      },
      {
        "description" : "verbose",
        "value" : -1
      },
      {
        "description" : "info",
        "value" : 1
      }
    ],

The generated JSON now always uses the latter, verbose format.

Besides normalizing possibleValues to a single format, it also preserves the type more closely (for the more compact format, the values always needed to be converted to strings).

felix-schwarz · 2020-11-09T16:02:50Z

@dschmidt I put together a script here now: https://github.com/owncloud/ios-app/blob/feature/settings-metadata/tools/GenerateDocs/generate_docs.sh

And also put your template here: https://github.com/owncloud/ios-app/blob/feature/settings-metadata/tools/GenerateDocs/templates/ios_mdm_tables.adoc.tmpl

When running the script from its folder, it runs gomplate with

gomplate -f templates/ios_mdm_tables.adoc.tmpl --context config=../../doc/CONFIGURATION.json -o ../../docs/modules/ROOT/pages/ios_mdm_tables.adoc

which partially produces the documentation, then fails with this error:

16:55:52 FTL  error="template: templates/ios_mdm_tables.adoc.tmpl:22:12: executing \"templates/ios_mdm_tables.adoc.tmpl\" at <$option.key>: can't evaluate field key in type interface {}"

dschmidt · 2020-11-10T10:38:13Z

I took the liberty to push to your branch:

d266176#diff-d5f4c9ac6c58d3c3c6c0bc1e7e11fd4a3e50d8faba23e0b7d505bd8dbbbaf067
this is the actual fix. The problem was that now (maybe after your fixes) there were categories with only one item of a specific status, that made jsonpath return a single item instead of a list (:sigh:) which wreaked havoc ...
d6d776b#diff-d5f4c9ac6c58d3c3c6c0bc1e7e11fd4a3e50d8faba23e0b7d505bd8dbbbaf067
reduces the duplication for single items. I had tried to use the very same approach in my initial implementation but maybe it was too late on that day or something ... but now it works, yay.
cad60c0#diff-d5f4c9ac6c58d3c3c6c0bc1e7e11fd4a3e50d8faba23e0b7d505bd8dbbbaf067
Added status column

How should possible values be rendered in the table?

michaelstingl · 2020-11-10T11:06:53Z

How should possible values be rendered in the table?

For me, just listing them inline would do the job…
"-1" : "verbose", "0" : "debug", "1" : "info", "2" : "warning", "3" : "error", "4" : "off"

@felix-schwarz does this make sense for you?

felix-schwarz · 2020-11-13T14:36:35Z

@dschmidt Thanks!

@michaelstingl Personally, I'd prefer a table with possible values in the description column below the description - provided that's possible:

`log-level`

integer

Controls how much details are logged.

Possible values

Value	Description
-1	Verbose
0	Debug
1	Info
2	Warning
3	Error
4	Off

We could have more than one word as description in the future, in which case that formatting would provide a better / easier overview.

michaelstingl · 2020-11-13T15:12:16Z

I'd prefer a table with possible values in the description column below the description

Fine for me. @dschmidt some refinement next week?

dschmidt · 2020-11-19T16:14:54Z

6551b2b
->
https://github.com/owncloud/ios-app/blob/6551b2bc241755a7e56c16b3ef8d8251bab9145f/docs/modules/ROOT/pages/ios_mdm_tables.adoc

dschmidt · 2020-11-19T22:19:29Z

Just noticed I accidentally commited commented out code. Added another commit to revert that. Feel free to fix up the original commit before merging, @felix-schwarz.

felix-schwarz · 2020-11-20T08:48:52Z

Thanks for the updates @dschmidt, that's looking really good!

I've made one little change by putting the possible values in code quotes and re-created the PR as #838 because somehow #831 no longer updated with the latest commits.

Result here now:
https://github.com/owncloud/ios-app/blob/feature/settings-metadata/docs/modules/ROOT/pages/ios_mdm_tables.adoc

felix-schwarz · 2020-11-20T09:06:32Z

I have also just updated the app-side JSON file generator to fix a bug that caused possible values to not be sorted.

michaelstingl added the feature:MDM all kind of MDM / EMM related features label Jul 21, 2020

michaelstingl added this to the 11.5.0-Next milestone Aug 31, 2020

michaelstingl mentioned this issue Oct 29, 2020

Add/document MDM option to enforce passcode #532

Closed

2 tasks

michaelstingl added a commit that referenced this issue Nov 8, 2020

Table-generation with gomplate for docs

3982f76

See #755 (comment)

felix-schwarz changed the title ~~[FEATURE REQUEST] Generate OCClasseSettings documentation and AppConfig.xml automatically~~ [FEATURE REQUEST] Generate OCClasseSettings documentation automatically Nov 23, 2020

felix-schwarz mentioned this issue Nov 23, 2020

[FEATURE REQUEST] Generate AppConfig spec file from OCClassSettings metadata automatically #842

Open

9 tasks

felix-schwarz linked a pull request Nov 30, 2020 that will close this issue

[feature/settings-metadata] Class Settings Metadata support #838

Merged

3 tasks

michaelstingl closed this as completed Jan 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE REQUEST] Generate OCClasseSettings documentation automatically #755

[FEATURE REQUEST] Generate OCClasseSettings documentation automatically #755

mneuwert commented Jul 21, 2020 •

edited by felix-schwarz

Loading

michaelstingl commented Oct 17, 2020 •

edited

Loading

michaelstingl commented Nov 2, 2020 •

edited

Loading

dschmidt commented Nov 2, 2020 •

edited

Loading

michaelstingl commented Nov 9, 2020

felix-schwarz commented Nov 9, 2020 •

edited

Loading

felix-schwarz commented Nov 9, 2020 •

edited

Loading

dschmidt commented Nov 10, 2020

michaelstingl commented Nov 10, 2020

felix-schwarz commented Nov 13, 2020 •

edited

Loading

Possible values

michaelstingl commented Nov 13, 2020

dschmidt commented Nov 19, 2020

dschmidt commented Nov 19, 2020 •

edited

Loading

felix-schwarz commented Nov 20, 2020

felix-schwarz commented Nov 20, 2020

[FEATURE REQUEST] Generate OCClasseSettings documentation automatically #755

[FEATURE REQUEST] Generate OCClasseSettings documentation automatically #755

Comments

mneuwert commented Jul 21, 2020 • edited by felix-schwarz Loading

TASKS

PR

michaelstingl commented Oct 17, 2020 • edited Loading

michaelstingl commented Nov 2, 2020 • edited Loading

dschmidt commented Nov 2, 2020 • edited Loading

michaelstingl commented Nov 9, 2020

felix-schwarz commented Nov 9, 2020 • edited Loading

felix-schwarz commented Nov 9, 2020 • edited Loading

dschmidt commented Nov 10, 2020

michaelstingl commented Nov 10, 2020

felix-schwarz commented Nov 13, 2020 • edited Loading

Possible values

michaelstingl commented Nov 13, 2020

dschmidt commented Nov 19, 2020

dschmidt commented Nov 19, 2020 • edited Loading

felix-schwarz commented Nov 20, 2020

felix-schwarz commented Nov 20, 2020

mneuwert commented Jul 21, 2020 •

edited by felix-schwarz

Loading

michaelstingl commented Oct 17, 2020 •

edited

Loading

michaelstingl commented Nov 2, 2020 •

edited

Loading

dschmidt commented Nov 2, 2020 •

edited

Loading

felix-schwarz commented Nov 9, 2020 •

edited

Loading

felix-schwarz commented Nov 9, 2020 •

edited

Loading

felix-schwarz commented Nov 13, 2020 •

edited

Loading

dschmidt commented Nov 19, 2020 •

edited

Loading