[pending] Documentation for pyDKB.common.json_utils. #169

Evildoor · 2018-09-06T09:09:50Z

Basic documentation for pyDKB.common.json_utils.

Waits for #165

Conflicts: Utils/Dataflow/091_datasetsRucio/datasets_processing.py

mgolosova · 2018-09-06T15:09:18Z

After (or with) every change in docs I would add new version of HTML and PDF documentation; after all, it is the destination of any changes in docstrings ;)

mgolosova · 2018-09-06T15:51:48Z

Utils/Dataflow/pyDKB/common/json_utils.py

-Utils to work with JSON (dict) objects.
+Utils to work with JSON objects.
+
+In context of python, JSON [#]_ objects may be considered as structures


According to my knowledge, JSON is not tied to python in any way - python is just another language which can work with this format. Other languages may not even have things such as 'dictionary' and use different terms to define the same object. Therefore, I believe that saying "you see, JSON file is usually just a big dictionary which may contain other dictionaries, lists, ..." is wrong without indicating that you are talking in terms of python (even considering the fact that this is mostly python project).
Side note - thanks, due to this comment I've found that I forgot to mention some other possible objects such as True/False.

Huh, bold symbols are not that bold when you don`t know they are here...
I meant "shouldn`t Python be capitalized?"

Ha, so that's what this was about! Sure, will do.

mgolosova · 2018-09-06T17:38:26Z

Utils/Dataflow/pyDKB/common/json_utils.py

        STRING key -- dot-separated list of nested keys.
-                      If a key contains dot itself, the key must be put between
-                      quotation marks.


I`d suggest to follow common style in docstrings.

I started to use this notation:

def func(arg1, agr2): """ Do this and that. Detailed description, warnings, notes and ToDo. :param arg1: first argument meaning :type arg1: str :param arg2: second argument meaning :type arg2: int, float, NoneType :return: this or that (None if failed) :rtype: dict, NoneType """

It produces pretty predictable view after building (please see e.g. pyDKB.common.custom_readline() in the compiled documentation -- it is not perfect, yet uses stuff I`m talking about).
If it is OK then let`s follow this style; if not, let`s find (or invent) another and use it everywhere.

Google also says that there`s a keyword :raises, yet I never tried it yet.

I think I am going to add a “howto” to the repo, so that we had a common guide for this work.

Sure thing, I'll take a look at custom_readline().

mgolosova · 2018-09-07T06:22:21Z

Utils/Dataflow/pyDKB/common/json_utils.py

+    **Examples:**
+        Transform STRING ``\'1.2.3'`` into LIST ``[\'1', \'2', \'3']``.
+
+        Transform STRING ``\'1.\"2.3".4'`` into


Talking about examples — I would suggest example in form of Python code, something like:

>>> nestedKeys('1.2.3') ['1', '2', '3']

Having less words it looks more illustrative for me.

Makes sense. However, per our discussion yesterday - tests will be used as examples, so I'll remove examples I've made here.

mgolosova · 2018-09-07T06:28:43Z

Utils/Dataflow/pyDKB/common/json_utils.py

        STRING key -- dot-separated list of nested keys.
-                      If a key contains dot itself, the key must be put between
-                      quotation marks.
+                      If a key contains dot itself, the key must be put


Please do not forget that originally it was written with no concern to Sphinx, reStructuredText and building documentation from this comment. Thus newline in the comment does not mean newline or new paragraph in the built version — check how it looks now. Is it what you expected to get there?

It means a newline in built version when [source] button is clicked.

Normal version is unaffected.

Sorry, didn`t say that I was talking about PDF version.
HTML looks fine -- couldn`t check it from tablet ;)

mgolosova · 2018-09-07T06:33:46Z

Utils/Dataflow/pyDKB/common/json_utils.py

 """


 def valueByKey(json_data, key):
    """ Return value by a chain (list) of nested keys.

-    Parameters:
+    It is common for JSON objects to contain many layers of dictionaries
+    nested in other dictionaries - this function extracts the data from


Single hyphen produces short dash, double — long one. In this case I guess you meant to use the long one, right?

You are right, thanks. Will fix.

Evildoor · 2018-09-12T11:57:00Z

Updated the docs according to comments.

@mgolosova, due to software reasons known to you I cannot make PDF version at the moment. Should I make and upload HTML version or leave this issue to you?

mgolosova · 2018-09-13T07:21:33Z

@Evildoor
I`m working on the issue you`re talking about, sorry it takes a bit long...
I think I will build the docs for this PR if fail to solve the issue soon enough.
Thank you for the changes — I`ll get through them carefully tonight, chacking how the built version looks.

Yet I have a couple of comment already.
It`s better not to refer in the commit logs to what can not be found in the log history. After all, one may be reading it in console ;)
Also, “minor fixes” commit contains three different changes. They are minor, right, but still logically independent. If you fill like joining commits — better to join with the related ones, e.g. with the first one, where the documentation has largely changed.
And, talking about commit organization — the very first one states “addaed documentation for...json_utils”, while what we see in the commit log is:

extended module description;
added detailed description of valueByKey;
changed formatting of parameters description;
added examples.

It is a bit too much for such a laconic log message ;) I see two ways:

make one commit “Update documentation (<module_name>)” (as you have done), but list changes in the log;
make 4 commits for the 4 mentioned changes.

I do remember that the last change (examples) was later reverted, so it is up to you — leave it and revert or simply remove — but the general idea is kind of this.

mgolosova · 2018-09-13T09:05:44Z

@Evildoor
I believe I finally managed to configure TeX on the bamboo server. Please check, and if there are any problems (except that pdflatex claims about fancyvrb.cfg) please let me know.

- Extended module description - Extended valueByKey() description - Reformatted the "parameters" sections - Added examples

It was decided that tests should be used as examples.

The following changes were also made to match the new style: - Rephrased "key" description in both functions because saying "list of keys" about a variable which is explicitly defined as "str" in the next line is confusing. - Removed type highlights from nestedKeys() description.

Evildoor · 2018-09-13T12:34:43Z

@mgolosova
Thank you, the PDF builds now. Also made the changes according to your comments.

mgolosova

Now almost everything looks fine, thank you!

mgolosova · 2018-09-13T14:20:58Z

Utils/Dataflow/pyDKB/common/json_utils.py

@@ -57,6 +57,8 @@ def nestedKeys(key):

    String should contain keys separated by dot. If a key contains
    dot itself, the key must be put between matching quotation marks.
+    Quotation marks inside the keys (not preceding or following a dot)
+    are ignored.


By "ignored" you mean "treated as ordinary symbols", right?
If I read the description without knowing how the function works (or supposed to), for a moment I might think that "ignored" means "removed from the output", like:

>>> nested_keys('ab.c"d.e') ['ab', 'cd', 'e']

mgolosova · 2018-09-13T14:28:36Z

Utils/Dataflow/pyDKB/common/json_utils.py

+    are ignored.
+
+    :param key: nested keys
+    :type key: str


:type key: str, list

If passed key is of type list, it is returned without changes.

mgolosova · 2018-09-13T14:30:22Z

Utils/Dataflow/pyDKB/common/json_utils.py

+    :param json_data: to search in
+    :type json_data: dict
+    :param key: nested keys
+    :type key: str


:type key: str, list

If passed key is of type list, it is treated as "already parsed" nested keys.

Nested keys can be list instead of str in both functions - it is treated as already processed str and used as-is.

Evildoor · 2018-09-14T10:12:35Z

Done.

mgolosova · 2018-09-26T13:06:00Z

Utils/Dataflow/pyDKB/common/json_utils.py

-        STRING key       -- dot-separated list of nested keys
+    It is common for JSON objects to contain many layers of dictionaries
+    nested in other dictionaries -- this function extracts the data from
+    such constructions according to given string or list with keys.


The description does not explain how keys should be passed. If usage of list version can be guessed, string is not clear at all.
What about something like this?

...according to a list of keys. <...> :param keys: sequence of nested keys (dot-separated string or list object)

?

Makes sense, will do.

mgolosova · 2018-09-26T13:06:42Z

Everything seems to be fine; now waiting for the pyDKB fix...

mgolosova added 2 commits September 4, 2018 13:01

Merge branch 'master' into pyDKB-master-merge

eec5452

Conflicts: Utils/Dataflow/091_datasetsRucio/datasets_processing.py

docs: make documentation corresponding to the current branch.

132b720

Evildoor requested a review from mgolosova September 6, 2018 09:09

mgolosova assigned Evildoor Sep 6, 2018

mgolosova changed the title ~~Added documentation for pyDKB.common.json_utils.~~ [pending] Added documentation for pyDKB.common.json_utils. Sep 6, 2018

mgolosova requested changes Sep 7, 2018

View reviewed changes

Evildoor added 5 commits September 13, 2018 15:10

Added documentation for pyDKB.common.json_utils.

6a8120d

- Extended module description - Extended valueByKey() description - Reformatted the "parameters" sections - Added examples

Removed examples.

2a94761

It was decided that tests should be used as examples.

Added clarification regarding quotation marks.

6e2467b

Rebuilt the PDF and HTML versions of documentation

a2ad012

Evildoor force-pushed the docs_json_utils branch from 230963c to a2ad012 Compare September 13, 2018 12:30

mgolosova reviewed Sep 13, 2018

View reviewed changes

Evildoor added 3 commits September 14, 2018 12:56

Added description of using lists in json_utils.

b6fa65b

Nested keys can be list instead of str in both functions - it is treated as already processed str and used as-is.

Clarified the treatment of quotation marks.

ac3c069

Remade the docs.

4b5db36

mgolosova added the Docs label Sep 18, 2018

mgolosova reviewed Sep 26, 2018

View reviewed changes

Clarification for key format in valueByKey().

b6122b8

mgolosova force-pushed the pyDKB-master-merge branch from 132b720 to fb7e780 Compare March 28, 2019 11:31

mgolosova force-pushed the pyDKB-master-merge branch from f415e34 to abd340a Compare June 6, 2019 12:14

Evildoor changed the title ~~[pending] Added documentation for pyDKB.common.json_utils.~~ [pending] Documentation for pyDKB.common.json_utils. Mar 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[pending] Documentation for pyDKB.common.json_utils. #169

[pending] Documentation for pyDKB.common.json_utils. #169

Evildoor commented Sep 6, 2018 •

edited by mgolosova

Loading

mgolosova commented Sep 6, 2018

mgolosova Sep 6, 2018

Evildoor Sep 7, 2018 •

edited

Loading

mgolosova Sep 7, 2018

Evildoor Sep 7, 2018 •

edited

Loading

mgolosova Sep 6, 2018

mgolosova Sep 7, 2018

Evildoor Sep 7, 2018

mgolosova Sep 7, 2018

Evildoor Sep 7, 2018

mgolosova Sep 7, 2018

Evildoor Sep 7, 2018 •

edited

Loading

mgolosova Sep 7, 2018 •

edited

Loading

mgolosova Sep 7, 2018

Evildoor Sep 7, 2018

Evildoor commented Sep 12, 2018

mgolosova commented Sep 13, 2018

mgolosova commented Sep 13, 2018 •

edited

Loading

Evildoor commented Sep 13, 2018

mgolosova left a comment

mgolosova Sep 13, 2018

mgolosova Sep 13, 2018

mgolosova Sep 13, 2018

Evildoor commented Sep 14, 2018

mgolosova Sep 26, 2018

Evildoor Sep 26, 2018

mgolosova commented Sep 26, 2018

[pending] Documentation for pyDKB.common.json_utils. #169

Are you sure you want to change the base?

[pending] Documentation for pyDKB.common.json_utils. #169

Conversation

Evildoor commented Sep 6, 2018 • edited by mgolosova Loading

mgolosova commented Sep 6, 2018

Choose a reason for hiding this comment

Evildoor Sep 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Evildoor Sep 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Evildoor Sep 7, 2018 • edited Loading

Choose a reason for hiding this comment

mgolosova Sep 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Evildoor commented Sep 12, 2018

mgolosova commented Sep 13, 2018

mgolosova commented Sep 13, 2018 • edited Loading

Evildoor commented Sep 13, 2018

mgolosova left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Evildoor commented Sep 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mgolosova commented Sep 26, 2018

Evildoor commented Sep 6, 2018 •

edited by mgolosova

Loading

Evildoor Sep 7, 2018 •

edited

Loading

Evildoor Sep 7, 2018 •

edited

Loading

Evildoor Sep 7, 2018 •

edited

Loading

mgolosova Sep 7, 2018 •

edited

Loading

mgolosova commented Sep 13, 2018 •

edited

Loading