Error in serializing datetime #35

foxpluto · 2023-03-03T08:15:01Z

I have a quite long openapi file with a lot of date-time like this one:

      - description: Start time stamp of the returned data interval
        example: 2022-08-17T18:00:00Z
        in: query
        name: fromInstant
        required: false
        schema:
          format: date-time
          type: string

but I have this error:

ERROR    -  Error reading page 'openapi/openapi.md': Object of type datetime is not JSON serializable
Traceback (most recent call last):
  File "/opt/homebrew/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/__main__.py", line 234, in serve_command
    serve.serve(dev_addr=dev_addr, livereload=livereload, watch=watch, **kwargs)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 83, in serve
    builder(config)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 76, in builder
    build(config, live_server=live_server, dirty=dirty)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/build.py", line 308, in build
    _populate_page(file.page, config, files, dirty)
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/commands/build.py", line 177, in _populate_page
    page.markdown = config.plugins.run_event(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/mkdocs/plugins.py", line 520, in run_event
    result = method(item, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/neoteroi/mkdocs/oad/__init__.py", line 44, in on_page_markdown
    return self.rx.sub(self._replacer, markdown)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/neoteroi/mkdocs/oad/__init__.py", line 34, in _replacer
    return handler.write()
           ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/v3/__init__.py", line 417, in write
    return self._writer.write(
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/jinja.py", line 109, in write
    return template.render(data, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/homebrew/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/v3/views_mkdocs/layout.html", line 14, in top-level template code
    {% include "partial/path-items.html" %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/v3/views_mkdocs/partial/path-items.html", line 29, in top-level template code
    {%- include "partial/request-responses.html" %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/v3/views_mkdocs/partial/request-responses.html", line 25, in top-level template code
    {% include "partial/content-examples.html" %}
^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/v3/views_mkdocs/partial/content-examples.html", line 4, in top-level template code
    {{handler.write_content_example(example, content_type) | indent(4) | safe}}
^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/v3/__init__.py", line 494, in write_content_example
    return example_handler.write(example.value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/openapidocs/mk/contents.py", line 33, in write
    return json.dumps(value, ensure_ascii=False, indent=4)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 202, in encode
    chunks = list(chunks)
             ^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 432, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict
    yield from chunks
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 439, in _iterencode
    o = _default(o)
        ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type datetime is not JSON serializable

Is this a bug?

Regards,
S.

The text was updated successfully, but these errors were encountered:

RobertoPrevato · 2023-03-03T08:50:34Z

Hi @foxpluto
Thank You for reporting this issue, it is definitely a bug. It's happening because:

YAML parsers parse dates automatically into instances of datetime (unlike JSON parsing standard)
I am using the built-in json.dumps method to serialize, which is in my opinion silly, throwing exception for common data types like UUID and datetime and other built-in types like dataclass, etc.

Good news: I even created in another library a method to avoid this kind of scenario, so to fix the error I would only need to replace the built-in json.dumps with essentials.json.dumps.

It uses a more user-friendly JSONEncoder class that handles those types.

foxpluto · 2023-03-03T10:57:20Z

Wonderful !!!
I'll wait the fix to test it.

Thanks,
S.

foxpluto · 2023-03-18T12:17:38Z

no ETA for this fix?

Sorry to bother...

RobertoPrevato · 2023-03-19T07:17:28Z

Hi @foxpluto
Sorry for taking longer than I promised at first, in the last weeks I was focused again on my web framework.

I fixed the bug and published a new version of the library where the bug happens 1.0.6. You can upgrade the essentials-openapi dependency and the error has been resolved.

However, while writing tests and the fix, I also realized that maintaining the exact format of the datetime as in the examples is not obvious because datetimes are parsed automatically by YAML, then serialized using a certain format for JSON (the built-in json.dumps throws exception probably exactly for this reason, that the desired serialization format for dates and times is not obvious).

To maintain the exact format you set for your examples, like 2022-08-17T18:00:00Z, you have two options:

change your YAML to have example datetimes defined as strings and avoid automatic parsing (quote the values) - this way they are kept as strings and displayed in JSON exactly the same. Note that this would have worked already before I applied the fix.

      - description: Start time stamp of the returned data interval
-       example: 2022-08-17T18:00:00Z
+       example: '2022-08-17T18:00:00Z'
        in: query
        name: fromInstant
        required: false
        schema:
          format: date-time
          type: string

keep your YAML fragments as they are, and set this environmental variable with the desired datetime format:
OPENAPI_DATETIME_FORMAT="{YOUR_DESIRED_FORMAT_FOR_DATETIME_strftime}". This is something I added to 1.0.6.

foxpluto · 2023-03-25T15:36:46Z

Wonderful !!!

At least I have my API documentation in order on my site.
I have used your second option with:

OPENAPI_DATETIME_FORMAT="%Y%m%dT%H%M%S.%fZ"

And works perfectly.

Thanks

RobertoPrevato · 2023-03-25T15:41:56Z

You're welcome 😄
I leave this issue open to recall about documenting this feature, when I get the time. Now I'm still dedicating time to my web framework.

RobertoPrevato changed the title ~~Error in parsing datetime~~ Error in serializing datetime Mar 19, 2023

RobertoPrevato closed this as completed Mar 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error in serializing datetime #35

Error in serializing datetime #35

foxpluto commented Mar 3, 2023

RobertoPrevato commented Mar 3, 2023 •

edited

Loading

foxpluto commented Mar 3, 2023

foxpluto commented Mar 18, 2023

RobertoPrevato commented Mar 19, 2023 •

edited

Loading

foxpluto commented Mar 25, 2023

RobertoPrevato commented Mar 25, 2023

Error in serializing datetime #35

Error in serializing datetime #35

Comments

foxpluto commented Mar 3, 2023

RobertoPrevato commented Mar 3, 2023 • edited Loading

foxpluto commented Mar 3, 2023

foxpluto commented Mar 18, 2023

RobertoPrevato commented Mar 19, 2023 • edited Loading

foxpluto commented Mar 25, 2023

RobertoPrevato commented Mar 25, 2023

RobertoPrevato commented Mar 3, 2023 •

edited

Loading

RobertoPrevato commented Mar 19, 2023 •

edited

Loading