Markdown not rendering due to missing `long_description_content_type` #395

RealPolitiX · 2020-09-16T20:14:53Z

Hi there, I have a markdown rendering issue on PyPI, but this doesn't seem to fit with any previous case.

My computer runs Windows 10.

When building and twin-checking the repo hdfio (https://github.com/RealPolitiX/hdfio), the following error appeared.

Checking dist\hdfio-0.2.1-py2.py3-none-any.whl: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.
Checking dist\hdfio-0.2.1.tar.gz: FAILED
  `long_description` has syntax errors in markup and would not be rendered on PyPI.
    line 39: Warning: Inline literal start-string without end-string.
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.

The upload to PyPI (https://pypi.org/project/hdfio/) doesn't render the text in README.md (version 0.2.0 has the same issue).

python --version = 3.6.4
twine --version = 3.2.0

twine was installed using pip.

To reproduce the error message above, use the following

python setup.py sdist bdist_wheel --universal
twine check dist/*

jamadden · 2020-09-16T22:06:59Z

Did you specify the type of syntax you were using? markdown is not the default.

RealPolitiX · 2020-09-16T22:52:04Z

Hi, @jamadden, I did specify in the setup.py that

long_description_content_type = 'text/markdown'

di · 2020-09-16T23:01:46Z

The issue is that you have Windows-style newlines in your README.md:

$ tar -xvvf hdfio-0.2.0.tar.gz
x drwxrwxrwx  0 0      0           0 Sep 16 07:04 hdfio-0.2.0/
x -rw-rw-rw-  0 0      0        1928 Sep 15 21:23 hdfio-0.2.0/.gitignore
x drwxrwxrwx  0 0      0           0 Sep 16 07:04 hdfio-0.2.0/hdfio/
x -rw-rw-rw-  0 0      0        3879 Sep 16 05:41 hdfio-0.2.0/hdfio/dict_io.py
x -rw-rw-rw-  0 0      0          92 Sep 16 05:41 hdfio-0.2.0/hdfio/json_io.py
x -rw-rw-rw-  0 0      0         196 Sep 16 05:41 hdfio-0.2.0/hdfio/__init__.py
x drwxrwxrwx  0 0      0           0 Sep 16 07:04 hdfio-0.2.0/hdfio.egg-info/
x -rw-rw-rw-  0 0      0           1 Sep 16 07:04 hdfio-0.2.0/hdfio.egg-info/dependency_links.txt
x -rw-rw-rw-  0 0      0        1341 Sep 16 07:04 hdfio-0.2.0/hdfio.egg-info/PKG-INFO
x -rw-rw-rw-  0 0      0          24 Sep 16 07:04 hdfio-0.2.0/hdfio.egg-info/requires.txt
x -rw-rw-rw-  0 0      0         272 Sep 16 07:04 hdfio-0.2.0/hdfio.egg-info/SOURCES.txt
x -rw-rw-rw-  0 0      0           6 Sep 16 07:04 hdfio-0.2.0/hdfio.egg-info/top_level.txt
x -rw-rw-rw-  0 0      0        1093 Sep 15 21:23 hdfio-0.2.0/LICENSE
x -rw-rw-rw-  0 0      0          43 Jun 12 11:21 hdfio-0.2.0/MANIFEST.in
x -rw-rw-rw-  0 0      0        1341 Sep 16 07:04 hdfio-0.2.0/PKG-INFO
x -rw-rw-rw-  0 0      0         714 Sep 16 06:35 hdfio-0.2.0/README.md
x -rw-rw-rw-  0 0      0          35 Sep 15 22:05 hdfio-0.2.0/requirements.txt
x -rw-rw-rw-  0 0      0         171 Sep 16 07:04 hdfio-0.2.0/setup.cfg
x -rw-rw-rw-  0 0      0        1412 Sep 16 06:21 hdfio-0.2.0/setup.py

$ cat -v hdfio-0.2.0/README.md
# hdfio^M
All the useful python I/O utility functions for HDF5 files^M
^M
## Rationale^M
Writing this software package aggregator is motivated by the multitude of read/write functions currently existing in the python open source community. The package combines existing mature hdf(5) I/O source code and a few similar isolated functions into a single simple and user-friendly platform to save the labor of writing newer functions of similar capabilities.^M
^M
## Resources^M
The major resources gathered here include the following packages,^M
- [h5py](https://github.com/h5py/h5py)^M
- [silx](https://github.com/silx-kit/silx)^M
- [deepdish](https://github.com/uchicago-cs/deepdish)^M
- [h5io](https://github.com/h5io/h5io)^M

This is causing wheel to write truncated metadata:

$ unzip hdfio-0.2.0-py2.py3-none-any.whl
Archive:  hdfio-0.2.0-py2.py3-none-any.whl
  inflating: hdfio/__init__.py
  inflating: hdfio/dict_io.py
  inflating: hdfio/json_io.py
  inflating: hdfio-0.2.0.dist-info/LICENSE
  inflating: hdfio-0.2.0.dist-info/METADATA
  inflating: hdfio-0.2.0.dist-info/WHEEL
  inflating: hdfio-0.2.0.dist-info/top_level.txt
  inflating: hdfio-0.2.0.dist-info/RECORD

$ cat -v hdfio-0.2.0.dist-info/METADATA
Metadata-Version: 2.1
Name: hdfio
Version: 0.2.0
Summary: All the useful python I/O utility functions for HDF5 files
Home-page: https://github.com/RealPolitiX/hdfio
Author: ['R. Patrick Xian', 'others']
Author-email:
License: MIT
Download-URL: https://github.com/RealPolitiX/hdfio/tarball/0.2.0
Requires-Dist: h5py
Requires-Dist: silx
Requires-Dist: deepdish
Requires-Dist: h5io

# hdfio

Can you tell us about how you built these distributions?

RealPolitiX · 2020-09-16T23:15:24Z

I see, unfortunately I don't have access to a linux computer atm. It's built using simply

python setup.py sdist bdist_wheel --universal

Is it possible to fix without using rst-formatted README (which I don't mind to switch to if needed to)?

wheel version = 0.35.1

di · 2020-09-16T23:21:02Z

I'm a little confused actually because the version in your repo does not have windows-style line endings:

$ wget https://raw.githubusercontent.com/RealPolitiX/hdfio/master/README.md
--2020-09-16 18:18:20--  https://raw.githubusercontent.com/RealPolitiX/hdfio/master/README.md
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1190 (1.2K) [text/plain]
Saving to: ‘README.md’

README.md           100%[===================>]   1.16K  --.-KB/s    in 0s

2020-09-16 18:18:20 (25.2 MB/s) - ‘README.md’ saved [1190/1190]

$ cat -v README.md
# hdfio
![GitHub license](https://img.shields.io/github/license/RealPolitiX/hdfio)

All the useful python I/O utility functions for HDF5 files

## Rationale
Writing this software package aggregator is motivated by the multitude of read/write functions currently existing in the python open source community. The package combines existing mature hdf(5) I/O source code and a few similar isolated functions into a single simple and user-friendly platform to save the labor of writing newer functions of similar capabilities. It's also the basis for building more sophisticate I/O operations.

## Resources
The major resources gathered here include the following packages that covers generic I/O and interconversions between HDF and other key-value-paired file formats, such as **dictionary**, **json**, **mat**.

- [h5py](https://github.com/h5py/h5py)
- [silx](https://github.com/silx-kit/silx)
- [deepdish](https://github.com/uchicago-cs/deepdish)
- [h5io](https://github.com/h5io/h5io)

## Installation

1. Install and upgrade to the latest updates

```
pip install --upgrade git+https://github.com/RealPolitiX/hdfio.git
```
2. Install the latest build from PyPI

```
pip install hdfio
```

Did something change? Rebuilding with this new README might fix it.

RealPolitiX · 2020-09-16T23:42:42Z

Hi @di, sorry, I might have confused you a bit. The error message is from the latest github repo (at version 0.2.1 and commit c5e4fe3600ed965b3ecd5c8460a736046027e1ff). What I meant is that the already uploaded version on PyPI (0.2.0) had similar problem, but since I needed it to be posted there for other developments, so I didn't correct this rendering error.

Your version of the README still has similar issue after building.

Checking dist\hdfio-0.2.1-py2.py3-none-any.whl: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.
Checking dist\hdfio-0.2.1.tar.gz: FAILED
  `long_description` has syntax errors in markup and would not be rendered on PyPI.
    line 54: Warning: Block quote ends without a blank line; unexpected unindent.
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.

I needed to add an additional enter/newline below the last code block to clear the block quote warning, so now it's only,

Checking dist\hdfio-0.2.1-py2.py3-none-any.whl: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.
Checking dist\hdfio-0.2.1.tar.gz: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.

It's some progress, but the README renderer is still defaulted to text/x-rst... I attach my
PKG-INFO.txt, in case it's helpful.

di · 2020-09-17T15:13:29Z

Can you tell us what version of setuptools, wheel and twine you have installed?

RealPolitiX · 2020-09-17T23:58:52Z

I have the following setup.

setuptools==47.1.1
setuptools-markdown==0.2
setuptools-scm==3.5.0
wheel==0.35.1
twine==3.2.0

sigmavirus24 · 2020-09-18T11:54:31Z

So the right thing to be looking at, as @di pointed out is the METADATA file. That's what we look for to find the information inside a wheel. That provides the insight we need to know about which is that it's using Windows-style line endings for some reason.

The line is not in the METADATA file and so twine can't use it. @di, can this issue be moved to a more appropriate repository since it's not a bug with Twine?

RealPolitiX · 2020-09-18T12:17:52Z

So basically windows-style line ending will trigger the long_description_content_type='text/markdown' specification not being read by twine?

sigmavirus24 · 2020-09-18T16:48:29Z

So basically windows-style line ending will trigger the long_description_content_type='text/markdown' specification not being read by twine?

No. It means your METADATA file will be incorrectly generated and that will prevent Description-Content-Type from being found because it isn't there (as Dustin showed in an earlier comment). Garbage in, garbage out.

RealPolitiX · 2020-09-18T20:43:12Z

I see the METADATA file now zipped in the whl package. So it should have the same content as the PKG-INFO file?

di · 2020-09-18T21:34:04Z

This is pypa/setuptools#1126

RealPolitiX · 2020-09-19T12:24:20Z

So I removed the encoding='utf-8' argument when reading the markdown file in setup.py. This made most thing work! See https://test.pypi.org/project/hdfio/

Except now that the fenced code block (with triple backticks) in README.md are not parsed and converted correctly, how would you solve that?

RealPolitiX · 2020-09-19T14:33:48Z

I realized that one can use the alternative markup of a code block to let the markdown parser read it correctly, like the following,
<pre><code class="console"> some code </code></pre>
It's rendered correctly now on PyPI (https://pypi.org/project/hdfio/)

di transferred this issue from pypa/twine Sep 18, 2020

RealPolitiX closed this as completed Sep 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Markdown not rendering due to missing `long_description_content_type` #395

Markdown not rendering due to missing `long_description_content_type` #395

RealPolitiX commented Sep 16, 2020

jamadden commented Sep 16, 2020

RealPolitiX commented Sep 16, 2020

di commented Sep 16, 2020

RealPolitiX commented Sep 16, 2020 •

edited

Loading

di commented Sep 16, 2020

RealPolitiX commented Sep 16, 2020 •

edited

Loading

di commented Sep 17, 2020

RealPolitiX commented Sep 17, 2020

sigmavirus24 commented Sep 18, 2020

RealPolitiX commented Sep 18, 2020

sigmavirus24 commented Sep 18, 2020

RealPolitiX commented Sep 18, 2020 •

edited

Loading

di commented Sep 18, 2020

RealPolitiX commented Sep 19, 2020 •

edited

Loading

RealPolitiX commented Sep 19, 2020

Markdown not rendering due to missing long_description_content_type #395

Markdown not rendering due to missing long_description_content_type #395

Comments

RealPolitiX commented Sep 16, 2020

jamadden commented Sep 16, 2020

RealPolitiX commented Sep 16, 2020

di commented Sep 16, 2020

RealPolitiX commented Sep 16, 2020 • edited Loading

di commented Sep 16, 2020

RealPolitiX commented Sep 16, 2020 • edited Loading

di commented Sep 17, 2020

RealPolitiX commented Sep 17, 2020

sigmavirus24 commented Sep 18, 2020

RealPolitiX commented Sep 18, 2020

sigmavirus24 commented Sep 18, 2020

RealPolitiX commented Sep 18, 2020 • edited Loading

di commented Sep 18, 2020

RealPolitiX commented Sep 19, 2020 • edited Loading

RealPolitiX commented Sep 19, 2020

Markdown not rendering due to missing `long_description_content_type` #395

Markdown not rendering due to missing `long_description_content_type` #395

RealPolitiX commented Sep 16, 2020 •

edited

Loading

RealPolitiX commented Sep 16, 2020 •

edited

Loading

RealPolitiX commented Sep 18, 2020 •

edited

Loading

RealPolitiX commented Sep 19, 2020 •

edited

Loading