Fail to show many English characters in Chinese Wikipedia

[ghost]

@wdscxsj commented on Sep 13, 2020, 9:11 AM UTC:

• Kiwix version: 2.0.4
• Affected local file: wikipedia_zh_all_maxi_2020-07.zim
• OS: Windows 10 Enterprise 2004 Simplified Chinese Edition

On my machine, Kiwix often fails to show English characters in the Chinese Wikipedia zim.

For example, this is the display of the online item "纽约" (New York) in Chrome:

And this is the same item displayed in Kiwix:

Kiwix doesn't provide a "devtools" to examine the zim data, but it's unlikely that the English text is missing in the zim. Judging from the online page, text snippets with a lang attribute, like <span lang="en">City of New York</span> and <span lang="la">Novum Eboracum</span>, fail to render properly. "Plain" text like 41 °C is OK.

By the way, the English Wikipedia works perfectly on the same machine. And it'll be so nice if custom CSS can be allowed, because the default choice of Chinese fonts is not quite visually appealing. Thanks.

This issue was moved by kelson42 from kiwix/kiwix-desktop#520.

[ghost]

@kelson42 commented on Sep 13, 2020, 9:24 AM UTC:

@wdscxsj Thank you fr your bug report. To better see the HTML of an article you can start the server mode and look via your usual Web browser.

[ghost]

@wdscxsj commented on Sep 13, 2020, 9:46 AM UTC:

@kelson42 Thank you so much for the pointer! It's quite surprising to see the English characters are missing from the DOM:

and the page source:

Any way to directly check the zim data?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[JokerQyou]

This is still an issue after almost two years.

[JokerQyou]

Take for example the "Antarctica" page, Wikipedia renders this:

while the same page in latest ZIM file from kiwix renders this:

You can see the <span lang="en">...</span> tag is missing.

the original wikitext content is:

'''南极洲'''（{{lang-en|Antarctica}}）是[[地球]]最南端的[[大洲|洲]]

in which the {{lang-en|Antarctica}} is a template for quickly creating English language variant for specific words. This kind of lead me to think if it's the parser failed to render this specific template.

However it's very confusing that the Japanese Wikipedia does not have the same issue, this page (and other archived Japanese pages) is OK:

This Japanese page uses another similar template called Lang-en-short.

[JokerQyou]

Upon further investigation and trying to understand what's going on here, it looks like <span lang="en"></span> elements are not the only ones affected. Texts are also missing in reference list and external link list of the exact same page:

It's normal on the original Wikipedia page:

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[Jaifroid]

Issue still current as of December 2023 (I tested with wikipedia_zh_history_maxi_2023-12.zim). I think this should be a higher priority for fixing. Can we find out whether the mobile endpoint is omitting the English-language text, or whether mwOffliner is somehow filtering it out? Since the text is not in the ZIM (it's not a case of it being hidden), it might be the former.