Support grabbing TeX source from MathML <annotation>s #27

distler · 2014-03-12T13:04:20Z

FeedParser changes might be better applied to latest FeedParser. But I see that's already a separate pull request.

Double-click on an equation to get the TeX source.

Also, some utf8 support improvements.

Errors like the following should be fixed now. ====================================================================== ERROR: test_content_html (tests.test_reconstitute.ReconstituteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 43, in <lambda> func = lambda self, name=root: self.eval(name) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 27, in eval scrub(testfiles%name, results) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/scrub.py", line 148, in scrub xhtml = serializer.XHTMLSerializer(inject_meta_charset = False) AttributeError: 'module' object has no attribute 'XHTMLSerializer' ====================================================================== ERROR: test_dc_lang (tests.test_reconstitute.ReconstituteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 43, in <lambda> func = lambda self, name=root: self.eval(name) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 28, in eval reconstitute(results, results.entries[0]).writexml(work) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/reconstitute.py", line 275, in reconstitute content(xentry, 'title', entry.get('title_detail',None), bozo) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/reconstitute.py", line 171, in content parser = html5parser.HTMLParser(tree=dom.TreeBuilder) AttributeError: 'module' object has no attribute 'TreeBuilder'

…mary_detail' Fixes errors like the following : ====================================================================== ERROR: test_content_html (tests.test_reconstitute.ReconstituteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 43, in <lambda> func = lambda self, name=root: self.eval(name) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 28, in eval reconstitute(results, results.entries[0]).writexml(work) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/reconstitute.py", line 267, in reconstitute id(xentry, entry) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/reconstitute.py", line 79, in id entry_id = (entry.summary_detail.base + "/" + File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/vendor/feedparser.py", line 416, in __getattr__ raise AttributeError, "object has no attribute '%s'" % key AttributeError: object has no attribute 'summary_detail'

Fixes : ====================================================================== FAIL: test_missing_item_pubDate (tests.test_reconstitute.ReconstituteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 43, in <lambda> func = lambda self, name=root: self.eval(name) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 38, in eval self.assertEqual(eval(rhs), eval(lhs, results.entries[0])) AssertionError: (2006, 6, 21, 13, 16, 41, 2, 172, 0) != time.struct_time(tm_year=2014, tm_mon=1, tm_mday=27, tm_hour=17, tm_min=16, tm_sec=24, tm_wday=0, tm_yday=27, tm_isdst=0) ----------------------------------------------------------------------

Fix errors like: ====================================================================== ERROR: test_spiderFeed_retroactive_filter (tests.test_spider.SpiderTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_spider.py", line 82, in test_spiderFeed_retroactive_filter self.spiderFeed(testfeed % '1b') File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_spider.py", line 50, in spiderFeed writeCache(feed_uri, feed_info, data) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/spider.py", line 232, in writeCache output = shell.run(filter, output, mode="filter") File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/shell/__init__.py", line 62, in run return module.run(template_resolved, doc, None, options) File "/usr/lib/python2.7/dist-packages/py/_apipkg.py", line 122, in __makeattr raise AttributeError(name) AttributeError: run

Fix errors like: ====================================================================== ERROR: test_apply_classic_fancy (tests.test_apply.ApplyTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_apply.py", line 54, in test_apply_classic_fancy self.apply_fancy() File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_apply.py", line 84, in apply_fancy splice.apply(self.feeddata) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/splice.py", line 142, in apply output_file = shell.run(template_file, doc) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/shell/__init__.py", line 66, in run module.run(template_resolved, doc, output_file, options) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/shell/tmpl.py", line 256, in run tp.set(key, value) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/vendor/htmltmpl.py", line 441, in set "be either a scalar or a list." % var TemplateError: Htmltmpl error: Value of toplevel variable 'date' must be either a scalar or a list.

Fix errors like : ====================================================================== ERROR: test_georss_box_latlong (tests.test_reconstitute.ReconstituteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 43, in <lambda> func = lambda self, name=root: self.eval(name) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_reconstitute.py", line 26, in eval results = feedparser.parse(data) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/vendor/feedparser.py", line 4001, in parse saxparser.parse(source) File "/usr/lib/python2.7/dist-packages/drv_libxml2.py", line 208, in parse eltName,eltQName,attributesNSImpl) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/vendor/feedparser.py", line 1801, in startElementNS raise UndeclaredNamespace, "'%s' is not associated with a namespace" % givenprefix UndeclaredNamespace: 'georss' is not associated with a namespace

Fix errors like this: ====================================================================== ERROR: test_django_entry_title (tests.test_filter_django.DjangoFilterTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/tests/test_filter_django.py", line 26, in test_django_entry_title os.path.realpath('tests/data/filter/django/title.html.dj'), input) File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/shell/dj.py", line 26, in run TEMPLATE_DIRS=(os.path.dirname(script),) File "/usr/lib/python2.7/dist-packages/django/conf/__init__.py", line 89, in configure raise RuntimeError('Settings already configured.') RuntimeError: Settings already configured.

I hope this isn't critical to the planet.

Solves errors like : ERROR:planet.runner:TypeError: 'NoneType' object has no attribute '__getitem__' ERROR:planet.runner: File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/spider.py", line 480, in spiderPlanet writeCache(uri, feed_info, data) ERROR:planet.runner: File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/spider.py", line 167, in writeCache scrub.scrub(feed_uri, data) ERROR:planet.runner: File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/scrub.py", line 149, in scrub node['value'] = ''.join([str(token) for token in tree]) ERROR:planet.runner: File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/vendor/html5lib/serializer/htmlserializer.py", line 199, in serialize for token in treewalker: ERROR:planet.runner: File "/home/olivier/svn/svn.debian.org/python-apps/planet-venus/git-svn/planet-venus/planet/vendor/html5lib/filters/optionaltags.py", line 18, in __iter__ type = token["type"]

Conflicts: planet/reconstitute.py planet/vendor/feedparser.py planet/vendor/html5lib/sanitizer.py

fred-wang · 2014-09-14T20:26:01Z

What's the status of this?

Update for feedparser 5.2.x

We were doing georss_box and georss_polygon wrong anyway.

distler and others added 19 commits July 26, 2012 10:48

Some Sanitizer fixes

fdb788e

Support for TeX source in <annotation>

eea8014

Double-click on an equation to get the TeX source.

Support LaTeX in img.latex

8693312

Also, some utf8 support improvements.

Update html5lib to 0.999

7e53bed

Update feedparser to 5.1.3

68a49a8

Proposed changes as feedparser seems to have a different behaviour.

88fa100

I hope this isn't critical to the planet.

More support for grabbing TeX source

01b09c3

Update feedparser

124d5f8

Merge branch 'update-deps' of https://github.com/olberger/venus

7ff1b74

Conflicts: planet/reconstitute.py planet/vendor/feedparser.py planet/vendor/html5lib/sanitizer.py

Update html5lib

af1a936

Yuck! That's a weird bug in HTML5lib.

13e8fc3

distler added 7 commits December 16, 2016 13:45

STIX Two + Astral plane characters in "narrow" Python builds

014375f

Might as well do this one too

89cd35f

Unvendor feedparser and httplib2

3405bc1

Update for feedparser 5.2.x

Fixes for new FeedParser

c0b3da2

A bit more responsive

9ccf292

Fix remaining broken tests

da2b0ca

We were doing georss_box and georss_polygon wrong anyway.

Update HTML5lib Sanitizer

b537638

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support grabbing TeX source from MathML <annotation>s #27

Support grabbing TeX source from MathML <annotation>s #27

distler commented Mar 12, 2014

fred-wang commented Sep 14, 2014

Support grabbing TeX source from MathML <annotation>s #27

Are you sure you want to change the base?

Support grabbing TeX source from MathML <annotation>s #27

Conversation

distler commented Mar 12, 2014

fred-wang commented Sep 14, 2014