ENH: Style blocks #15954

TomAugspurger · 2017-04-08T22:25:20Z

ENH: Add blocks to Styler template

This will make subclassing the Styler and extending
the templates easier.

REF: Move template to its own file

Use Environment and PackageLoader to load it

TomAugspurger · 2017-04-08T22:27:11Z

Here's a gist with the new docs: http://nbviewer.jupyter.org/gist/anonymous/db1b30a4ba4085bda612944c6ba9b64e#Extensibility

I think right now the template isn't included in the package, need to update setup.py. Will do later but have to run for now.

jreback

all for the reorg, do we need this in the top-level namespace?

maybe make a sub-dir for the style things at some point (e.g. pandas/formats/style (but another PR if you think its worth it).

jreback · 2017-04-08T22:30:59Z

doc/source/whatsnew/v0.20.0.txt

@@ -358,6 +358,9 @@ Other Enhancements
 - ``DataFrame.to_excel()`` has a new ``freeze_panes`` parameter to turn on Freeze Panes when exporting to Excel (:issue:`15160`)
 - ``pd.read_html()`` will parse multiple header rows, creating a multiindex header. (:issue:`13434`).
 - HTML table output skips ``colspan`` or ``rowspan`` attribute if equal to 1. (:issue:`15403`)
+- ``pd.Styler`` template now has blocks for easier extension (:issue:`15649`)


is this the right one? (maybe use a :class:`....` ?

jreback · 2017-04-08T22:31:24Z

pandas/__init__.py

@@ -56,6 +56,7 @@
 from pandas.util.print_versions import show_versions
 from pandas.io.api import *
 from pandas.util._tester import test
+from pandas.formats.style import Styler


you really want to add this to the top-level? (does this import stuff)?

I do think it needs to be importable somewhere 'public', not sure if top level is the right spot, could argue either way. Do we need a pandas.api.io?

jreback · 2017-04-08T22:32:20Z

pandas/formats/templates/html.tpl

@@ -0,0 +1,61 @@
+{%- block before_style -%}{%- endblock before_style -%}


make sure this is in the MANIFEST.IN

chris-b1

Looks pretty good, only one significant comment on structure of template

chris-b1 · 2017-04-09T12:48:34Z

pandas/formats/templates/html.tpl

+{%- endblock thead %} 
+{%- block tbody %} 
+<tbody> 
+    {%- for r in body %} 


Could you add a rows block here? (or alternatively empty before_rows and after_rows). What I was trying to do is add additional rows at the end of the table which I don't think is possible with these extension points.

Added a before_rows, after_rows and tr blocks. Here's the new structure

chris-b1 · 2017-04-09T12:51:16Z

pandas/__init__.py

@@ -56,6 +56,7 @@
 from pandas.util.print_versions import show_versions
 from pandas.io.api import *
 from pandas.util._tester import test
+from pandas.formats.style import Styler


I do think it needs to be importable somewhere 'public', not sure if top level is the right spot, could argue either way. Do we need a pandas.api.io?

chris-b1 · 2017-04-09T13:03:10Z

pandas/formats/style.py

@@ -400,7 +360,7 @@ def format(self, formatter, subset=None):
                self._display_funcs[(i, j)] = formatter
        return self

-    def render(self):
+    def render(self, **kwargs):
        """
        Render the built up styles to HTML



Add kwargs behavior to docstring

chris-b1 · 2017-04-09T13:03:52Z

From the notebook

For convenience, we provide the styler_factory function that does the same as the custom subclass.

styler_factory -> from_custom_template

TomAugspurger · 2017-04-09T13:16:15Z

Thanks. About the API: I don't really want it at the top level but it has to be public if people are subclassing. I just wasn't sure how we resolved the public API and submodules issue :)

…

On Apr 9, 2017, at 8:03 AM, chris-b1 ***@***.***> wrote: From the notebook For convenience, we provide the styler_factory function that does the same as the custom subclass. styler_factory -> from_custom_template — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

jreback · 2017-04-09T13:42:36Z

I do think it needs to be importable somewhere 'public', not sure if top level is the right spot, could argue either way. Do we need a pandas.api.io?

yes I think this is the way to go.

jorisvandenbossche · 2017-04-10T12:12:17Z

Do we need a pandas.api.io?

We already have a publicly accessible pandas.io, so if we want it in io, I would put it there.

TomAugspurger · 2017-04-10T12:52:08Z

We already have a publicly accessible pandas.io, so if we want it in io, I would put it there.

Yeah, having both would be confusing. Are people ok with pandas.io.Styler? (or would it be pandas.io.api.Styler?)

codecov · 2017-04-12T20:19:45Z

Codecov Report

Merging #15954 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #15954      +/-   ##
==========================================
+ Coverage      91%      91%   +<.01%     
==========================================
  Files         145      145              
  Lines       49581    49586       +5     
==========================================
+ Hits        45123    45128       +5     
  Misses       4458     4458

Flag	Coverage Δ
#multiple	`88.77% <100%> (ø)`	⬆️
#single	`40.57% <53.84%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/formats/style.py	`96.28% <100%> (+0.1%)`	⬆️
pandas/__init__.py	`92.85% <100%> (+0.17%)`	⬆️
pandas/core/algorithms.py	`94.53% <0%> (-0.05%)`	⬇️
pandas/core/base.py	`95.51% <0%> (ø)`	⬆️
pandas/core/series.py	`94.89% <0%> (ø)`	⬆️
pandas/indexes/base.py	`96.09% <0%> (ø)`	⬆️
pandas/core/categorical.py	`95.85% <0%> (ø)`	⬆️
pandas/tseries/index.py	`95.43% <0%> (ø)`	⬆️
pandas/util/testing.py	`81.13% <0%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c4d71ce...5bff74f. Read the comment docs.

codecov · 2017-04-12T20:19:50Z

Codecov Report

Merging #15954 into master will decrease coverage by 0.02%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master   #15954      +/-   ##
==========================================
- Coverage   91.02%      91%   -0.03%     
==========================================
  Files         145      146       +1     
  Lines       50391    50416      +25     
==========================================
+ Hits        45870    45880      +10     
- Misses       4521     4536      +15

Flag	Coverage Δ
#multiple	`88.8% <50%> (-0.03%)`	⬇️
#single	`40.32% <28.57%> (-0.02%)`	⬇️

Impacted Files	Coverage Δ
pandas/util/importing.py	`0% <0%> (ø)`
pandas/formats/style.py	`96.28% <100%> (+0.1%)`	⬆️
pandas/io/api.py	`71.42% <20%> (-28.58%)`	⬇️
pandas/core/common.py	`90.68% <0%> (-0.35%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 614a48e...c6765f3. Read the comment docs.

TomAugspurger · 2017-04-12T21:43:29Z

jinja2 being optional makes putting this in the namespace a bit strange. Happy to hear suggestions on ways to make that / the API tests better :/

jreback · 2017-04-14T13:37:48Z

pandas/formats/style.py

-    from jinja2 import Template
+    from jinja2 import (
+        PackageLoader, Environment, ChoiceLoader, FileSystemLoader
+    )
 except ImportError:
    msg = "pandas.Styler requires jinja2. "\


do

try: from jinja2 ..... except ImportError: class Styler(ImportError): pass return

will give you a Styler but if you try to use it and jinja2 is not installed will raise I think you can give it a nice message a well.

@jreback didn't quite work since you can't early-return from a module.

However, this does work:

try: from pandas.formats.style import Styler except: from pandas.compat import add_metaclass # We want to *not* raise an ImportError upon init # We *do* want to raise an ImportError with a custom message # when the class is instantiated or subclassed. _msg = ("pandas.io.api.Styler requires jinja2. " "Please install with `conda install Jinja2` " "or `pip install Jinaj2`") class _Check(type): """Raises import error upon subclassing.""" def __init__(cls, name, bases, clsdict): if len(cls.mro()) > 2: raise ImportError(_msg) super(_Check, cls).__init__(name, bases, clsdict) @add_metaclass(_Check) class Styler(object): def __init__(self, *args, **kargs): raise ImportError(_msg)

Are we ok with a metaclass, just for a nice error message? :D

oh if that works great.

chris-b1 · 2017-04-14T16:15:34Z

pandas/api/io/__init__.py

@@ -0,0 +1,2 @@
+""" Public IO API """
+from pandas.formats.style import Styler  # noqa


I'm fine with having a pandas.api.io, but isn't somewhat the opposite of the recommendation in #13634? Which is the top level modules would be 'public' and internals moved into core - so this should be exposed as pd.io.Styler? (I could be misunderstanding that issue)

pandas.io is going to stay, so importing from pandas.io is fine.

Huh, thought I removed that. Also might have just messed up my rebase. This isn't supposed to be there. It's supposed to be pandas.io.api

I think it should be just be pandas.io? The second level api are import * into the top level.

pandas.io.api imports * to pandas namespace; so you could make Styler just show up there (or you can leave the from pandas.io.api import * and just del Styler (prob shorter)

Use Environment and PackageLoader to load it TODO: packaging, __init__.py? ENH: Add blocks to Styler template This will make subclassing the Styler and extending the templates easier. Jinja2 is optional, so Styler is maybe not there Nicer import failure Fixed rebase

jorisvandenbossche · 2017-04-14T16:17:08Z

As I said above, we already have public things in pandas.io, so I don't really see the point of creating a new pandas.api.io ?

jorisvandenbossche · 2017-04-14T16:17:37Z

Ah, what @chris-b1 said :-)

jorisvandenbossche · 2017-04-14T16:23:59Z

Currently, our API documentation lists it as pandas.formats.style.Styler (so when moving, we should deprecate that). So one possibility would be to move the full formats module into pandas.io ? Or move to core and only expose the public ones in pandas.io (or in pandas.io.formats / pandas.io.style).

If we decide to move the formats module (which would certainly be for another PR), we could also just leave Styler where it is for now in this PR, and do the move / change of locations / deprecation in the other PR.

TomAugspurger · 2017-04-14T16:24:00Z

OK, should be good now. Sorry about that.

jreback · 2017-04-14T16:26:21Z

I am shortly going to move a bunch of things around. But pandas.io is going to stay (pandas.formats will move).

jreback · 2017-04-14T16:27:21Z

you could move .style under .io if you want.

jorisvandenbossche

api.rst will need an update as well, depending on what we decide

jorisvandenbossche · 2017-04-14T16:26:05Z

pandas/formats/style.py

@@ -410,6 +370,11 @@ def render(self):
        -------
        rendered: str
            the rendered HTML
+        **kwargs:


you need to make the docstring a raw string (or escape the *'s), otherwise sphinx will complain :-)

jorisvandenbossche · 2017-04-14T16:30:44Z

doc/source/whatsnew/v0.20.0.txt

@@ -482,6 +482,9 @@ Other Enhancements
 - ``DataFrame.to_excel()`` has a new ``freeze_panes`` parameter to turn on Freeze Panes when exporting to Excel (:issue:`15160`)
 - ``pd.read_html()`` will parse multiple header rows, creating a multiindex header. (:issue:`13434`).
 - HTML table output skips ``colspan`` or ``rowspan`` attribute if equal to 1. (:issue:`15403`)
+- ``pd.io.api.Styler`` template now has blocks for easier extension (:issue:`15649`)


You have an example in the notebook I suppose. Is it possible to put a link here to such an example? Not sure how it works with nbsphinx included files to put links to specific sections

TomAugspurger · 2017-04-14T17:21:39Z

You have an example in the notebook I suppose. Is it possible to put a link here to such an example? Not sure how it works with nbsphinx included files to put links to specific sections

@jorisvandenbossche yeah that worked! It's just

:ref`text <notebook.ipynb#section-title>`

jreback · 2017-04-14T17:32:43Z

pandas/io/api.py

+    # We *do* want to raise an ImportError with a custom message
+    # when the class is instantiated or subclassed.
+    @_add_metaclass(_UnSubclassable)
+    class Styler(object):


isn't this jinja2 ?

jreback · 2017-04-14T17:33:56Z

pandas/tests/api/test_api.py

@@ -49,7 +49,8 @@ class TestPDApi(Base, tm.TestCase):
               'Period', 'PeriodIndex', 'RangeIndex', 'UInt64Index',
               'Series', 'SparseArray', 'SparseDataFrame',
               'SparseSeries', 'TimeGrouper', 'Timedelta',
-               'TimedeltaIndex', 'Timestamp', 'Interval', 'IntervalIndex']
+               'TimedeltaIndex', 'Timestamp', 'Interval', 'IntervalIndex',
+               'Styler']


you can actually del Styler from pandas namespace

jreback · 2017-04-14T17:38:20Z

@TomAugspurger lgtm. merge when ready.

jreback · 2017-04-14T17:40:19Z

FYI: http://pandas-docs.github.io/pandas-docs-travis/style.html#Fun-stuff is raising in built dev docs.

jreback · 2017-04-14T17:40:44Z

ipywidgets I think is needed in doc-build.

TomAugspurger · 2017-04-14T18:33:23Z

ipywidgets I think is needed in doc-build.

Did you see that in one of the travis logs? https://travis-ci.org/pandas-dev/pandas/jobs/222148043 seems to be ok.

jreback · 2017-04-14T18:35:30Z

@TomAugspurger see the link above, its in the built docs.

TomAugspurger · 2017-04-14T18:37:16Z

Whoops, looked right past that. Yeah I guess they've split that out now.

I can do now, or in a separate PR since all the checks already passed here.

jreback · 2017-04-14T18:41:04Z

either way
or can add and push then merge

jreback · 2017-04-14T21:48:04Z

good to go?

jreback · 2017-04-14T21:48:57Z

pandas/io/api.py

@@ -17,6 +17,23 @@
 from pandas.io.pickle import read_pickle, to_pickle
 from pandas.io.packers import read_msgpack, to_msgpack
 from pandas.io.gbq import read_gbq
+try:
+    from pandas.formats.style import Styler
+except:


ImportError

jreback · 2017-04-15T13:59:04Z

thanks @TomAugspurger

jorisvandenbossche · 2017-04-15T14:23:22Z

Opened a follow-up issue for the remaining issues that were being discussed: #16009

jreback · 2017-04-15T14:34:06Z

http://pandas-docs.github.io/pandas-docs-travis/style.html

maybe internally should just catch warnings?

/home/travis/miniconda3/envs/pandas/lib/python3.5/site-packages/matplotlib/colors.py:581: RuntimeWarning: invalid value encountered in less
  cbook._putmask(xa, xa < 0.0, -1)

TomAugspurger added IO HTML read_html, to_html, Styler.apply, Styler.applymap Output-Formatting __repr__ of pandas objects, to_string labels Apr 8, 2017

TomAugspurger added this to the 0.20.0 milestone Apr 8, 2017

TomAugspurger requested a review from chris-b1 April 8, 2017 22:25

jreback requested changes Apr 8, 2017

View reviewed changes

chris-b1 reviewed Apr 9, 2017

View reviewed changes

TomAugspurger force-pushed the style-blocks branch from ee5f953 to 5bff74f Compare April 12, 2017 20:19

TomAugspurger force-pushed the style-blocks branch from 3c73d1c to bb58289 Compare April 12, 2017 21:42

jreback reviewed Apr 14, 2017

View reviewed changes

jreback approved these changes Apr 14, 2017

View reviewed changes

TomAugspurger force-pushed the style-blocks branch from f18025d to ca0622c Compare April 14, 2017 16:09

chris-b1 reviewed Apr 14, 2017

View reviewed changes

TomAugspurger force-pushed the style-blocks branch from ca0622c to 2ea04be Compare April 14, 2017 16:17

TomAugspurger added 2 commits April 14, 2017 11:20

Removed api.io

8c0d014

Fix references to pd.Styler

6e7a8d1

jorisvandenbossche reviewed Apr 14, 2017

View reviewed changes

jreback mentioned this pull request Apr 14, 2017

API: pandas.api #13634

Closed

DOC: Style docs fixups

b1ba5be

jreback reviewed Apr 14, 2017

View reviewed changes

Typo in error message

e9759b5

DEP: Added ipywidgets to doc build

4553308

jreback reviewed Apr 14, 2017

View reviewed changes

fix bare except

c6765f3

jreback merged commit 413e2c6 into pandas-dev:master Apr 15, 2017

jorisvandenbossche mentioned this pull request Apr 15, 2017

Follow-up #15954: public API for Styler #16009

Closed

3 tasks

This was referenced Apr 15, 2017

Styler render creates extra indent which when exported to markdown becomes a code block #15651

Closed

DOC: Use nbsphinx #15581

Merged

TomAugspurger deleted the style-blocks branch May 29, 2017 20:44

		@@ -0,0 +1,61 @@
		{%- block before_style -%}{%- endblock before_style -%}

		@@ -0,0 +1,2 @@
		""" Public IO API """
		from pandas.formats.style import Styler # noqa

ENH: Style blocks #15954

ENH: Style blocks #15954

Conversation

TomAugspurger commented Apr 8, 2017

TomAugspurger commented Apr 8, 2017

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chris-b1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chris-b1 commented Apr 9, 2017

TomAugspurger commented Apr 9, 2017 via email

jreback commented Apr 9, 2017

jorisvandenbossche commented Apr 10, 2017

TomAugspurger commented Apr 10, 2017

codecov bot commented Apr 12, 2017

Codecov Report

codecov bot commented Apr 12, 2017 • edited Loading

Codecov Report

TomAugspurger commented Apr 12, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche commented Apr 14, 2017

jorisvandenbossche commented Apr 14, 2017

jorisvandenbossche commented Apr 14, 2017

TomAugspurger commented Apr 14, 2017

jreback commented Apr 14, 2017

jreback commented Apr 14, 2017

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger commented Apr 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Apr 14, 2017

jreback commented Apr 14, 2017

jreback commented Apr 14, 2017

TomAugspurger commented Apr 14, 2017

jreback commented Apr 14, 2017

TomAugspurger commented Apr 14, 2017

jreback commented Apr 14, 2017

jreback commented Apr 14, 2017

Choose a reason for hiding this comment

jreback commented Apr 15, 2017

jorisvandenbossche commented Apr 15, 2017

jreback commented Apr 15, 2017

codecov bot commented Apr 12, 2017 •

edited

Loading

TomAugspurger commented Apr 14, 2017 •

edited

Loading