Added HTML repr for Parameterized

[jbednar]

Parameterized's repr works well for small numbers of parameters, but as it's all on a single line of text, quickly becomes not very useful for larger classes:

import param
class A(param.Parameterized):
   b = param.Integer(5)
   c = param.Selector([param.Parameterized(name="Z"), param.Parameterized(name="X")])

a = A()
repr(a)

"A(b=5, c=Parameterized(name='Z'), name='A00108')"

This PR adds an HTML representation that is similar to the parameter listing in the help output:

but as a table listing only the parameters:

[coveralls]

Coverage decreased (-0.6%) to 75.966% when pulling 391e4dc on htmlrepr into 97a26a7 on master.

[jbednar]

It's still WIP, with various open issues:

• What should happen to Parameterized subobjects? Right now they are shown as closed details items, but expanding them doesn't format well and presumably there will be problems for any circular references as for the string repr:
  
• Can we include the docstrings somehow as hover text or as details to expand? It would be great to be able to show the full details for any parameter as in the example at https://datatables.net/ , but I haven't found a suitable pure-CSS or at least simple JS way to achieve that.
• Should we use something like this to support automatically generated reference material for the various HoloViz projects? E.g. instead of saying "For full documentation and the available style and plot options, use hv.help(hv.Area)" as we do now in the Reference Gallery for e.g. hv.Area, we could add a section of actual (auto-generated) HTML that lets users see what parameters are defined on that object. We might also consider extending this HTML representation to something HoloViews specific, also displaying the list of supported options. I think that could solve some of the gaps between our explicit reference material and our API docs, with a representation like this concisely documenting the full set of available parameters (and options, for hv) without us having to maintain it.
• I've mapped the HTML representation to Parameterized._html_repr_, but that probably needs updating to use a mimebundle because otherwise it will cause issues with HoloViews by overriding the mimebundle-based representation it already has:
  
• The implementation is currently py3 only, and probably needs some attention to make it not cause errors on py2 (though of course it could easily be made py2/3 compatible).

[jbednar]

I've mapped the HTML representation to Parameterized.html_repr, but that probably needs updating to use a mimebundle because otherwise it will cause issues with HoloViews by overriding the mimebundle-based representation it already has:

Actually, should I be focusing on making an HTML repr for obj.param, not obj? That way there would never be any conflict with the repr for the object itself; this would be a repr for the Parameterized aspects of the object. Hmm..

[philippjfr]

Actually, should I be focusing on making an HTML repr for obj.param, not obj? That way there would never be any conflict with the repr for the object itself; this would be a repr for the Parameterized aspects of the object. Hmm..

Yes, I like this, it's also less discoverable but on balance I think this is nice. Alternatively just do both and then if the Parameterized repr is overridden you can still view it via .param.

[jbednar]

Sounds good; always have obj.param's repr available, but also by default have it as the repr for the object itself. For that to work, any idea how to avoid Param's repr hiding the HoloViews repr (which it does in the current implementation)?

[philippjfr]

Rebased and did a few things:

• Implemented the repr for classes
• Implemented the repr on Parameterized.param
• Display bounds and valid objects (for Selectors) in the same column:

[philippjfr]

any idea how to avoid Param's repr hiding the HoloViews repr (which it does in the current implementation)?

Not sure why but this no longer appears to be an issue.

[MarcSkovMadsen]

A way to see the doc string would be very valuable. For example for interactive docs and cheat sheets

[jbednar]

This looks great, thanks! Any answers for the checklist items above? On a quick scan they nearly all still appear to be valid issues.

[maximlt]

@jbednar @philippjfr do you think this is going to be ready for 2.0?

[philippjfr]

I think it's a huge usability improvement so I'd like to push this over the finish line.

[philippjfr]

Docstring now displayed as tooltips:

And recursion is handled:

[maximlt]

Just noticed that pn.panel would now display Parameterized objects with the HTML pane, calling _repr_html_, instead of building a Param pane:

[maximlt]

Is it supposed to display classes with the rich output? If so that's not what I see:

[jbednar]

Yes, I would expect and hope for rich output for both classes and instances. Maybe a missing @bothmethod elsewhere?

[maximlt]

The current implementation of bothmethod was turning the method into a functools.partial object whose type is not MethodType (not FunctionType either). IPython checks the type of the repr function (e.g. _repr_html_) on the object to be displayed:

https://github.com/ipython/ipython/blob/9da6376aa74dad72f5a84cdd0bf5ad0f1d809308/IPython/utils/dir2.py#L78-L79

This was failing with the current implementation, which I have replaced in 618b88e to satisfy IPython's inspection approach.

[maximlt]

[maximlt]

Almost 3 years in the making, great work @jbednar and @philippjfr !

[jbednar]

@philippjfr , the intention was to require .param to invoke this, right? Right now it's using the HTML repr for the instance or class itself, as well as for .param. I'm not sure whether that's a bad thing, but I don't think it's what we agreed on.

[maximlt]

Sounds good; always have obj.param's repr available, but also by default have it as the repr for the object itself. For that to work, any idea how to avoid Param's repr hiding the HoloViews repr (which it does in the current implementation)?

Didn't you agree there the rich display should be used in both cases?

[philippjfr]

Yep, what @maximlt said.

[maximlt]

Ah! I know what happened with your HoloViews examples @jbednar in #425 (comment).

So when you forget to load the extension it will fallback to displaying the rich representation.

Same for Panel objects, except that an informative warning is displayed:

Presumably HoloViews should also display the same warning?

[philippjfr]

Ah! I know what happened with your HoloViews examples @jbednar in #425 (comment).

For HoloViews that's a problem, because the custom HoloViews repr is meant to show the structure and dimensions of the object. We probably need to find a clean entrypoint for extending the repr with that info.

[maximlt]

Isn't it always a mistake not to load the HoloViews extension in a notebook?

Usually when I want to see the custom HoloViz repr I simply print the object, calling repr works too of course:

[hoxbro]

I would not expect to see this output when calling type on the object:

[philippjfr]

Why not?

[philippjfr]

That said, I do think if we're displaying the class it should include the module like a regular type repr would.

[hoxbro]

I would expect it to show me the type that is what I asked for. It did not ask to show all the param objects.

[philippjfr]

Personally don't feel super strongly about it but in the discussion above we did agree to implement the repr for classes, instances and for the .param accessor.

[hoxbro]

I would say this is an unexpected side effect of the discussion made above.

[philippjfr]

How so? type(obj) just returns the class and we explicitly enabled the repr on a class. @jbednar made the point to me that the class is precisely the point at which you want to see the parameters since it gives you information about what you can set in the constructor.

[hoxbro]

I understand that from a technical perspective, it makes sense why it appears. But for me, as a user, I do not see type(pn.widgets.IntSlider()) to be equivalent to pn.widgets.IntSlider when running them, even though they are returning the same in a notebook.

I would expect type(pn.widgets.IntSlider()) to show me the type and only the type of the object. That is what other object does in a notebook.

When calling pn.widgets.IntSlider directly I have no problem with it showing me the parameters.

[maximlt]

I have mixed feelings about this.

@jbednar made the point to me that the class is precisely the point at which you want to see the parameters since it gives you information about what you can set in the constructor.

Isn't that conflating a little the help with the repr? If we really want the rich repr to be used as an alternative help, shouldn't we go one step further and make it more look like the current help? E.g. including the class docstring (e.g. Panel objs include a link to the docs which is very useful), making the Parameters' doc first-class citizens. Oh and we all know that the (rich) repr isn't providing complete information about the type of objects you can provide to a constructor, as a class can implement some conversion in its __init__.

Looking at the examples below, there are a few things that can be confusing:

• neither the rich repr nor the help include the string repr
• the help has a table representation of the object that doesn't represent its current state (it's built when the Parameterized class is declared, I guess?)

Class:

Instance:

[jbednar]

@jbednar made the point to me that the class is precisely the point at which you want to see the parameters since it gives you information about what you can set in the constructor.

Yes, but I think I was answering a different question than I was being asked. I thought the question was whether classes should provide an HTML repr as well as instances, and my answer is as stated (yes!). But there's a different question about whether it needs to be invoked with .param or not. I'm still on the fence about that given the various examples above; e.g. I agree with @hoxbro that it's surprising and confusing that type(A) gives an HTML repr like this.

For comparison, Xarray does not:

I agree with @maximlt that we are conflating the help with the repr. If it's feasible, I'd like the repr to be a representation that is much more compact and brief than the help, but conveying the same information when the user interacts with it by hovering or by expanding sections. Conversely, invoking help would give you a fuller version, already expanded and more easily scannable by eye. In any case, yes, it's odd not to include the docstring.

[maximlt]

Given the doubts above, how about we remove Parameterized._repr_html before releasing Param 2.0? The rich repr would still be available on .param, for both instances and classes. We can still refine later given our experience of using it and user feedback. That'd be my vote.

Simon also just found out that VS Code:

1. computes and stores the output of Parameterize._repr_html when displaying a Panel object
2. re-opening a closed VS Code notebook, this rich repr is displayed instead of the plain output (VS Code doesn't seem to display widgets, I tried with ipywidgets directly and you just get the widget repr)

We'll have to disentangle that but it's another argument for removing Parameterize._repr_html, for now.

[philippjfr]

Okay, I think apart from the overall merits it's clear that this has too much scope to disrupt things so I vote removing it too.