Error when running the create_dendrogram example

[soungalo]

I am running plotly.py version 4.8.2 in a jupyter notebook.
I ran the following code, copied from the documentation page:

import plotly.figure_factory as ff
import numpy as np
np.random.seed(1)

X = np.random.rand(10,10)
fig = ff.create_dendrogram(X)
fig.update_layout(width=800, height=500)
fig.show()

and getting the following error message:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-1-386e391a19fb> in <module>
      4 
      5 X = np.random.rand(10,10)
----> 6 fig = ff.create_dendrogram(X)
      7 fig.update_layout(width=800, height=500)
      8 fig.show()

~/miniconda3/envs/jupyterlab_new/lib/python3.8/site-packages/plotly/figure_factory/_dendrogram.py in create_dendrogram(X, orientation, labels, colorscale, distfun, linkagefun, hovertext, color_threshold)
     89         distfun = scs.distance.pdist
     90 
---> 91     dendrogram = _Dendrogram(
     92         X,
     93         orientation,

~/miniconda3/envs/jupyterlab_new/lib/python3.8/site-packages/plotly/figure_factory/_dendrogram.py in __init__(self, X, orientation, labels, colorscale, width, height, xaxis, yaxis, distfun, linkagefun, hovertext, color_threshold)
    143             distfun = scs.distance.pdist
    144 
--> 145         (dd_traces, xvals, yvals, ordered_labels, leaves) = self.get_dendrogram_traces(
    146             X, colorscale, distfun, linkagefun, hovertext, color_threshold
    147         )

~/miniconda3/envs/jupyterlab_new/lib/python3.8/site-packages/plotly/figure_factory/_dendrogram.py in get_dendrogram_traces(self, X, colorscale, distfun, linkagefun, hovertext, color_threshold)
    335                 y=np.multiply(self.sign[self.yaxis], ys),
    336                 mode="lines",
--> 337                 marker=dict(color=colors[color_key]),
    338                 text=hovertext_label,
    339                 hoverinfo="text",

KeyError: 'C1'

I've tried downgrading to 4.8.1, but got the same error.
My working env is created with miniconda3, using the yml file:

name: jupyterlab_new
channels:
  - conda-forge
  - bioconda
dependencies:
  - jupyterlab
  - pandas
  - numpy
  - seaborn
  - plotly
  - biopython

Thanks!

[nicolaskruchten]

I can't replicate this in my local setup, so it seems to depend on a specific version of some dependency. Will keep digging.

[nicolaskruchten]

@emmanuelle @nicholas-esterer @jonmmease are any of you able to replicate this error? It's showing up in CI here: https://circleci.com/gh/plotly/plotly.py/42585?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

[emmanuelle]

can't replicate with python 3.7.3

[nicolaskruchten]

OK so it looks like scipy 1.5.0 breaks this. Works for me in 1.4.1 and not in 1.5.0, which was released Jun 21

[nicolaskruchten]

See also https://community.plotly.com/t/error-in-create-dendrogram-example/42006/2

[nicholas-esterer]

So, as @emmanuelle suggested, it is because the color_list returned by scipy now uses the cyclical color format from matplotlib: (see https://matplotlib.org/3.1.0/api/colors_api.html#module-matplotlib.colors, note about "CN" color spec).
However the new color_list is ['C1', 'C2', 'C0', 'C3', 'C4', 'C0', 'C0', 'C0', 'C0'] which while be isomorphic to the old color_list ['g', 'r', 'b', 'c', 'm', 'b', 'b', 'b', 'b'], does not represent the same set of colors, nor do the actual color values match any of those in the original list
So my idea was to also add these "C1", "C2" ... keys to the default_colors list and have them map to some colors like was done with the single letter named colors. But how to choose this mapping?
This is the new color map if one uses the corresponding colors in matplotlib.rcParams['axes.prop_cycle']

So do we want to replace these as well with different colors and if so, which ones? Because there are some colors that aren't in the list we already have (e.g., orange).

[nicolaskruchten]

Is there an easy fix such that we keep the colors we used to have?

[nicholas-esterer]

it would be mapping 'c1' to what 'g' was mapped to before, etc. but then what do we do for c5-c9?

[nicolaskruchten]

what did we do before for more than 4 colors? Sorry, I'm not following the before vs after :)

[nicholas-esterer]

Before the default color_list was ['g', 'r', 'b', 'c', 'm', 'b', 'b', 'b', 'b'] and now the default color_list is ['C1', 'C2', 'C0', 'C3', 'C4', 'C0', 'C0', 'C0', 'C0']. None of keys exist in the dictionary returned by default by _dendrite.get_color_dict ( dictionary that looks roughly like: { 'r': "rgb(255,65,54)", 'b': "rgb(0,116,217)", ... }), hence the key error.
Anyhow if plotly.py doesn't support this "cyclic format" then we actually just need to add 'C1' to the get_color_dict dictionary, because the only case that the color 'C1' would be specified is when the user doesn't specify a color and so the default color is looked up, it would just probably be more robust to map all the colors so that if the color_list changes again (say it changes the order, or the set of colors), you still get something. But maybe not.

[nicholas-esterer]

By before I mean with scipy===1.4.1 and now I mean scipy===1.5.1.