Added rope to forestplot.py in forestplot() #448

GWeindel · 2018-12-11T09:57:14Z

The new argument rope adds a shaded region of practical equivalence for all displayed variables in forestplot, shade is controlled by rope_alpha (default =.5). Contrary to posteriorplot rope can only be a tuple of length 2.
Not big but it is highly useful for me hence I assume it should be useful for others too.

ahartikainen · 2018-12-11T10:37:35Z

Sounds good. Can you post example?

GWeindel · 2018-12-11T11:12:12Z

@ahartikainen Sure, by the way it allowed me to spot an error in the ymax definition.
I post a reproducible example and link a plot I use.

Reproducible example :

import pystan
import arviz as az

data = dict(N1=10, y1=2, N2=10, y2=6)

model = """//  Comparison of two groups with Binomial
data {
  int<lower=0> N1;
  int<lower=0> y1;
  int<lower=0> N2;
  int<lower=0> y2;
}
parameters {
  real<lower=0,upper=1> theta1;
  real<lower=0,upper=1> theta2;
}
model {
  theta1 ~ beta(1,1);
  theta2 ~ beta(1,1);
  y1 ~ binomial(N1,theta1);
  y2 ~ binomial(N2,theta2);
}
generated quantities {
  real oddsratio;
  oddsratio = (theta2/(1-theta2))/(theta1/(1-theta1));
}
"""

sm = pystan.StanModel(model_code=model)
fit = sm.sampling(data=data, iter=1000, chains=2)

az.plot_forest(fit, var_names=["theta1","theta2"], credible_interval=0.95, combined=True, rope=(-.1,.1),
               figsize=(7,7))

aloctavodia · 2018-12-11T16:36:54Z

I like the idea, but it seems to me that the user should be allowed to set a rope per variable.

canyon289 · 2018-12-11T17:26:20Z

Without having reviewed the details of the code, I would vote for consistency between the various plots

…

On Tuesday, December 11, 2018, Osvaldo Martin ***@***.***> wrote: I like the idea, but it seems to me that the user should be allowed to set a rope per variable. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#448 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AG4S4YOVTOp48XuKAEgD4D-QFIFtEkkSks5u398ngaJpZM4ZNCYm> .

GWeindel · 2018-12-12T07:53:55Z

Sure I get it, I have no use for it but some might and consistency is defintely important. I will adapt the rope code of posteriorplot() to forestplot() when I have a bit time and perhaps generalize it to the ridgeplot option. Thanks for this highly useful package by the way.

aloctavodia · 2018-12-12T12:55:48Z

Great! And thank you for this contribution.

Initial commit arviz-devs#448 completely rewritten. Rope can be a list of 2 or a dict like in posteriorplot() passed to the rope argument Argument rope_values writes ROPE values in plot when multiple ropes are given.

GWeindel · 2018-12-17T11:00:42Z

I severely rewrote my initial commit. Now rope has the same behavior as posteriorplot. I provide a reproductible example and a plot I could use.

Reproductible example :

import pystan
import arviz as az
import stan_utility
import matplotlib.pyplot as plt

data = dict(N1=10, y1=2, N2=10, y2=6)

model = """//  Comparison of two groups with Binomial
data {
  int<lower=0> N1;
  int<lower=0> y1;
  int<lower=0> N2;
  int<lower=0> y2;
}
parameters {
  real<lower=0,upper=1> theta1;
  real<lower=0,upper=1> theta2;
}
model {
  theta1 ~ beta(1,1);
  theta2 ~ beta(1,1);
  y1 ~ binomial(N1,theta1);
  y2 ~ binomial(N2,theta2);
}
generated quantities {
  real oddsratio;
  oddsratio = (theta2/(1-theta2))/(theta1/(1-theta1));
}
"""

sm = pystan.StanModel(model_code=model)
fit = sm.sampling(data=data, iter=1000, chains=2)

rope = {'theta1' : [{'rope': (-.1, .1)}],
 'theta2' : [{'rope': (.2, .5)}],
 'oddsratio' : [{'rope': (10, 20)}]}

ropeList = (-.1, .1)

az.plot_forest(fit, var_names=["theta1","theta2"],credible_interval=0.95, combined=True, rope=rope, rope_values=True)
plt.show()
az.plot_forest(fit, var_names=["theta1","theta2"], credible_interval=0.95, combined=True, rope=ropeList,  figsize=(7,7))
plt.show()

GWeindel · 2018-12-17T11:01:34Z

However, I don't master classes so I might have introduced a lot of redundancy...

GWeindel · 2018-12-17T18:20:56Z

Faster reproducible and more complete example taken from the gallery :

size = (1, 50)

data = {'normal': np.random.randn(*size),
    'gumbel': np.random.gumbel(size=size),
    'student t': np.random.standard_t(df=6, size=size),
    'exponential': np.random.exponential(size=size)}

rope = {'normal': [{"rope":(-.5,.5)}],
        'gumbel': [{"rope":(0,.5)}],
        'student t': [{"rope":(-.5,1)}],
        'exponential': [{"rope":(-.5,1)}]
       }

az.plot_forest(data, rope=(-1,1))
plt.show()
az.plot_forest(data, rope=rope)
plt.show()
az.plot_forest(data, rope=rope, rope_values=False)
plt.show()

aloctavodia · 2018-12-19T10:20:47Z

I like it. I think the rope should not be labeled here. I know it is for the posterior plot, by for that plot the hpd is also labeled.

GWeindel · 2018-12-19T14:32:09Z

You're right, I was hesitating whether to keep it.

canyon289

When running `./scripts/lint.sh/ I'm getting the following error

SRC_DIR=/home/tbayes/repos/arviz

echo 'Checking documentation...'
Checking documentation...

python -m pydocstyle --convention=numpy /home/tbayes/repos/arviz/arviz/
/home/tbayes/repos/arviz/arviz/plots/forestplot.py:246 in public method display_multiple_ropes:
D102: Missing docstring in public method

arviz/plots/forestplot.py

canyon289 · 2018-12-20T17:42:19Z

@GWeindel Thank you for the quick edits. I'll do another round of review either today or tomorrow.

canyon289 · 2018-12-23T15:58:55Z

@GWeindel Thanks for the fixes. Unfortunately our CI checks are broken at this commit so I'm running things manually as you make changes. Here's the errors I'm getting.

If you'd like you can rebase your changes on top of the current arviz master to get CI checks to work again. If you need help with this let me know.

Alternatively you can run

./scripts/lint.sh

and it will generate the output below

(arviz) tbayes@Users-MBP-2:~/repos/arviz$ ./scripts/lint.sh
++ pwd

SRC_DIR=/Users/tbayes/repos/arviz

echo 'Checking documentation...'
Checking documentation...

python -m pydocstyle --convention=numpy /Users/tbayes/repos/arviz/arviz/
/Users/tbayes/repos/arviz/arviz/plots/forestplot.py:246 in public method display_multiple_ropes:
D210: No whitespaces allowed surrounding docstring text
/Users/tbayes/repos/arviz/arviz/plots/forestplot.py:246 in public method display_multiple_ropes:
D400: First line should end with a period (not 'd')
/Users/tbayes/repos/arviz/arviz/plots/forestplot.py:246 in public method display_multiple_ropes:
D401: First line should be in imperative mood ('Display', not 'Displays')

ahartikainen · 2018-12-31T09:58:03Z

LGTM.

I fixed some annoying pylint errors.

canyon289 · 2018-12-31T16:41:12Z

Does not LGTM yet,
Need test for new multi rope functionality. I can add one later today

https://coveralls.io/builds/20848553/source?filename=arviz%2Fplots%2Fforestplot.py#L257
https://coveralls.io/builds/20848553/source?filename=arviz%2Fplots%2Fforestplot.py#L352

canyon289 · 2019-01-02T06:11:30Z

Give me a couple more days, got caught up with other tasks

canyon289 · 2019-01-04T07:18:21Z

Trying to add testing but messing up on arguments somewhere. The following code generates an exception

import arviz as az
import matplotlib.pyplot as plt

az.style.use('arviz-darkgrid')

centered_data = az.load_arviz_data('centered_eight')

rope = {'mu': [{'rope': (-.1, .1)}], 'tau': [{'rope': (.2, .5)}]}
_, axes = az.plot_forest(centered_data,
                         var_names=['mu', 'tau'],
                         rope=rope)
axes[0].set_title('Estimated theta for eight schools model')
plt.show()

will continue working on this

canyon289 · 2019-01-05T20:14:03Z

This implementation seems to break when each variable has multiple chains

Combined = True works

import arviz as az
import matplotlib.pyplot as plt
az.style.use('arviz-darkgrid')
centered_data = az.load_arviz_data('centered_eight')
rope = {'mu': [{'rope': (-.1, .1)}], 'tau': [{'rope': (.2, .5)}]}
_, axes = az.plot_forest(centered_data,
                         var_names=['mu', 'tau'],
                         combined=True,
                         rope=rope)
axes[0].set_title('Estimated theta for eight schools model')
plt.show()

Combined = False fails

import arviz as az
import matplotlib.pyplot as plt
az.style.use('arviz-darkgrid')
centered_data = az.load_arviz_data('centered_eight')
rope = {'mu': [{'rope': (-.1, .1)}], 'tau': [{'rope': (.2, .5)}]}
_, axes = az.plot_forest(centered_data,
                         var_names=['mu', 'tau'],
                         combined=False,
                         rope=rope)
axes[0].set_title('Estimated theta for eight schools model')
plt.show()

I have a couple suggested paths but happy to hear any others

Figure out how to make solution work with multiple chains
Provide a warning that per var rope won't work if multiple chains

I won't have time to code solution 1 if we choose to go that route, but can do solution 2.
Depending on which version we go for I can write the appropriate tests

GWeindel · 2019-01-08T14:08:07Z

I won't have the time (and perhaps sufficient skills) for solution 1. Hence I would suggest providing a warning.

ahartikainen · 2019-01-08T14:35:08Z

So what fails is

... label[ticks == y][0]

Which would mean that we don't have label for all y. Maybe test if y is found?

edit.

Line 331

self.display_multiple_ropes(rope, ax, y, linewidth, label[ticks == y][0])

to

if (ticks == y).any():
    self.display_multiple_ropes(rope, ax, y, linewidth, label[ticks == y][0])

canyon289 · 2019-01-08T15:07:27Z

What happens is that ArviZ is trying to match the label with the line. When the chains are compressed label position matches the y position in the numpy array so one value is found, we get it out of the array in index zero.

In multi chain the ticks, and the y position, are off every so slightly, .175 if I remember correctly. The equality operator doesn't match anything and the array is empty, hence the index error.

I hope this not so great description yields some insight. I can provide more detail if needed

ahartikainen · 2019-01-14T00:40:00Z

Ok

We need y_group variable which is the mean of y locations per parameter (or value to be plotted). VarHandler probably needs to calculate these before hand, because all the yielding is disabling to do that later?

Initial commit arviz-devs#448 completely rewritten. Rope can be a list of 2 or a dict like in posteriorplot() passed to the rope argument Argument rope_values writes ROPE values in plot when multiple ropes are given.

ahartikainen · 2019-01-14T12:31:03Z

Combined True

Combined False

canyon289 · 2019-01-14T16:14:17Z

Now we just need to write a couple more tests and we're good!

The new argument rope adds a shaded region of practical equivalence for all displayed variables in forestplot, shade is controlled by rope_alpha (default =.5). Contrary to posteriorplot rope can only be a tuple of length 2. Not big but it is highly useful for me hence I assume it should be useful for others too.

taking len(values) instead of values[-1] for ymax argument in axvspan()

Initial commit arviz-devs#448 completely rewritten. Rope can be a list of 2 or a dict like in posteriorplot() passed to the rope argument Argument rope_values writes ROPE values in plot when multiple ropes are given.

Fixed bug in ticks and title when specifying mutliple ROPES

- Removed unused variables - Moved the label/ticks collection to the case where a dict > 2 is provided - Added doc string to display_multiple_ropes() - Removed unused Variable xt_labelsize from ridgeplot() arguments

Additional changes made afterwards

canyon289 · 2019-01-15T06:42:44Z

Thanks for the contribution @GWeindel

GWeindel · 2019-01-15T07:14:41Z

Great ! Thanks for the help with the multilple chains and the tests !

canyon289 previously requested changes Dec 20, 2018

View reviewed changes

arviz/plots/forestplot.py Outdated Show resolved Hide resolved

arviz/plots/forestplot.py Show resolved Hide resolved

arviz/plots/forestplot.py Outdated Show resolved Hide resolved

arviz/plots/forestplot.py Outdated Show resolved Hide resolved

arviz/plots/forestplot.py Show resolved Hide resolved

canyon289 changed the title ~~Added rope to forestplot.py in forestplot()~~ [WIP] Added rope to forestplot.py in forestplot() Jan 4, 2019

ahartikainen force-pushed the patch-1 branch from 8de47ef to bdb0a5b Compare January 14, 2019 08:55

GWeindel and others added 19 commits January 14, 2019 21:51

Update forestplot.py

694a3c6

Corrected error in defining ymax

749e2c8

taking len(values) instead of values[-1] for ymax argument in axvspan()

Added possibility for multiple ropes

228a974

Initial commit arviz-devs#448 completely rewritten. Rope can be a list of 2 or a dict like in posteriorplot() passed to the rope argument Argument rope_values writes ROPE values in plot when multiple ropes are given.

Fixed ticks and title

fe70eea

Fixed bug in ticks and title when specifying mutliple ROPES

Removed rope_values

72819a2

Integration of @canyon289 comments

dce1818

- Removed unused variables - Moved the label/ticks collection to the case where a dict > 2 is provided - Added doc string to display_multiple_ropes() - Removed unused Variable xt_labelsize from ridgeplot() arguments

Fix linting errors

3bdd983

fix ridge input

2b63458

Fix whitespace issue

4fe42de

black by hand

8b72930

fix line-too-long and return ax from forestplot

fe29735

Fix pylint false-positive error

eb4f1e2

Move pylint comment

08715ff

black - remove whitespace

ce34ed8

WIP Commit for forest plot test

8e0cd53

y to y_mean

f5d9e1a

fix label issue

322ca0a

fix multimodel

ace6949

Add additional tests for plot_forest

75a2746

canyon289 force-pushed the patch-1 branch from e1fc65f to 75a2746 Compare January 15, 2019 05:52

canyon289 changed the title ~~[WIP] Added rope to forestplot.py in forestplot()~~ Added rope to forestplot.py in forestplot() Jan 15, 2019

canyon289 merged commit 836d9b9 into arviz-devs:master Jan 15, 2019

GWeindel deleted the patch-1 branch January 15, 2019 07:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added rope to forestplot.py in forestplot() #448

Added rope to forestplot.py in forestplot() #448

GWeindel commented Dec 11, 2018

ahartikainen commented Dec 11, 2018

GWeindel commented Dec 11, 2018

aloctavodia commented Dec 11, 2018

canyon289 commented Dec 11, 2018 via email

GWeindel commented Dec 12, 2018

aloctavodia commented Dec 12, 2018

GWeindel commented Dec 17, 2018

GWeindel commented Dec 17, 2018

GWeindel commented Dec 17, 2018

aloctavodia commented Dec 19, 2018

GWeindel commented Dec 19, 2018

canyon289 left a comment

canyon289 commented Dec 20, 2018

canyon289 commented Dec 23, 2018

ahartikainen commented Dec 31, 2018

canyon289 commented Dec 31, 2018 •

edited

Loading

canyon289 commented Jan 2, 2019

canyon289 commented Jan 4, 2019 •

edited

Loading

canyon289 commented Jan 5, 2019 •

edited

Loading

GWeindel commented Jan 8, 2019

ahartikainen commented Jan 8, 2019 •

edited

Loading

canyon289 commented Jan 8, 2019

ahartikainen commented Jan 14, 2019

ahartikainen commented Jan 14, 2019

canyon289 commented Jan 14, 2019

canyon289 commented Jan 15, 2019

GWeindel commented Jan 15, 2019

Added rope to forestplot.py in forestplot() #448

Added rope to forestplot.py in forestplot() #448

Conversation

GWeindel commented Dec 11, 2018

ahartikainen commented Dec 11, 2018

GWeindel commented Dec 11, 2018

aloctavodia commented Dec 11, 2018

canyon289 commented Dec 11, 2018 via email

GWeindel commented Dec 12, 2018

aloctavodia commented Dec 12, 2018

GWeindel commented Dec 17, 2018

GWeindel commented Dec 17, 2018

GWeindel commented Dec 17, 2018

aloctavodia commented Dec 19, 2018

GWeindel commented Dec 19, 2018

canyon289 left a comment

Choose a reason for hiding this comment

canyon289 commented Dec 20, 2018

canyon289 commented Dec 23, 2018

ahartikainen commented Dec 31, 2018

canyon289 commented Dec 31, 2018 • edited Loading

canyon289 commented Jan 2, 2019

canyon289 commented Jan 4, 2019 • edited Loading

canyon289 commented Jan 5, 2019 • edited Loading

GWeindel commented Jan 8, 2019

ahartikainen commented Jan 8, 2019 • edited Loading

canyon289 commented Jan 8, 2019

ahartikainen commented Jan 14, 2019

ahartikainen commented Jan 14, 2019

canyon289 commented Jan 14, 2019

canyon289 commented Jan 15, 2019

GWeindel commented Jan 15, 2019

canyon289 commented Dec 31, 2018 •

edited

Loading

canyon289 commented Jan 4, 2019 •

edited

Loading

canyon289 commented Jan 5, 2019 •

edited

Loading

ahartikainen commented Jan 8, 2019 •

edited

Loading