Improve plot clarity with combined mean and HDI legend elements #145

drbenvincent · 2022-12-27T20:54:42Z

Closes #142 and #143.

Previously the figure legends were quite messy. The main thing this PR does is to create single legend entries which include both the central tendency and the HDI. This is actually a bit involved and so we now build up a list of handles and labels in the plotting functions. With this we can call ax.legend once at the end of the plot functions. See my StackOverflow question and post on Twitter.
I updated the plot methods for all the pymc experiments to use this new approach. There is certainly still scope for slight improvement, but I think this is a good step forward in improving the plots.

Examples for synthetic control plots...

Before

After

Note the combined line+shaded region in the legend

Examples for difference in differences

Before

After

review-notebook-app · 2022-12-27T20:54:46Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

drbenvincent · 2022-12-27T22:35:43Z

I should update the plots in README

drbenvincent · 2022-12-28T07:13:37Z

I originally tried to do this on a better way, with everything handled internally in the plot function. But I ran into problems. It may be that I discovered a bug in matplotlib. See the solution here https://stackoverflow.com/a/74935897 If this can be made to work, it would be much cleaner.

juanitorduz · 2022-12-28T19:11:17Z

causalpy/plot_utils.py

    include_label: bool = True,
-) -> None:
+):


type hints are missing

updated in an upcoming commit

juanitorduz

I added some small suggestions. Mostly regarded commented code. This is very hard to maintain and we always have the git history to come back to them anyway. In addtion, what if we put the legends outside the figure as described in https://stackoverflow.com/questions/4700614/how-to-put-the-legend-outside-the-plot ?

juanitorduz · 2022-12-28T19:13:34Z

causalpy/plot_utils.py

@@ -13,28 +14,46 @@ def plot_xY(
    ax: plt.Axes,
    plot_hdi_kwargs: Optional[Dict[str, Any]] = None,
    hdi_prob: float = 0.94,
+    label: Optional[str] = "",


Remember Optional[str] is equivalent to Union[str, None. I think we probably do not want this so we should remove Optional. See #117

Changed to Union[str, None]. This also allowed me to remove a kwarg

juanitorduz · 2022-12-28T19:14:38Z

causalpy/plot_utils.py

        x,
        Y,
        hdi_prob=hdi_prob,
        fill_kwargs={
            "alpha": 0.25,
-            "label": f"{hdi_prob*100}% HDI" if include_label else None,
+            "label": " ",  # f"{hdi_prob*100}% HDI" if include_label else None,


shall we start removing commented code? this makes the code hard to maintain.

Fixed in an upcoming commit

juanitorduz · 2022-12-28T19:15:03Z

causalpy/plot_utils.py

+    # if include_label:
+    #     handles, labels = ax.get_legend_handles_labels()
+    #     ax.legend(
+    #         handles=[(h1, h2) for h1, h2 in zip(handles[::2], handles[1::2])],
+    #         # labels=[l1 + " + " + l2 for l1, l2 in zip(labels[::2], labels[1::2])],
+    #         labels=[l1 for l1 in labels[::2]],
+    #     )


same here regarding commented code

Fixed in an upcoming commit

juanitorduz · 2022-12-28T19:15:48Z

causalpy/pymc_experiments.py

            alpha=0.25,
-            label="Causal impact",
+            # label="Causal impact",


remove this duplicated line

Fixed in an upcoming commit

juanitorduz · 2022-12-28T19:16:23Z

causalpy/pymc_experiments.py

@@ -189,10 +209,14 @@ def plot(self):
                ls="-",
                lw=3,
                color="r",
-                label="Treatment time",
+                # label="Treatment time",


remove commented line?

Fixed in an upcoming commit

drbenvincent · 2023-01-04T11:58:28Z

Thanks for the comments @juanitorduz. I believe I've dealt with all those changes except for an external axis legend. I agree that it's suboptimal for the synthetic control plots, but I'll improve it in a separate PR.

I also re-ran some notebooks because I was not fond of the green shaded region for the causal impact.

juanitorduz · 2023-01-04T13:54:01Z

causalpy/pymc_experiments.py

@@ -137,7 +136,6 @@ def plot(self):
            self.datapost.index,
            self.post_pred["posterior_predictive"].mu,
            ax=ax[0],
-            include_label=False,
            # label="Synthetic control",


shall we remove this line?

juanitorduz · 2023-01-04T13:55:31Z

Just left a comment regarding a commented line: either remove or uncomment ;) Feel free to merge.

improve plots + plot posterior mu rather than posterior predictive

c98cfd9

drbenvincent requested a review from juanitorduz December 27, 2022 20:54

juanitorduz reviewed Dec 28, 2022

View reviewed changes

juanitorduz requested changes Dec 28, 2022

View reviewed changes

addressing Juan's comments

b1c417b

drbenvincent added 2 commits January 4, 2023 12:13

regenerate images for README

49cb072

fix minor error in notebook to generate images for README

b7f1b6d

juanitorduz reviewed Jan 4, 2023

View reviewed changes

juanitorduz approved these changes Jan 4, 2023

View reviewed changes

remove commented line

4a769b4

drbenvincent merged commit 2b1d82a into main Jan 4, 2023

drbenvincent deleted the improve-plot-legends branch January 4, 2023 14:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve plot clarity with combined mean and HDI legend elements #145

Improve plot clarity with combined mean and HDI legend elements #145

drbenvincent commented Dec 27, 2022

review-notebook-app bot commented Dec 27, 2022

drbenvincent commented Dec 27, 2022

drbenvincent commented Dec 28, 2022

juanitorduz Dec 28, 2022 •

edited

Loading

drbenvincent Jan 4, 2023

juanitorduz left a comment •

edited

Loading

juanitorduz Dec 28, 2022

drbenvincent Jan 4, 2023

juanitorduz Dec 28, 2022

drbenvincent Jan 4, 2023

juanitorduz Dec 28, 2022

drbenvincent Jan 4, 2023

juanitorduz Dec 28, 2022

drbenvincent Jan 4, 2023

juanitorduz Dec 28, 2022

drbenvincent Jan 4, 2023

drbenvincent commented Jan 4, 2023

juanitorduz Jan 4, 2023

drbenvincent Jan 4, 2023

juanitorduz commented Jan 4, 2023

Improve plot clarity with combined mean and HDI legend elements #145

Improve plot clarity with combined mean and HDI legend elements #145

Conversation

drbenvincent commented Dec 27, 2022

Examples for synthetic control plots...

Before

After

Examples for difference in differences

Before

After

review-notebook-app bot commented Dec 27, 2022

drbenvincent commented Dec 27, 2022

drbenvincent commented Dec 28, 2022

juanitorduz Dec 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juanitorduz left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drbenvincent commented Jan 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juanitorduz commented Jan 4, 2023

juanitorduz Dec 28, 2022 •

edited

Loading

juanitorduz left a comment •

edited

Loading