Compatibility with seaborn 0.13

[getzze]

fixes #81 #122 #131 #133 #143

Compatibility with seaborn>=0.11
All tests are passing

In seaborn 0.12, internals were rewritten for categorical plots and in version 0.13 again.
There were also some changes to the interface:

• new parameters native_scale and formatter (I added support for them only for seaborn 0.13)
• hue can be used redundantly with a x or y variable to signal that the colors should change depending on this variable. Therefore I added code (and tests) to check for a redundant hue parameter.

I also removed the dependency blocker for pandas<2.0.0 because it does not affect statannotations but seaborn, therefore it should be dealt with in seaborn. Test are passing with pandas==2.2.2.

Organization of this PR:

• all the code dependent on seaborn version was put in a compat.py file.
• _GroupsPositions.py and _Plotter.py were reworked to be agnostic to seaborn internals.

[getzze]

Tests don't pass only for python 3.6 which is EOL since 2021-12-23. Python 3.7 is also EOL, and python 3.8 is soon-to-be btw.

Tests for python 3.12 are passing in local (only compatible with seaborn 0.13)

@trevismd you can review when you have time! Thanks

[mvarma-caris]

It'd be great if this could be merged!

[dnjst]

I can confirm this works on my computer! Thank you so much.

[getzze]

In the meantime, it would be great if people could test intensively. I only checked that tests were passing and that it was looking good for my usage.

So it needs testing visually all the plot types (boxplot, violinplot, swarmplot, stripplot and barplot), with and without hue, horizontal and vertical. I also didn't check compatibility with the new logscale option in seaborn 0.12.

Thanks!

[ryan-boobybiome]

Hi @getzze I really like these changes and, in some cases, work well for me. I have found an issue with this code from stackoverflow

import seaborn as sns
from statannotations.Annotator import Annotator

tips = sns.load_dataset("tips")
tips

args = dict(x="sex", y="total_bill", data=tips, hue="smoker", hue_order=["Yes","No"], order=['Male', 'Female'])

g = sns.catplot(edgecolor="black", err_kws={'color': 'black', 'linewidth': 1.5}, capsize = 0.1, height=4, aspect=.7,alpha=0.5, kind="bar", errorbar = "sd", row="time", **args)
g.map(sns.stripplot, args["x"], args["y"], args["hue"], hue_order=args["hue_order"], order=args["order"], palette='dark', dodge=True, alpha=0.6, linewidth=1)

pairs = [
    (("Male", "Yes"), ("Male", "No")),
    (("Female", "Yes"), ("Female", "No"))
]

pvalues = ['p=0.13', 'p=0.69']

for ax_n in g.axes:
    for ax in ax_n:
        annot = Annotator(ax, pairs, **args)
        annot.configure(test='Mann-Whitney', text_format='simple', loc='inside', verbose=2)
        annot.apply_test().annotate()

I get several warnings about group positions

[/Users/user/.virtualenvs/data-analysis/lib/python3.9/site-packages/statannotations/_GroupsPositions.py:133](https://file+.vscode-resource.vscode-cdn.net/Users/ryanames/.virtualenvs/data-analysis/lib/python3.9/site-packages/statannotations/_GroupsPositions.py:133): UserWarning: Invalid x-position found. Are the same parameters passed to seaborn and statannotations calls? Or are there few data points? The closest group position to -0.2 is 0
  warnings.warn(msg)

and the image isn't quite right (see lines for p-values)

[simbru]

Hi @getzze I really like these changes and, in some cases, work well for me. I have found an issue with this code from [stackoverflow]

@ryan-boobybiome I get these warnings also when using statannotations 0.6 main and sns.violinplot or sns.strippplot (Seaborn 0.12) with hue parameter set to a dataframe column. It doesn't seem to originate from getzze's branch.

But I can confirm the positioning and formatting of the p-value lines is also incorrect for me in the code you posted. With some further testing, it seems that anything that involves the hue parameter comparing two paired groups has this effect. The positioning being incorrect only occurs if loc = 'inside'. However, the lack of a horizontal bar occurs either way (inside/outside).

import seaborn as sns

from statannotations.Annotator import Annotator

df = sns.load_dataset("tips")
x = "day"
y = "total_bill"
order = ['Sun', 'Thur', 'Fri', 'Sat']

ax = sns.boxplot(data=df, x=x, y=y, order=order, hue = "sex")

# pairs=[("Thur", "Fri"), ("Thur", "Sat"), ("Fri", "Sun")]
pairs =[
    (("Sat", "Male"), ("Sat", "Female")),
    (("Thur", "Male"), ("Thur", "Female")),
    (("Sun", "Male"), ("Sun", "Female")),
    (("Thur", "Male"), ("Fri", "Female")),
    (("Fri", "Male"), ("Fri", "Female")),
    (("Sun", "Male"), ("Fri", "Female")),
    ]
annotator = Annotator(ax, pairs, data=df, x=x, y=y, order=order, hue = "sex")
annotator.configure(test='Mann-Whitney', text_format='star', loc='inside')
annotator.apply_and_annotate()

[getzze]

Thanks for reporting, I'll check that.

[getzze]

I fixed the problem with the position (it was a stupid missing not ). I also fixed displaying the annotation above violinplots (with Seaborn v0.13, the kde for the violins has to be computed again, it can slow down plotting).

[ryan-boobybiome]

@getzze many thanks - how do I update my installation from this branch to test? I did pip3 install git+https://github.com/getzze/statannotations.git@compat-seaborn-13 --upgrade but that doesn't appear to do anything (all requirements satisfied).

[getzze]

@ryan-boobybiome
I don't see why it wouldn't upgrade. You can always uninstall and install again.

[ryan-boobybiome]

@getzze uninstall and reinstall worked well. Not sure what happened with upgrade! But I can confirm it all looks good now. Many thanks for the help

[StevenSong]

@getzze uninstall and reinstall worked well. Not sure what happened with upgrade! But I can confirm it all looks good now. Many thanks for the help

the version is the same so upgrade won't reinstall unless forced or manually uninstalled/reinstalled. could maybe bump minor version to 0.7.0 since dropping support for python 3.6, otherwise maybe just a patch version bump to 0.6.1?

edit: version file is at statannotations/_version.py

[codecov]

Codecov Report

Attention: Patch coverage is 77.22222% with 164 lines in your changes missing coverage. Please review.

Project coverage is 91.37%. Comparing base (8f148c1) to head (1dd7714).

Files	Patch %	Lines
statannotations/compat.py	70.32%	135 Missing ⚠️
statannotations/_GroupsPositions.py	75.58%	21 Missing ⚠️
statannotations/_Plotter.py	92.13%	7 Missing ⚠️
statannotations/utils.py	90.90%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (8f148c1) and HEAD (1dd7714). Click for more details.

HEAD has 7 uploads less than BASE

Flag	BASE (8f148c1)	HEAD (1dd7714)
	12	5

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #155      +/-   ##
==========================================
- Coverage   97.11%   91.37%   -5.74%     
==========================================
  Files          29       31       +2     
  Lines        1939     2493     +554     
==========================================
+ Hits         1883     2278     +395     
- Misses         56      215     +159     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[getzze]

I tested with the script in #156 and corrected some bugs.
I also improved coverage.

[simbru]

What is currently blocking the merge? After some digging it seems that Seaborn 0.13 itself is not compatible with Python 3.6.

[trevismd]

A huge thank you for this contribution, @getzze . I'll soon merge this almost as is because I'm finding out I currently don't have the time to go in the nitty-gritty and the community really needs this.

I'll officially remove support for 3.6 too for this new 0.7.0-alpha.

[getzze]

Thanks @trevismd , let me know if I can help!

[jsgounot]

If anyone needs to use this nice package and can't downgrade their seaborn, here is a temporary fix:

1. Clone gettze fork version:
   git clone -b compat-seaborn-13 https://github.com/getzze/statannotations.git
2. Check you have all dependencies and you're using the correct environment. You can comment the install_requires in the setup.py if you don't want pip to download things.
3. Move to the cloned directory and install with pip locally: pip install .

[trevismd]

Hello @getzze ,

I started looking further into this and found a couple of issues when trying to run the usage notebook.

One is the creation of _Plotter.struct which fails if not all possible groups have data, even if not in the passed pairs.
I could make it work by adding a parameter allow_empty to get_group_data for _plotter.iter_data, but I'm not sure it's the right way to fix this. What do you think?

The other is that (I'm not sure if it's related to the above temp fix or not) annotations positions do not seem correct in the case of example_barplot_hue.png (the others seem OK) where we get

instead of

This is with python 3.12, seaborn 0.13.2, matplotlib 3.9.2.

Other than that, I see we lose the use of a palette when not defining a hue. This is probably a change from Seaborn itself, but we could migrate the example code to add the necessary parameters to keep the same rendering with this upgrade.

Thank you very much again for the thorough work and your patience.

[getzze]

I am checking it now and I also noticed that 'example_hue_layout.png' is completely wrong. Let me investigate...

Env: python 3.12, seaborn 0.13.2, matplotlib 3.9.2.

Other than that, I see we lose the use of a palette when not defining a hue. This is probably a change from Seaborn itself, but we could migrate the example code to add the necessary parameters to keep the same rendering with this upgrade.

Yes, it's a change in Seaborn, I will update the code to keep the same rendering.

[getzze]

One is the creation of _Plotter.struct which fails if not all possible groups have data, even if not in the passed pairs. I could make it work by adding a parameter allow_empty to get_group_data for _plotter.iter_data, but I'm not sure it's the right way to fix this. What do you think?

I implemented what you said (calling the argument strict).

The other is that (I'm not sure if it's related to the above temp fix or not) annotations positions do not seem correct in the case of example_barplot_hue.png (the others seem OK) where we get

This was the same error as in 'example_hue_layout.png', I fixed it by storing hue_order if it's provided, as it is not stored in the seaborn instance anymore in seaborn 0.13.

I also added the test_script.py file that I also added in PR #156 , but I think it makes more sense to add it in this PR. Running it with python usage/test_script.py creates images for a bunch of plots in vertical and horizontal, to check visually that everything is correct. It has command line arguments to run only specific data, plot_type and orientation.