Skip to content

Backport PR #43608: REGR: spearman corr raising on 32-bit #43614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 17, 2021
Merged

Backport PR #43608: REGR: spearman corr raising on 32-bit #43614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
36a818b
Backport PR #43608: REGR: spearman corr raising on 32-bit
mzeitlin11 Sep 16, 2021
87a99b0
int64 -> intp
mzeitlin11 Sep 16, 2021
251bea2
fixups
mzeitlin11 Sep 16, 2021
8977507
Merge remote-tracking branch 'upstream/1.3.x' into backport-of-pr-436…
mzeitlin11 Sep 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.3.4.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Fixed regressions
~~~~~~~~~~~~~~~~~
- Fixed regression in :meth:`merge` with integer and ``NaN`` keys failing with ``outer`` merge (:issue:`43550`)
- Fixed performance regression in :meth:`MultiIndex.equals` (:issue:`43549`)
- Fixed regression in :meth:`DataFrame.corr` raising ``ValueError`` with ``method="spearman`` on 32-bit platforms (:issue:`43588`)
-

.. ---------------------------------------------------------------------------
Expand Down
6 changes: 3 additions & 3 deletions pandas/_libs/algos.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,11 +389,11 @@ def nancorr_spearman(ndarray[float64_t, ndim=2] mat, Py_ssize_t minp=1) -> ndarr
int64_t nobs = 0
bint no_nans
float64_t vx, vy, sumx, sumxx, sumyy, mean, divisor
const int64_t[:] labels_n, labels_nobs
const intp_t[:] labels_n, labels_nobs

N, K = (<object>mat).shape
# For compatibility when calling rank_1d
labels_n = np.zeros(N, dtype=np.int64)
labels_n = np.zeros(N, dtype=np.intp)

# Handle the edge case where we know all results will be nan
# to keep conditional logic inside loop simpler
Expand Down Expand Up @@ -451,7 +451,7 @@ def nancorr_spearman(ndarray[float64_t, ndim=2] mat, Py_ssize_t minp=1) -> ndarr
with gil:
# We need to slice back to nobs because rank_1d will
# require arrays of nobs length
labels_nobs = np.zeros(nobs, dtype=np.int64)
labels_nobs = np.zeros(nobs, dtype=np.intp)
rankedx = rank_1d(np.array(maskedx)[:nobs],
labels=labels_nobs)
rankedy = rank_1d(np.array(maskedy)[:nobs],
Expand Down
3 changes: 0 additions & 3 deletions pandas/tests/frame/methods/test_cov_corr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,6 @@ def test_corr_scipy_method(self, float_frame, method):

# ---------------------------------------------------------------------

@td.skip_if_no_scipy
def test_corr_non_numeric(self, float_string_frame):
# exclude non-numeric types
result = float_string_frame.corr()
Expand All @@ -125,11 +124,9 @@ def test_corr_nooverlap(self, meth):
assert rs.loc["B", "B"] == 1
assert isna(rs.loc["C", "C"])

@td.skip_if_no_scipy
@pytest.mark.parametrize("meth", ["pearson", "spearman"])
def test_corr_constant(self, meth):
# constant --> all NA

df = DataFrame(
{
"A": [1, 1, 1, np.nan, np.nan, np.nan],
Expand Down