Skip to content

BUG: MultiIndex.set_levels with emtpy levels #16987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from

Conversation

leej3
Copy link

@leej3 leej3 commented Jul 16, 2017

Helped by @TomAugspurger for this one. May he can take a review the changes too.

Thanks.

@@ -428,6 +428,50 @@ def test_xs_multiindex(self):
expected.columns = expected.columns.droplevel('lvl1')
tm.assert_frame_equal(result, expected)

def test_set_level_checkall(self):

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference issue number here.

@pep8speaks
Copy link

pep8speaks commented Jul 17, 2017

Hello @leej3! Thanks for updating the PR.

Line 236:41: E202 whitespace before ')'

Comment last updated on September 07, 2017 at 23:02 Hours UTC

@leej3
Copy link
Author

leej3 commented Jul 17, 2017

I'll try pass some more tests for this tomorrow.

Copy link
Contributor

@TomAugspurger TomAugspurger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good so far. Sorry this ended up being such a slog :)

@@ -107,6 +107,13 @@ Other API Changes
- Compression defaults in HDF stores now follow pytable standards. Default is no compression and if ``complib`` is missing and ``complevel`` > 0 ``zlib`` is used (:issue:`15943`)
- ``Index.get_indexer_non_unique()`` now returns a ndarray indexer rather than an ``Index``; this is consistent with ``Index.get_indexer()`` (:issue:`16819`)
- Removed the ``@slow`` decorator from ``pandas.util.testing``, which caused issues for some downstream packages' test suites. Use ``@pytest.mark.slow`` instead, which achieves the same thing (:issue:`16850`)
- func:`pandas.MultiIndex.set_levels()` now allows the setting of empty levels and, when `level` is a list-like object, each element of levels is confirmed to be list-like (:issue:`16147`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can put this under bug fixes. And I think it'll be

:meth:`pandas.MultiIndex.set_levels` (but `:func:` may do the same thing)

Finally, I think you can remove the "when level is a list-like object, each element of levels is confirmed to be list-like" section. That's more of an internal thing I think, that we're checking all the items in levels instead of just the first.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, thanks. I'll sort this stuff out when I pass some of the other tests. I haven't accounted for all cases. Looks like it is indeed linked with #14823. I tried merging my code with that PR but still fail some tests, notably when levels consists of the Index class. I won't have much time over the next 2 weeks but we shall see.

"list-like object containing list-like objects")
if not levels_list_like:
raise TypeError(levels_error)
elif not level_list_like:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can just be else I think?

levels_error = ("`levels` and `level` are incompatible. "
"When `level` is a scalar, `levels` must be a "
" list-like object of scalars.")
if levels_list_like:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will behave incorrectly for setting index.set_index(levels=[], level=0). That should work, just setting that level's levels to []. In other words, this test should pass

def test_thing(self):
    index = pd.MultiIndex(levels=[['a', 'b'], [1, 2]], labels=[[], []])
    result = index.set_levels([], level=0)
    expected = pd.MultiIndex(levels=[[], [1, 2]], labels=[[], []])
    tm.assert_index_equal(result, expected)

lmk if that makes sense. I may be incorrect.

names=[u'foo', u'bar'])
tm.assert_index_equal(result, expected)

result = idx.set_levels(['a', 'b'], level=0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we have these tests passing, it may be good to split these up into multiple tests so it's easier to review.

@jreback jreback changed the title Issue #16147 fix BUG: MultiIndex.set_levels with emtpy levels Jul 17, 2017
- func:`pandas.MultiIndex.set_levels()` now allows the setting of empty levels and, when `level` is a list-like object, each element of levels is confirmed to be list-like (:issue:`16147`)

.. _whatsnew_0210.api:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this Other API Changes code as this has already been changed in master.

raise TypeError("Levels must be list-like")
if is_list_like(levels[0]):
raise TypeError("Levels must be list-like")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this validation is quite similar to .from_arrays. and should coordinate with changes in #14823 (cc @groutr)

raise TypeError("Levels must be list-like")

level_list_like = level is None or is_list_like(level)
levels_list_like = (is_list_like(levels) and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

further some of this validation is duplicative of _set_levels. So rather add any additional validation there directly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a try at this. Still some work to do on it

@leej3 leej3 force-pushed the issue_#16147_fix branch from 181e22b to 4d1655e Compare July 18, 2017 03:17
@jreback
Copy link
Contributor

jreback commented Sep 7, 2017

can you rebase and update

@jreback
Copy link
Contributor

jreback commented Oct 29, 2017

@leej3 can you rebase and update

@jreback
Copy link
Contributor

jreback commented Nov 25, 2017

can you rebase / update

@jreback
Copy link
Contributor

jreback commented Dec 28, 2017

closing as stale, if you want to work on this, pls ping.

@jreback jreback closed this Dec 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MultiIndex.set_levels and empty levels
5 participants