Improve(?) explanation of SettingWithCopy warning

[ischwabacher]

After playing with R a bunch, I started feeling like the explanation of SettingWithCopy wasn't getting to the core of the matter, which is actually an essential consequence of python slice assignment semantics. Here's how python handles chained assignment:

df['foo']['bar'] = quux
# becomes
df.__getitem__('foo').__setitem__('bar', quux)

whereas in R, it's this:

df["foo"]["bar"] <- quux
# becomes
df["foo"] <- `[<-`(df["foo"], "bar", quux)
# becomes
df <- `[<-`(df, "foo", `[<-`(`[`(df, "foo"), "bar", quux))

That last is a lot of line noise, though the R method names [ and [<- are more concise than __getitem__ and __setitem__! But imagine that you could call __setitem__ with a kwarg inplace=False that would cause it to return a modified copy instead of modifying the original object. Then the R version would translate to this in python:

df = df.__setitem__('foo',
                    df.__getitem__('foo')
                        .__setitem__('bar', quux, inplace=False),
                    inplace=False)

This is incredibly awkward, but it has the advantage of making SettingWithCopy unnecessary— everything is a copy, and yet things get set nonetheless.

So this commit is an attempt to explain this without requiring the reader to know R.

[jreback]

thanks!

but you are making a wild assumption here, namely that users actually read documentation! (lmao)

[jreback]

note #11500 (hopefully going to happen for 0.18.0)

[jreback]

First sentence is odd here.

[ischwabacher]

Well, it'll be nice if this is obsolete by the time it hits a release, but it was fun to write anyway. :) And users do read documentation. Sometimes. If you go on StackOverflow and direct them to it.

[jreback]

I get what you are saying, but I think this is non-sensical to a user

[ischwabacher]

Maybe something like this:

You might be wondering whether dfmi.loc might be a copy of dfmi rather than a view, which would break the first indexing method. Pandas guarantees that dfmi.loc (and dfmi.ix and dfmi.iloc) are views into dfmi, which it can do because these must see all of dfmi. But dfmi.__getitem__(idx) is some subset or reordering of dfmi and therefore can't receive this guarantee.

[jreback]

I think better to explain that .loc does both operations (getting & setting) simultaneously and thus can guarnteed operate on the original object.

[ischwabacher]

Yeah, that's a better angle. It's the whole point of .loc, after all.

You may be wondering whether we should be concerned about the loc property in the first example. But dfmi.loc is guaranteed to be dfmi itself with modified indexing behavior, so dfmi.loc.__getitem__ / dfmi.loc.__setitem__ operate on dfmi directly. Of course, dfmi.loc.__getitem__(idx) may be a view or a copy of dfmi.

[jreback]

yep looks good..

squash em & ping

[ischwabacher]

Squashed. Just noticed I used "You might/may be wondering" to start two consecutive paragraphs. Oh well.

[jreback]

ok, feel free to edit again :)

[ischwabacher]

There. Plus some emphasis. :)

[jreback]

ok thanks! (and pls look at the built docs and see if anything needs tweaking)......usually at least 1 hr before travis builds..