Review notebook 1 "Pandas is table oriented" #2

jorisvandenbossche · 2019-09-10T19:36:05Z

No description provided.

review-notebook-app · 2019-09-10T19:36:10Z

Check out this pull request on

You'll be able to see Jupyter notebook diff and discuss changes. Powered by ReviewNB.

TomAugspurger · 2019-09-16T18:35:04Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


Do we use column *names* or *labels* more often?

I typically prefer *labels*, since "names" can conflict with Index.name / MultiIndex.names.

Reply via ReviewNB

ok with using labels. I'll start listing these type of _conventions_ in order to make them consistent in the following tutorials.

TomAugspurger · 2019-09-16T18:35:05Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


Let's provide index= to the DataFrame call above, so that we have an Index object here and not a RangeIndex.

Reply via ReviewNB

ok, good point

TomAugspurger · 2019-09-16T18:35:05Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


Not quite true about object, especially if https://github.com/pandas-dev/pandas/pull/27949 is merged for 1.0.

Reply via ReviewNB

Even when that is merged, it will not yet be the default, right?

I would reword to somethinglike "any python object, but in practice often means strings"

As pointed in https://github.com/pandas-dev/pandas/issues/26831#issuecomment-529716754 we will exclude the discussion on object types in the first tutorial.

TomAugspurger · 2019-09-16T18:35:05Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


typo on Pandas (capital P).

Rather than "single column version", say something like "each column in a DataFrame is a Series".

Reply via ReviewNB

I'll convert all Pandas references with capital P to lowercase in all of the tutorials

'each column in a DataFrame is a Series' -> indedd, adjusted

jorisvandenbossche · 2019-09-21T15:09:29Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


characters -> strings? (I find that a bit more common name, although in SQL land character is maybe often used)

Reply via ReviewNB

i'll make sure adjustments and other tutorials will be adjusted to using strings (cfr. https://github.com/pandas-dev/pandas/pull/27949)

jorisvandenbossche · 2019-09-21T15:09:30Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


Instead of focusing on "a dataframe has attributes", I would maybe rather start with explaining the concepts of column names and index. And then only after that explain that those two "things" are attributes ?

Reply via ReviewNB

left discussion on attributes of a tabel out of the starting tutorial to descrease technical elements

jorisvandenbossche · 2019-09-21T15:09:30Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,620 @@
+{


I suppose the reason that you don't show that a column of a dataframe is a series, is because indexing is only explained in a later tutorial?
I would maybe consider to just show df[col] already here, which might give a more natural introduction to series = column of dataframe.

And then in a second step we can still show that you can create one manually as well

Reply via ReviewNB

jorisvandenbossche · 2019-09-21T15:14:16Z

A general remark on this tutorial is that it is "not very interesting". I am not sure how to make it more, though.

I fully admit I have this in my tutorials as well, but showing the .index, .columns, .dtypes attributes of a dataframe as a starter is maybe not attractive? I am wondering if we can have some other things in this first notebook, but the difficulty is of course that doing something actually useful directly needs some of the methods that is only seen in later notebooks ..

stijnvanhoey · 2019-09-23T10:52:13Z

I refactored with an attempt to keep a balance between:

introduction to dataframe/series concept
show that the aim is to DO something with it
make it more engaing
keep the technical level low enough

stijnvanhoey · 2019-09-23T10:58:55Z

@datapythonista and @WillAyd can you have a look to see if this improves the first tutorial?

WillAyd · 2019-09-23T15:31:23Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


These images (?) aren't rendering - maybe just an issue with ReviewNB?

Reply via ReviewNB

WillAyd · 2019-09-23T15:31:23Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


A few comments:

I think would be nice to just use df as the name of this instead of my_dataframe
I would defer introducing Categorical types until later
We don't need to specify the index= here as it matches the default

Reply via ReviewNB

Because df is such a widely used convention for naming DataFrames, I agree with @WillAyd that it would be better to use df than my_dataframe.

I also agree that there's no need to specify the index here. I think that @TomAugspurger suggested specifying the index to avoid a RangeIndex, but the index is no longer being printed out in this notebook, so it's not an issue.

my_dataframe -> df; good point, adjusted
index is indeed of no more usage, removed
I added categorical to make sure that when later on applying the describe method it is not taken into account. To overcome this, I now used Sexas a column instead of Pclass.

WillAyd · 2019-09-23T15:31:23Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


The Note seems a little strange to me. I get what you are saying but for a new user it doesn't really explain much, so probably best to remove

Reply via ReviewNB

On overcoming too much links towards _future_ tutorials and due to the potential confusion for new users, I removed this note.

WillAyd · 2019-09-23T15:31:24Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


I would delete the second sentence - I think users should be going through sequentially and not hopping around tutorial

Reply via ReviewNB

I think I'm in agreement with @WillAyd's suggestion here. Perhaps don't encourage users to jump ahead to future lessons, though in future lessons it does make sense to link back to previous lessons.

ok, good point!

WillAyd · 2019-09-23T15:31:24Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


I'd remove this comment as well - I don't think we care to draw that comparison this early on

Reply via ReviewNB

If you decide to keep this language, I would change "When you are familiar" to "If you are familiar".

Also, "base" should be "based".

I left the note there for the moment. I do think Python users will _like_ the link with something they already know.

WillAyd · 2019-09-23T15:31:24Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


"Aggregating Data" would be a more appropriate title

Reply via ReviewNB

I do not want to stress a specific function in that section and use the aggregation just as an example of applying _methods_ in general. As such, I use _do something_ in order to explain the general concept of methods applied on DataFrame/Series.

justmarkham · 2019-09-27T16:02:05Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


I would use the term "alias" rather than "shortcut".

Reply via ReviewNB

justmarkham · 2019-09-27T16:05:17Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


I've only ever seen the word "latter" used when there are two things discussed ("former" and "latter"). Since there are three things being discussed in this case, I would replace "the latter" with "the Pclass column".

Reply via ReviewNB

justmarkham · 2019-09-27T16:05:17Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


Typo: "parenthesis" should be "parentheses"

Reply via ReviewNB

justmarkham · 2019-09-27T16:09:26Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


Rather than saying "methods are like functions", I think we should just say "methods are functions".

Reply via ReviewNB

justmarkham · 2019-09-27T16:09:27Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


Typo: "PClass" should be "Pclass"

Also, I think one useful point to mention here is that many pandas operations return a DataFrame or a Series, and this is an example of a pandas operation (describe) returning a Series.

Reply via ReviewNB

Good point, added to the tutorial.

justmarkham · 2019-09-27T16:09:27Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,484 @@
+{


I've never heard the phrase "besides the looks", so perhaps change the wording?

Reply via ReviewNB

Adusted to:

> This is just a starting point. Similar to spreadsheet software, pandas represents data as a table with columns and rows. Apart from the representation, also the data manipulations and calculations you would do in spreadsheet software are supported by pandas. Continue reading the next tutorials to get you started!

Further improvement welcome

jorisvandenbossche · 2019-10-21T13:34:55Z

notebooks/1_table_oriented.ipynb

@@ -0,0 +1,476 @@
+{


But a Series still has a name ...
Is that something to mention? In the example above we also give it a name.

Reply via ReviewNB

Add 1_table_oriented.ipynb from master

13d241f

jorisvandenbossche mentioned this pull request Sep 10, 2019

Reviewing the content here #1

Open

TomAugspurger mentioned this pull request Sep 12, 2019

Add ReviewNB GitHub app dask/dask-examples#97

Closed

TomAugspurger reviewed Sep 16, 2019

View reviewed changes

jorisvandenbossche commented Sep 21, 2019

View reviewed changes

jorisvandenbossche mentioned this pull request Sep 23, 2019

DOC: Discussion on new getting started page (Numfocus Grant) pandas-dev/pandas#26831

Closed

Update from master

3c1838b

WillAyd reviewed Sep 23, 2019

View reviewed changes

justmarkham reviewed Sep 27, 2019

View reviewed changes

update from master

344cba9

jorisvandenbossche commented Oct 21, 2019

View reviewed changes

Review notebook 1 "Pandas is table oriented" #2

Are you sure you want to change the base?

Review notebook 1 "Pandas is table oriented" #2

Conversation

jorisvandenbossche commented Sep 10, 2019

review-notebook-app bot commented Sep 10, 2019

TomAugspurger Sep 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger Sep 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger Sep 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger Sep 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche Sep 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche Sep 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche Sep 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche commented Sep 21, 2019

stijnvanhoey commented Sep 23, 2019

stijnvanhoey commented Sep 23, 2019

WillAyd Sep 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd Sep 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd Sep 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd Sep 23, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justmarkham Sep 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justmarkham Sep 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justmarkham Sep 27, 2019 • edited Loading

Choose a reason for hiding this comment

justmarkham Sep 27, 2019 • edited Loading

Choose a reason for hiding this comment

justmarkham Sep 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger Sep 16, 2019 •

edited

Loading

TomAugspurger Sep 16, 2019 •

edited

Loading

TomAugspurger Sep 16, 2019 •

edited

Loading

TomAugspurger Sep 16, 2019 •

edited

Loading

jorisvandenbossche Sep 21, 2019 •

edited

Loading

jorisvandenbossche Sep 21, 2019 •

edited

Loading

jorisvandenbossche Sep 21, 2019 •

edited

Loading

WillAyd Sep 23, 2019 •

edited

Loading

WillAyd Sep 23, 2019 •

edited

Loading

WillAyd Sep 23, 2019 •

edited

Loading

WillAyd Sep 23, 2019 •

edited

Loading

justmarkham Sep 27, 2019 •

edited

Loading

justmarkham Sep 27, 2019 •

edited

Loading

justmarkham Sep 27, 2019 •

edited

Loading

justmarkham Sep 27, 2019 •

edited

Loading

justmarkham Sep 27, 2019 •

edited

Loading