Fixed bug #9733 where stat functions returned a python scalar for empty series

[remiremi]

closes #9733

[jreback]

can you run thru all dtypes (look in pandas/src/inference.pyx) for a list
and do this for say sum/mean on empties and see what comes back and post it here. thanks.

obviously some of these will raise.

[jreback]

@remiremi can you update

[remiremi]

Yes, I have updated unit tests to loop through types and methods.

For numpy types, I compare the result type with the equivalent function in numpy.

For the string types, I guess the result should be empty string for the sum and nan for the rest.

It still needs some work: master...Remiremi:issue_9733_bis

Let me know if you don't think this is the right direction.

[jreback]

you can't change this line, otherwise other platforms will break
the_sum = values.sum(axis, dtype=dtype_max)

uints are very odd, not sure how to handle all this

[jreback]

@remiremi can you update? I

[jreback]

can you rebase / update?

[jreback]

@remiremi this looked ok, but I would like to run thru a number of dtypes comprehensively for this.

[jreback]

need a release note as well

[remiremi]

Just found some time to update the pull request, and added mention in v0.17.0.txt. Let me know

[jreback]

@remiremi well, you have taken on a task here. Can you update for the failures.

[remiremi]

Mmmh yes, my fix is taking the wrong approach. I'll update later this week.

[remiremi]

Just updated the branch but there's still a test failure (in test_resample, test_aggregate_with_nat). It looks to me like a bug in TimeGrouper. It somehow calls nanprod on an empty series, which expectedly returns 1.

[jreback]

@remiremi this whole thing is a can of worms, I have a very similar fix, see here for #9422

but actually getting this to work is annoying. Because the tests are now dependent on whether bottleneck is installed for testing (and what version), e.g. < 1.0 is the old behavior, >= 1.0 is the new. To make more complicated we actually turn off bottleneck for testing several places (though that is now fixed and consistent).

So need to fix both these issues, and they follow pretty much the same general soln. Want to have go?

[remiremi]

Yeah, will give it a shot when I have some time next week

On Fri, Aug 21, 2015 at 3:36 PM, Jeff Reback notifications@github.com
wrote:

@remiremi https://github.com/remiremi this whole thing is a can of
worms, I have a very similar fix, see here
#10815 for #9422
#9422

but actually getting this to work is annoying. Because the tests are now
dependent on whether bottleneck is installed for testing (and what
version), e.g. < 1.0 is the old behavior, >= 1.0 is the new. To make more
complicated we actually turn off bottleneck for testing several places
(though that is now fixed and consistent).

So need to fix both these issues, and they follow pretty much the same
general soln. Want to have go?

—
Reply to this email directly or view it on GitHub
#9829 (comment).

[remiremi]

@jreback I rebased today and ran the tests locally and the test test_aggregate_with_nat from test_resample is now successful! Do you know what change might have fixed the issue in TimeGrouper's groupby?

I guess all that needs to be done now is to make sure the tests still pass with both bottle < 1.0 and bottle >= 1.0, right?

[remiremi]

All tests passed with bottleneck 1.0 but there was a failure with 0.8... Getting closer

[jreback]

I know this is a beast!

closing, but pls reopen if you'd like to work on this again.