Fix master np113 by bjlittle · Pull Request #2616 · SciTools/iris

bjlittle · 2017-06-22T11:43:08Z

This PR resolves the current travis-ci issues for master, which arise from the latest release of numpy, which bumped from v1.12.1 to v1.13.0.

The issues are related to:

each output nD coordinate dtype produced by numpy.meshgrid is now the same as its associated 1D input coordinate dtype
a change within numpy.core.arrayprint.array2string means that we require to be more general with the formatter option i.e. numpystr -> all
correcting a data_manager unit test

bjlittle · 2017-06-22T13:45:35Z

What the heck is going on with travis ... sigh 😢

bjlittle · 2017-06-22T14:00:15Z

Ping @SciTools/iris-devs ... anyone fancy taking this on?

All PRs on master are broken until this gets merged ...

QuLogic · 2017-06-22T19:56:40Z

But, it... passes?

bjlittle · 2017-06-23T08:52:37Z

@QuLogic For example, this open PR #2614 is failing, see it's travis-ci build ... this PR will resolve those issues.

pp-mo · 2017-06-23T10:05:33Z

I see that, outside the tests, we still have just a couple of usages of the original meshgrid in iris.plot._map_common

Is there a particular reason for that, or did it just get overlooked ?

Obviously it's making no difference at this point, but I guess that would be true of some others.
I can understand why you don't bother changing all the test usages, if it rocks no boats. I guess that makes good sense.

pp-mo · 2017-06-23T10:06:52Z

lib/iris/__init__.py


 # Iris revision.
-__version__ = '2.0.0-DEV'
+__version__ = '2.0.dev0'


Can you maybe explain exactly what this is about, just for future reference ?

To be fair, this doesn't belong in this PR, so I'll pull it ... but the __version__ should be PEP440.

I'll save that for another day and another PR 👍

pp-mo · 2017-06-23T10:13:20Z

lib/iris/tests/results/analysis/rotated_pole.0.rlat.json

@@ -1 +1 @@
-{"std": 3.7195861803241614, "min": -5.593200206756592, "max": 8.72130012512207, "shape": [93, 75], "masked": false, "mean": 1.6348200228305594}
+{"std": 3.7195863723754883, "min": -5.5932002067565918, "max": 8.7213001251220703, "shape": [93, 75], "masked": false, "mean": 1.6348202228546143}


It might have been "nicer" here to reduce the tolerance on the test for when it happens again (!).
I see these values only just fail the test at a relative tolerance of 1e-7, as default for assertArrayAllClose (+hence assertDataAlmostEqual) : mean value relative change is approx 1.22e-7.
You could maybe add something like , reltol=5.0e-6 to the 4 calls here
You could do that instead, or as well if you prefer (changes are very small after all).

Just a suggestion -- skip if you don't want.

@pp-mo I'm really against fudging the tolerances. We took that step previously with testing and it ultimately lead to a bad place, particularly with graphical testing.

So, I'd rather not take that option, unless you consider it unwise.

I don't consider it fudging as long as the value used remains appropriate.
I know that was not always the case for our image comparison tolerance, where it got stupidly huge for a while... !
The default value of 1e-7 is designed to cope with float32 rounding errors, but can easily be a bit fine for testing other calculated values, like coordinate projections.
For instance, I just encountered some other tests failing at that level when I was messing about with different dask chunkings.

pp-mo · 2017-06-23T10:16:41Z

lib/iris/analysis/cartography.py

    return (x_range, y_range)


+def _meshgrid(x, y):


I think on balance I'd prefer this to live in iris.util, but only slightly -- your call.
Please do keep it private !

Agreed. Is this not behaviour that would be useful to have consistent across the entire package?

Perhaps if util were made a sub-package, then it would no longer be such a difficult call to add add additional 'utilities' for fear of further discontinuity:

iris.util.ndarray.meshgrid

In ANTS we have the following structure, which has served us very well:

ants.utils.ndarray ants.utils.coord ants.utils.cube

Not looked at the details, but numpy/numpy#5302 refers.

pp-mo · 2017-06-23T10:18:16Z

A couple of piffling suggestions : I should really have grouped them into a review, but now it's too late.

No show-stoppers, I am fully 👍 + happy to merge on your response.

pelson

I question whether we should be (effectively) mocking out meshgrid's behaviour and whether the numpy change was done intentionally - I'd be interested in having that chased up. In the meantime, 👍.

pelson · 2017-06-23T15:45:11Z

lib/iris/analysis/cartography.py

    return (x_range, y_range)


+def _meshgrid(x, y):


Not looked at the details, but numpy/numpy#5302 refers.

bjlittle · 2017-06-23T23:31:53Z

@pp-mo and @pelson I've addressed the review actions, so I think this PR is good to go ...

bjlittle added 2 commits June 22, 2017 12:30

Fix np113 meshgrid output dtype change.

ca4fabf

Fix np113 array2string formatter option.

73dbba0

bjlittle mentioned this pull request Jun 22, 2017

Calculation of derived coords points and bounds is always lazy. #2604

Merged

Fix data manager unit test.

3ab0d26

bjlittle force-pushed the fix-master-np113 branch from fbcca35 to 3ab0d26 Compare June 22, 2017 11:56

PEP440 version

5a6fba0

pp-mo reviewed Jun 23, 2017

View reviewed changes

pelson approved these changes Jun 23, 2017

View reviewed changes

Review actions.

4b7a191

This was referenced Jun 23, 2017

Update the documentation's copyright year... it's 2017. #2614

Merged

Added automatic push of docs to scitools-docs/iris. #2610

Merged

TEST: Extends #2569 to unpickle #2608

Merged

pelson merged commit ceaddb5 into SciTools:master Jun 24, 2017

bjlittle deleted the fix-master-np113 branch June 25, 2017 22:18

		@@ -1 +1 @@
		{"std": 3.7195861803241614, "min": -5.593200206756592, "max": 8.72130012512207, "shape": [93, 75], "masked": false, "mean": 1.6348200228305594} No newline at end of file
		{"std": 3.7195863723754883, "min": -5.5932002067565918, "max": 8.7213001251220703, "shape": [93, 75], "masked": false, "mean": 1.6348202228546143} No newline at end of file

Conversation

bjlittle commented Jun 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bjlittle commented Jun 22, 2017

Uh oh!

bjlittle commented Jun 22, 2017

Uh oh!

QuLogic commented Jun 22, 2017

Uh oh!

bjlittle commented Jun 23, 2017

Uh oh!

pp-mo commented Jun 23, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pp-mo Jun 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pp-mo commented Jun 23, 2017

Uh oh!

pelson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bjlittle commented Jun 23, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bjlittle commented Jun 22, 2017 •

edited

Loading

pp-mo Jun 23, 2017 •

edited

Loading