REGR: Index.union preserve numerical index for range/float and int/float combinations

[jorisvandenbossche]

Recent change in master:

In [11]: index = pd.RangeIndex(0, 10)

In [12]: index.union(pd.Index([0.5, 1.5]))
Out[12]: Index([0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0], dtype='object')

the dtype should be float.

cc @topper-123

[topper-123]

Yeah, I'll look into it tonight

[jreback]

this follows our general rule which is that dissimilar cast to object

are you proposing a rule that we cast to float if float and numbering ?

[jorisvandenbossche]

this follows our general rule which is that dissimilar cast to object

That is not the "general" rule, in case of numerical indices. Eg (released pandas):

In [1]: pd.Index([1, 2, 3]).union(pd.Index([.5, 1.5]))
Out[1]: Float64Index([0.5, 1.0, 1.5, 2.0, 3.0], dtype='float64')

Or at least, was not the general rule, as I see that the above example also changed on master. So @topper-123 it is not specific to RangeIndex (I was thinking it might have been related to some of the changes to it you have been doing recently).

are you proposing a rule that we cast to float if float and numbering ?

For me, that seems very logical that int + float = float (for set ops), and this is also what we have been doing up to recently.

[jorisvandenbossche]

Possibly related to #23538

[topper-123]

If this has been done by me, it has not been on purpose. I agrre It is more logical and more efficient that unions of numerical indices return a numerical index rather than an object index

[jreback]

http://pandas-docs.github.io/pandas-docs-travis/whatsnew/v0.25.0.html#incompatible-index-type-unions

not averse to coercing to float if we have a float for one
we already coerce between range / int indices

[jorisvandenbossche]

Yes, that's the PR I was pointing at.
Updating the issue to reflect the discussion.

[topper-123]

I've bisected this and the cause of the change shown by @jorisvandenbossche is indeed #23538 (commit 20d0ad1).

I also agree this is a blocker a s this should not return a object dtype, as that will likely cause slowness and cause memory-usage to increase.

[jorisvandenbossche]

cc ArtinSarraf would you be interested to look into this?