BUG: SparseDataFrame from scipy.sparse.dok_matrix returns incorrect dataframe in python 2.7

[keitakurita]

Code Sample, a copy-pastable example if possible

Python 2.7

>>> import sys
>>> sys.version
'2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:05:08) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]'
>>> import numpy as np
>>> import scipy.sparse
>>> a = np.arange(1, 5).reshape(2,2)
>>> a
array([[1, 2],
       [3, 4]])
>>> spm = scipy.sparse.dok_matrix(a)
>>> spm
<2x2 sparse matrix of type '<type 'numpy.int64'>'
	with 4 stored elements in Dictionary Of Keys format>
>>> pd.SparseDataFrame(a)
   0  1
0  1  2
1  3  4
>>> pd.SparseDataFrame(spm)
   0  1
0  3  2
1  1  4

Python 3.6

>>> import sys
>>> sys.version
'3.6.1 |Continuum Analytics, Inc.| (default, Mar 22 2017, 19:25:17) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]'
>>> a = np.arange(1, 5).reshape(2,2)
>>> spm = scipy.sparse.dok_matrix(a)
>>> pd.SparseDataFrame(spm)
   0  1
0  1  2
1  3  4

Problem description

Initialization of SparseDataFrame with scipy.sparse.dok_matrix returns a different result from direct initialization from np.ndarray only in Python 2.7. This is inconsistent and unestimated behavior.
The scipy.sparse.dia_matrix shows similar buggy behavior.

Expected Output

>>> spm = scipy.sparse.dok_matrix(np.arange(1, 5).reshape(2,2))
>>> pd.SparseDataFrame(spm)
   0  1
0  1  2
1  3  4

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: 075eca1 python: 2.7.13.final.0 python-bits: 64 OS: Darwin OS-release: 16.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: ja_JP.UTF-8 LOCALE: None.None

pandas: 0.20.0rc1+29.g075eca1
pytest: 3.0.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.18.1
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None

[TomAugspurger]

Do you have the same pandas and scipy versions on both?

[keitakurita]

Yes, I have scipy version 0.18.1 installed on both environments (Python 3.6 and Python 2.7)

[jreback]

cc @kernc