pd.io.parsers.read_csv ignores skiprows when parse_dates is set to a dict

[cancan101]

For example:

pd.io.parsers.read_csv("http://www.datazoa.com/publish/export.asp?hash=yjPceG6fHL&uid=dzadmin&a=exportcsv",skiprows=range(1,13+1),skipfooter=4,parse_dates={"date":[0]})

has 907 rows.

As does:

pd.io.parsers.read_csv("http://www.datazoa.com/publish/export.asp?hash=yjPceG6fHL&uid=dzadmin&a=exportcsv",skipfooter=4,parse_dates={"date":[0]})

whereas:

pd.io.parsers.read_csv("http://www.datazoa.com/publish/export.asp?hash=yjPceG6fHL&uid=dzadmin&a=exportcsv",skiprows=range(1,13+1),skipfooter=4,parse_dates=[0])

has 894 rows.

I am on Pandas v0.11.0

[guyrt]

This is a symptom of a bigger problem:

s = "a,b,c\n" + "\n".join([",".join([str(i), str(i+1), str(i+2)]) for i in xrange(500)])
print pd.read_csv(StringIO(s),skiprows=[200, 202], engine='python')
  <class 'pandas.core.frame.DataFrame'>
  Int64Index: 500 entries, 0 to 499
  Data columns (total 3 columns):
  a    500  non-null values
  b    500  non-null values
  c    500  non-null values
  dtypes: int64(3)

Somehow, skiprows got removed in python engine except in the code that sniffs for the header. That's why the header isn't getting properly removed. Fix coming.

[cancan101]

I assume the issue exists in both the python and non-python engines?

[guyrt]

Just python. However, since skiprows is defined in the example, read_csv silently fails over to the python engine.

This is a prime example of why I don't like to have silent failover to an unanticipated code path.