Validation error thrown for number that should be recognized as a number?

[nathantchan]

I have a csv with two columns, the first of which is the number of seconds since the unix epoch began, with nanosecond decimals, and a float after it.

timestamp, value
1519955200.898057929, 1.0
1519955200.899679433, 2.0
1519955200.900753377, 3.0

I added it to qri with the following command:

qri add --data <file.csv> <dataset name>

It threw a validation error:

qri validate <dataset name>
0: expected "1519955200.898057929" to be of type number

Seems like this should be recognized as a number, though.

[b5]

right you are, this is most likely a bug in our csv schema detection. Given that you've provided example data we have everything we need to replicate see if we can't ship a fix in the next release (we're aiming for Monday).

Thanks for filing @nathantchan!

[ramfox]

Great! So on further investigation, it looks like this is an issue with the dsio package.

In order to use the jsonschema validator on CSV data, we manipulate the csv data into a slice of bytes using the dsio.NewValueReader and dsio.NewValueBuffer functions.

NewValueReader, unfortunately, does not "encode" the way json.Marshal does. So, we end up with an array of strings, rather than an array of the most naturally fitting type. This creates the weird error in jsonschema. The schema specifies that it should be a number, but the data passed in has encoded it as a string.