in fread integer64 argument is ignored if int64 type-bump occurs out-of-sample

[st-pasha]

dt0 = data.table(a=seq(10000), b="100")
dt0[111, b := "1000000000000"]
f = tempfile()
fwrite(dt0, f)

fread(f, integer64="numeric")
           a     b
       <int> <i64>
    1:     1   100
    2:     2   100
    3:     3   100
    4:     4   100
   ---            
 9996:  9996   100
 9997:  9997   100
 9998:  9998   100
 9999:  9999   100
10000: 10000   100

fread(f, integer64="character")
           a     b
       <int> <i64>
    1:     1   100
    2:     2   100
    3:     3   100
    4:     4   100
   ---            
 9996:  9996   100
 9997:  9997   100
 9998:  9998   100
 9999:  9999   100
10000: 10000   100

[MichaelChirico]

Just came across this in fixing #3683. See test 899.1 with integer64 = 'character'

[MichaelChirico]

Useful comment from Matt here:

#2607 (comment)

[petres]

The workaround #2607 (comment) could be implemented quite easily in https://github.com/Rdatatable/data.table/blob/master/R/fread.R, as long as the bug isn't fixed (possible fix described here: #2607 (comment)). If not, at least the documentation of fread https://github.com/Rdatatable/data.table/blob/master/man/fread.Rd should be adjusted and mention it. Should I send a pull request?

[jangorecki]

@petres yes please, also please link this issue. Eventually if issue gets resolved and someone will forget to update documentation then status of the issue can be obtained after following link.

[greg-minshall]

hi. sorry, but does "type-bump occurs out-of-sample" mean, effectively, that the column looked, in the first however-many-rows, like "int", but then, later, oops, (2**32)+1000 showed up? (i'm getting integer64's on a specific column in a file, in spite of integer64 = "numeric" in my fread call.)

[MichaelChirico]

almost, not quite. the sample is not the first n rows, but rather 100 equally-spaced blocks of 100 rows. so the non-int row happens to fall outside those regions.

PS you may want to specify colClasses to avoid the type bump warning.

[greg-minshall]

thanks, @MichaelChirico. yes, i already specify colClasses. (my code was failing later, i guess tripping over the integer64 column, not having bit64.)