robust encoding in fread (like 'fread("iconv -f ISO-8859-1 -t UTF-8 mytextfile.txt")')

[andreasio]

Since #563 is closed, I am adding this as a feature request.

fread still causes trouble on most of the files I work with (danish windows files). E.g. I have an iso-8850-1 windows file, that I can't fread correctly on linux. This workaround (#563 (comment) works). And read.csv works (i.e. produces the correct øæå letters, instead of e.g. \xe6), but fread(... encoding = "Latin-1") does not.

Sincerely

[arunsrinivasan]

Please provide a reproducible example.

Possible duplicate of #1726

[andreasio]

path <- "http://is.gd/0UMAfv"
testdata <- fread(path)

now when you do testdate$ rstudio will respond with this message, the moment you enter the $ sign:

Error in tolower(completions) : invalid multibyte string 1

names(testdata) returns this:

names(testdata)

[1] "F\xe6rdig"           "Forventet udl\xe6rt" "Afslut.\xe5rsag"     "Virk. navn"         
[5] "Lsted_id"  

None of theese issues happen if I perform them on windows, or if I save the file and do

fread("iconv -f ISO-8859-1 -t UTF-8 filename")

Hope this helps

Sincerely

[jangorecki]

@andreasio is it what you expect?

path <- "http://is.gd/0UMAfv"
testdata <- fread(path, encoding="Latin-1")
names(testdata)
#[1] "Færdig"           "Forventet udlært" "Afslut.årsag"     "Virk. navn"       "Lsted_id"   

If so, please re-run on latest devel version, it may have been fixed since you install it. If it didn't help please come back with sessionInfo().

[andreasio]

Yes Thats what i expect, but on my linux Box Thats not what Happens?
On Jun 24, 2016 22:44, "Jan Gorecki" notifications@github.com wrote:

@andreasio https://github.com/andreasio is it what you expect?

path <- "http://is.gd/0UMAfv"testdata <- fread(path, encoding="Latin-1")
names(testdata)#[1] "Færdig" "Forventet udlært" "Afslut.årsag" "Virk. navn" "Lsted_id"

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1748 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AA1xnxW6ebxnSFOM_Xm2ApvXelic8meWks5qPEGXgaJpZM4I7kJi
.

[andreasio]

in 1.9.7 it works as expected. Thanks.
On Jun 24, 2016 23:58, "Andreas Christoffersen" iam@andreas.io wrote:

Yes Thats what i expect, but on my linux Box Thats not what Happens?
On Jun 24, 2016 22:44, "Jan Gorecki" notifications@github.com wrote:

@andreasio https://github.com/andreasio is it what you expect?

path <- "http://is.gd/0UMAfv"testdata <- fread(path, encoding="Latin-1")
names(testdata)#[1] "Færdig" "Forventet udlært" "Afslut.årsag" "Virk. navn" "Lsted_id"

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1748 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AA1xnxW6ebxnSFOM_Xm2ApvXelic8meWks5qPEGXgaJpZM4I7kJi
.