API change: Support different file encodings

[lorenzwalthert]

Migrated from ##374 (comment).

@krlmlr Do you think we should support more file encodings? Is the package enc limited to utf8 and latin1? If not, we could add an argument file_encoding to the exported style_*() functions. As mentioned in #374, formatR seems to handle the problems outlined in #374 well.

[krlmlr]

I'd rather ask users to convert their files/packages to UTF-8.

[lorenzwalthert]

I agree that one should use UTF-8 in principal, but I am not sure whether styler is the right place to tell people what encoding they should use 😊
Let's see what a quick poll yields.

[krlmlr]

See checks in r-pkgs, second bullet point ;-)

This is about DESCRIPTION, which also affects the encoding used for source files AFAIK.

[lorenzwalthert]

Ok, thanks.
I could not check because I am on a Unix machine now but is the default file encoding with RStudio UTF8 or the native encoding?

[lorenzwalthert]

As a result of this poll, which is probably not entirely free of bias, only 60% of R users use utf8 for sure. Around 35% use some default.

[lorenzwalthert]

If the default of windows users is the native encoding (and hence most likely latin1 in most cases), I think we can't ignore this.

[krlmlr]

Yeah, or change that default ;-) Maybe we could supply another package that reencodes as UTF-8 first, and have styler detect non-UTF-8 encoding and direct users there? Could be a part of usethis, too -- can you please check if there is something related to this and perhaps open an issue if not?

[lorenzwalthert]

You mean changing the RStudio default encoding for new R Scripts? I mean for packages, I think it might be utf8 already, but we also have users that are not package developers. Is utf8 encoding a standard or planned to become one in the tidyverse?

[krlmlr]

We should teach users/package authors/... to use the One True Character Encoding. I forgot about user scripts, but these too should be UTF-8 in my opinion. That standard has been around long enough, it doesn't seem anything else is going to replace it anytime soon.

Windows users still won't be able to use characters from non-native locales (e.g., can't use a Chinese letter on a US-English Windows) due to limitations of source(), but reading characters representable in the own locale works, I just checked.

[lorenzwalthert]

We should teach users/package authors/...

That sounds like we should consider doing a PR to tidyverse/style, adding a rule to use utf8 throughout.