Protect against escapes in strings

[MilesMcBain]

I saw a string with this sequence crash paint: 23\xbfC

[MilesMcBain]

Some reprex material:

foo <- "23\xbfC"
crayon::col_nchar(foo)

## Error in base::nchar(strip_style(x), ...): invalid multibyte string, element 1

Encoding(foo) <- "latin1"
crayon::col_nchar(foo)

## [1] 4

df <- data.frame(
    bad = "23\xbfC"
  )

paint(df)

## Error in base::nchar(strip_style(x), ...): invalid multibyte string, element 1

Encoding(df$bad) <- "latin1"

Idea: option to set a default encoding to be used in the event of an
exception?

[MilesMcBain]

From Dan Wilson, validEnc("23\xbfC")
From @joelnitta, stringi::stri_trans_general.

I think these two together might solve the problem

  library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#>     filter, lag
#> The following objects are masked from 'package:base':
#>
#>     intersect, setdiff, setequal, union
df <- data.frame(
    bad = "23\xbfC"
  )

df %>%
  mutate(
    good = case_when(
      !validEnc(bad) ~ stringi::stri_trans_general(bad, "latin-ascii")
    )
  ) %>%
  select(good) %>%
  paint::paint()
#> data.frame [1, 1]
#> good chr 23�C

^{Created on 2022-10-04 with reprex v2.0.2}

[danwwilson]

The other aspect to think about might be how to identify data that has been transformed to allow printing. Maybe invert background/foreground colours?

[MilesMcBain]

I like this idea @danwwilson!

[MilesMcBain]

Just hit this one "2015 President and Vice-Chancellor\x92s Alumni Scholarship Appeal" 🤪

[danwwilson]

another option is to try enc2utf8()

> enc2utf8("2015 President and Vice-Chancellor\x92s Alumni Scholarship Appeal")
[1] "2015 President and Vice-Chancellor<92>s Alumni Scholarship Appeal"