ColorLS: incompatible character encodings: UTF-8 and ASCII-8BIT

[gogainda]

ColorLS fails with similar issues:

ColorLS::Flags with no flags does not list file info

     Failure/Error: entry = "#{logo.encode(Encoding.default_external, undef: :replace, replace: '')}  #{name}"

     Encoding::CompatibilityError:

       incompatible character encodings: UTF-8 and ASCII-8BIT

     # ./lib/colorls/core.rb:293:in `fetch_string'

     # ./lib/colorls/core.rb:302:in `block in ls_line'

     # <internal:core> core/enumerable.rb:339:in `each_with_index'

     # ./lib/colorls/core.rb:301:in `ls_line'

     # ./lib/colorls/core.rb:49:in `block in ls'

     # ./lib/colorls/layout.rb:14:in `block in each_line'

     # <internal:core> core/enumerator.rb:93:in `block in each'

     # <internal:core> core/enumerator.rb:101:in `block in each_with_block'

     # <internal:core> core/enumerable.rb:612:in `block in each_slice'

     # <internal:core> core/enumerable.rb:608:in `each'

     # <internal:core> core/enumerable.rb:608:in `each_slice'

     # <internal:core> core/enumerator.rb:100:in `each_with_block'

     # <internal:core> core/enumerator.rb:93:in `each'

     # ./lib/colorls/layout.rb:14:in `each_line'

     # ./lib/colorls/core.rb:48:in `ls'

     # ./lib/colorls/flags.rb:48:in `block in process'

     # <internal:core> core/enumerable.rb:339:in `each_with_index'

     # ./lib/colorls/flags.rb:43:in `process'

     # ./spec/color_ls/flags_spec.rb:9:in `subject'

     # <internal:core> core/hash.rb:223:in `fetch'

     # <internal:core> core/hash.rb:223:in `fetch'

     # ./spec/color_ls/flags_spec.rb:18:in `block (4 levels) in <top (required)>'

     # ./spec/color_ls/flags_spec.rb:18:in `block (3 levels) in <top (required)>'

Full logs are here

[chrisseaton]

with similar issues

Is this in reference to another issue?

[gogainda]

I wanted to say that all failures there are similar and the one I posted just an example

[chrisseaton]

I reduced to a simple test-case.

string = "Gl\xC3\xBCck".force_encoding(Encoding::ASCII_8BIT)
encoded = string.encode(Encoding.find('filesystem'), Encoding.default_external, invalid: :replace, undef: :replace)
p encoded.encoding
    # MRI => #<Encoding:UTF-8>
    # TruffleRuby => #<Encoding:ASCII-8BIT>

[LillianZ]

In Chris' test-case, if we remove "Encoding.default_external" we do get encoded.encoding = #Encoding:UTF-8; since "Encoding.default_external" is #Encoding:UTF-8, same as the destination encoding, the TR code skips the conversion step when destination encoding is the same as given source encoding. Unsure if ignoring user-inputted source encoding would break anything (probably?) and what the fix would be in that case.

However, even when source encoding is properly set, encoded = "Glück" in MRI and "Gl��ck", so something in Encoding::Converter also needs to be fixed.

[bjfish]

This is now fixed with the PR #2095 merged. bundle exec rake spec for colorls passed for me. Thank you @LillianZ .

[gogainda]

@LillianZ thank you