Unbaking Japanese

[wimglenn]

>>> import ftfy
>>> s = "Ä×èÈÄÄî▒è¤ô_üiâAâjâüâpâXüj_10òb.png"
>>> ftfy.ftfy(s)
'Ä×èÈÄÄî▒è¤ô_üiâAâjâüâpâXüj_10òb.png'
>>> s.encode('cp850').decode('shift-jis')
'時間試験観点（アニメパス）_10秒.png'

Possible? Thanks

[rspeer]

Neither Shift-JIS nor codepage 850 are encodings that ftfy currently tries.

I can see the reason to want to support Shift-JIS; it's pretty much the reason for the word "mojibake", after all. It's something I want to support but I haven't found a good heuristic for distinguishing it from EUC-JP. It even coincidentally sometimes looks like UTF-8.

[rspeer]

This is something I want to make another attempt at. I think that understanding the nuances of Shift-JIS mojibake is going to require talking to someone who spends a lot of time on the Japanese-language Internet though

[wimglenn]

Good luck. I am not that person. I nominate @bobince.