Encoding aliases not supported when fetching webpages.

[hanubeki]

Issue Summary

Follow up for #1858 and #2015

Lemmy detects and decodes encoding when fetching articles, but not completely support encoding aliases such as Shift_JIS.
Example: https://www.itmedia.co.jp/news/spv/2212/16/news161.html

rust-encoding supports Shift_JIS as an alias of Windows-31J.
https://github.com/lifthrasiir/rust-encoding/blob/4e79c35ab6a351881a86dbff565c4db0085cc113/src/codec/japanese.rs#L454

Current implementation only checks for name but not whatwg_name which has shift_jis.

lemmy/crates/api_common/src/request.rs

Line 60 in ceff2ec

if let Some(encoding_ref) = encodings().iter().find(|e| e.name() == charset) {

So Lemmy doesn't currently decode Japanese pages which uses Shift_JIS.

Steps to Reproduce

1. Create a post with a page encoded with Shift_JIS
2. Page title and content are not decoded correctly.

Technical details

• I don't host any Lemmy instance, I report it as a Lemmy instance user.

[dessalines]

Feel free to make a PR, seems like an easy fix.

[hanubeki]

I never wrote any Rust code. That's why I opened an issue instead of a PR.

[dessalines]

Stale issue, can re-open if someone wants to take it on.

[Nutomic]

I dont think it makes sense to mark an issue like this as "stale" when it can still be reproduced and still needs to be fixed.

[Nutomic]

This appears to be fixed, the preview shows Japanese characters.