Our last post : #1333

It turned out to be necessary to add "utf8mb3: 45".

I don’t know if this should be added to the main branch ...

P.S.

MySQL server version: 8.0.26-0ubuntu0.20.04.3

Hi @egorovd - can you post a small self contained example to reproduce that?
Do you see this in any mysql 8 server or is it specific to version 8.0.26?

ok, so if I understand correctly you passed utf8mb3 as encoding name in connection options and got Error: Encoding not recognised: 'undefined' error. Two issues here we need to fix: 1) add utf8mb3 to the list of known encodings ( I need to double check if iconv-lite supports it and what is corresponding iconv-lite encoding name we should use ) and 2) error is not very helpful, ideally we need to communicate both what happened and how to fix that. I'll close #1333 as its a full duplicate of that one and keep this issue to track both 1) and 2) fixes

I encountered this today and It turned out to be necessary to add "utf8mb3: 45".

mysql> show variables like "character_set_%";
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | utf8mb3 |
| character_set_connection | utf8mb3 |
| character_set_database | utf8mb3 |
| character_set_filesystem | binary |
| character_set_results | utf8mb3 |
| character_set_server | utf8mb3 |
| character_set_system | utf8mb3 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.01 sec)

Can this support be generally added by default?

@snxraven can you try to run https://github.com/sidorares/node-mysql2/blob/master/tools/generate-charset-mapping.js on your database? Yes, likely we want to add utf8mb3: 45, line here:

This is the requested output @sidorares

https://pastebin.com/p2kqqDxY

could you filter the first array to only include items where Charset starts with "utf8" ? console.log clips output with ... 172 more items

@sidorares This output may be more useful
https://pastebin.com/L4EnRwJD

hm, can't see a charset with utf8mb3 name

https://dev.mysql.com/doc/refman/8.0/en/charset-unicode-utf8mb3.html

need to confirm which iconv encoding should we map utf8mb3

Yeah, thats odd, when sorting just for "utf8mb3" nothing does come up.

I've encountered this issue with a MySQL server that is configured to use utf8 charset (not utf8mb4) for historical reason.

My workaround is to add an entry utf8mb3: 45 to node_modules/mysql2/lib/constants/encoding_charset.js.

Note it seems .NET official client made a change to fix a similar issue: Bug #33556024 - Connector/Net doesn't recognize utf8mb3 · mysql/mysql-connector-net@16ad749, implying clients are expected to support utf8mb3.

internally we map mysql utf8 to iconv's cesu8, probably need to do the same for utf8mb3

If thats a correct mapping, extra line utf8mb3: 'cesu8', is needed here:

After that regenerate https://github.com/sidorares/node-mysql2/blob/master/lib/constants/charset_encodings.js

I guess charset_encodings.js can be unchanged because the inversion mapping (45 -> utf8) is already there.

It seems system variable character_set_client is set for some reason in my case. Small repro is:

    await conn.execute("SET @@character_set_client = 'utf8mb3'")
    await conn.execute("SELECT 'something'") // throws

Self-contained version is: repro-mysql-encoding.zip

• Environments
  • MySQL 8.0.32
  • Node 18.15.0
  • mysql2 3.2.0
• Notes
  • This line did the lookup EncodingToCharset['utf8mb3'] (evaluating to undefined): https://github.com/sidorares/node-mysql2/blob/v3.2.0/lib/packets/resultset_header.js#L65
  • Adding server configurations didn't make a repro; such as [mysqld] character_set_server = utf8mb3, [mysql] default_character_set = utf8mb3.

In case that someone runs into the same problem and I can save his time with this post...

I requested my database from our companies Database team and gave them an sql file to import for the database structure (In our company you are not able to be database admin. Its all done by a special database team). In the sql file I explicitly set all character sets to utf8mb4.

But when they created the database, I run in the same error:

2024-06-12T08:46:56.989Z error: Encoding not recognized: 'undefined' (searched as: 'undefined')

It took me hours to find out, that they have created my database with utf8mb4 but they created the information_schema in utf8mb3. (Who will think of that?)

Adding utf8mb3 in the encoding file worked. But of course I have requested DB team to change charset of information_schema to utf8mb4.

Is there any problem adding utf8mb3 to the encoding file per default for you lib?

In the short-term (prior to fixing our database's default charset) we've worked around this by monkey patching the utf8mb3 mapping into the library at runtime like this: iFixit/pulldasher@b4f26d8