Wrong encoding detection while reading a text from the MS Word document

[xm74]

Describe the bug and add attachments

I encountered an issue with some documents of MsDoc type when parsing the structure returns text in wrong encoding.
Here is one of such documents and the code used for parsing. In this case instead of the original text Test credit card 4316020000630490 Thank you it returns 敔瑳挠敲楤⁴慣摲㐠ㄳ〶〲〰㘰〳㤴ര桔湡.
Unfortunately, Github does not allow you to attach this file as an attachment, so here is a link to download it.

Expected behavior

Returns the text in the correct encoding

Steps to reproduce

Here is the sample (simplified) code used for parsing.

$text = '';

$parser = \PhpOffice\PhpWord\IOFactory::load('/path/to/base_c1.doc', 'MsDoc');
$sections = $parser->getSections();
foreach ($sections as $section) {

	$elements = $section->getElements();
	parse_document($elements, $text);
}

function parse_document($elements, &$text) {

	foreach ($elements as $element) {

		$class = get_class($element);
		if (method_exists($class, 'getText')) {
			$text .=  $element->getText() . PHP_EOL;
		} else {
			if ($class == 'PhpOffice\PhpWord\Element\TextRun') {
				parse_document($element->getElements(), $text);
			}			
			$text .= PHP_EOL;
		}
	}
}

PHPWord version(s) where the bug happened

1.3.0

PHP version(s) where the bug happened

8.1

Priority

• I want to crowdfund the bug fix (with @algora-io) and fund a community developer.
• I want to pay the bug fix and fund a maintainer for that. (Contact @Progi1984)

[Physikbuddha]

I ran into the same issue with .doc files today.

The key issue is with how character data is read by the MsDoc.php reader. Word documents can store text in two main ways:

• Single-byte encoding (ASCII/ANSI): 1 byte per character (for Western languages)
• Double-byte encoding (UTF-16): 2 bytes per character (for Unicode support)

The issue appears with documents using single-byte encoding, while the reader always assumes double-byte encoding.

To fix the issue, the reader should detect the document encoding and use different reading strategies depending on the encoding. This information is stored in the fExtChar flag of the File Information Block (FIB) in the .doc file.

I tried hacking a quick implementation for that. I modified readRecordPlcfBtePapx() in MsDoc.php like:

// Replace this
for ($inc = 0; $inc < ($strLen * 2); ++$inc) {
    $byte = self::getInt2d($this->dataWorkDocument, $arrayRGFC[$key] + ($inc * 2));

    [...]
}

// Replace it with
$isUnicode = $this->isUnicodeDocument();
for ($inc = 0; $inc < ($strLen * ($isUnicode ? 2 : 1)); ++$inc) {
    if ($isUnicode) {
        // Read 2 bytes at a time for Unicode
        $byte = self::getInt2d($this->dataWorkDocument, $arrayRGFC[$key] + ($inc * 2));
    } else {
        // Read 1 byte at a time for ANSI/ASCII
        $byte = self::getInt1d($this->dataWorkDocument, $arrayRGFC[$key] + $inc);
    }

    [...]
}

private function isUnicodeDocument(): bool
{
    // Check the `fExtChar` flag in the FIB.
    $fibFlags = self::getInt2d($this->dataWorkDocument, 10); // wFlags offset in the FIB
    return ($fibFlags & 0x08) !== 0; // Check the 4th bit (fExtChar)
}