March 20, 2019

[buskamuza]

Please add your topic as a comment to the issue. Use following format:
Topic description and link to PR, if any (duration in min)

Meeting Notes

See below.

🎥 Recording

[lenaorobei]

Is there any cases when object instantiation via new keyword is allowed in Magento? magento/magento-coding-standard#64 (5 min)

[kandy]

Issue: \Magento\Framework\Serialize\Serializer\Json is not binary safe
see example

<?php
$examples = array(
    'Valid ASCII' => "a",
    'Valid 2 Octet Sequence' => "\xc3\xb1",
    'Invalid 2 Octet Sequence' => "\xc3\x28",
    'Invalid Sequence Identifier' => "\xa0\xa1",
    'Valid 3 Octet Sequence' => "\xe2\x82\xa1",
    'Invalid 3 Octet Sequence (in 2nd Octet)' => "\xe2\x28\xa1",
    'Invalid 3 Octet Sequence (in 3rd Octet)' => "\xe2\x82\x28",
    'Valid 4 Octet Sequence' => "\xf0\x90\x8c\xbc",
    'Invalid 4 Octet Sequence (in 2nd Octet)' => "\xf0\x28\x8c\xbc",
    'Invalid 4 Octet Sequence (in 3rd Octet)' => "\xf0\x90\x28\xbc",
    'Invalid 4 Octet Sequence (in 4th Octet)' => "\xf0\x28\x8c\x28",
    'Valid 5 Octet Sequence (but not Unicode!)' => "\xf8\xa1\xa1\xa1\xa1",
    'Valid 6 Octet Sequence (but not Unicode!)' => "\xfc\xa1\xa1\xa1\xa1\xa1",
);

foreach ($examples as $str) {

    if (json_encode($str) === false) {
        echo "Cannot json encode str :[$str] - Error: " . json_last_error_msg() ."\n";
    }
}

will output:

annot json encode str :[�(] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[��] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[�(�] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[�(] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[�(��] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[�(�] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[�(�(] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[�����] - Error: Malformed UTF-8 characters, possibly incorrectly encoded
Cannot json encode str :[������] - Error: Malformed UTF-8 characters, possibly incorrectly encoded

[buskamuza]

@kandy , for the encoding:

• create an item in the backlog and prioritize with PO
• possible options:
  • use serialization/unserialization w/ no objects allowed
  • use base64. In this case we need to traverse through arrays, it's more complicated and slow
  • discuss with security team
  • discuss with @piotrekkaminski

[buskamuza]

\Magento\Framework\Encryption\Encryptor::encrypt() uses base64 which increases data being stored, for example, in cache.
@kandy:

• remove base64 from encrypt
• make encoding binary safe (see above discussion). E.g., use serialization instead for caching scenario

@buskamuza :

• it's a breaking change
• first, let's resolve the encoding issue

[piotrekkaminski]

@buskamuza @kandy strongly disagree on adding serialization. We had tons of issues in the past and even right now with unserialize leading to RCE. Why is base64 a problem? Is it a measurable performance impact? @szurek ??

[kandy]

@piotrekkaminski see related ticket magento/magento2#21334

[piotrekkaminski]

Also - what are the use cases for binary data (as unicode seems to work ok)?

[piotrekkaminski]

@kandy they mention encryption in the ticket as the culprit. How it relates to serialization/json encoding?

[kandy]

@piotrekkaminski The encryption produces a binary data that cannot be serialized, as a result, we add base64 encoding on top of encryption that increase traffic between servers and cause performance degradation