Backport tests and fix for CVE-2018-3740 to 2.x branch. Resolves #187

[dometto]

See #187

I'm getting six test failures locally, but upon inspection it seems that I already get these when I checkout the 2.x branch unmodified. The failures are all of this form:

-"<b>Lorem</b> ipsum <strong>dolor</strong> sit amet script&gt;alert(\"hello world\");"
+"<b>Lorem</b> ipsum <strong>dolor</strong> sit amet &lt;script&gt;alert(\"hello world\");"

That is, there is an unexpected < before "script". We'll see what happens on Travis.

The newly backported tests in test_malicious_html.rb and test_clean_element.rb all pass, though.

[dometto]

It seems at this point it is already possible that attr has been unlinked, but the do loop continues with the same attr. Should the new code somehow be made conditional on whether attr has been unlinked yet, for performance reasons? (No use doing the strip and the gsub if attr is already sanitized, I take it?)

[dometto]

Getting the same failures on Travis. As I said I'm getting the same failures on a fresh checkout of 2.x, so I don't think we should let this be a showstopper -- but of course it would be nice to get rid of the failures.

On a closer look, there seems to be one additional failure on this branch than on the 2.x branch:

 3) Failure:
Config::RELAXED#test_0020_should not allow protocol-based JS injection: spaces and entities [/home/travis/build/rgrove/sanitize/test/test_sanitize.rb:276]:
Expected: "<img src=\"\">"
  Actual: "<img src>"

EDIT: just changing the expected result is not acceptable because <img src> is (I assume) not valid HTML.

[dometto]

It's the attr.value = attr.value.strip line that's causing the single additional failure. In this test, before the strip call, attr is this:

#<Nokogiri::XML::Attr:0x3fcaf1c1bd98 name="src" value=" ">

After the strip call, it becomes:

#<Nokogiri::XML::Attr:0x3fcc0417ee30 name="src">

So it seems that in nokogiri, setting value to "" causes value to be removed from the Attr entirely.

I am now omitting the strip if attr.value.strip.empty?. After doing that, the actual output from this test becomes "<img src=\"%20\">", and the test still fails. So I have further omitted the gsub in case attr.value.strip.empty?, which passes the test. @rgrove: is this an acceptable way of dealing with the edge case?

The five remaining failures are those that are already present on vanilla 2.x.

[rgrove]

Thanks!

The test failures on the 2.x branch are most likely due to changes in the output of newer versions of Nokogiri that didn’t exist when the tests were written. As long as the output produced by modern Nokogiri is safe and sane, it should be fine to update the expected output to match Nokogiri’s current output.

EDIT: just changing the expected result is not acceptable because <img src> is (I assume) not valid HTML.

<img src> isn’t technically valid, but it is safe, and safety is more important than validity. I would prefer this output over skipping stripping/escaping on space-only values. It would also be fine to remove the empty src attribute, since it’s useless (more on this in my diff comments).

[dometto]

Thanks for the comments, @rgrove! Will get to work on them.

[dometto]

Green lights. Please have a look at the new implementation in clean_element.rb.

I took the liberty of bumping version and updating HISTORY for your convenience (though do change the release date if you don't get round to cutting the gem today).

[rgrove]

Thanks @dometto! I've pushed 2.1.1 to RubyGems.