Fix single line comment lexing with CRLF line ending

[tofpie]

This Pull Request fixes #963.

It changes the following:

• Lexing of single line comment

[codecov]

Codecov Report

Merging #964 (c5d3219) into master (98e11d7) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #964      +/-   ##
==========================================
+ Coverage   59.29%   59.33%   +0.03%     
==========================================
  Files         166      166              
  Lines       10689    10687       -2     
==========================================
+ Hits         6338     6341       +3     
+ Misses       4351     4346       -5     

Impacted Files	Coverage Δ
boa/src/builtins/boolean/mod.rs	30.00% <ø> (ø)
boa/src/builtins/function/mod.rs	73.73% <ø> (ø)
boa/src/builtins/map/map_iterator.rs	71.92% <ø> (ø)
boa/src/builtins/number/conversions.rs	68.42% <ø> (ø)
boa/src/builtins/number/mod.rs	60.56% <ø> (ø)
boa/src/builtins/object/mod.rs	64.93% <ø> (ø)
boa/src/context.rs	61.26% <ø> (ø)
boa/src/environment/function_environment_record.rs	35.10% <ø> (ø)
boa/src/object/gcobject.rs	64.20% <ø> (ø)
boa/src/object/internal_methods.rs	57.14% <ø> (ø)
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 98e11d7...7281047. Read the comment docs.

[Lan2u]

Do you also need to update the multi-line comment code?

I think your first idea of making the lexer always return LF for the end of a line would be the best course as it makes lexing (very slightly) more simple (relevant if in future more tokens are added).

[tofpie]

I tried, but the lexer for single line comment uses peek() and not next_byte(), so it did not work.

boa/boa/src/syntax/lexer/comment.rs

Lines 33 to 40 in 0c068cb

	while let Some(ch) = cursor.peek()? {
	if ch == b'\n' {
	break;
	} else {
	// Consume char.
	cursor.next_byte()?.expect("Comment character vansihed");
	}
	}

I did not see any issues in the lexer for multi-line comment.

[Razican]

This looks good :) thanks

[Razican]

Hmmm surprisingly, this made 6 tests fail, but maybe that's correct.

Test262 conformance changes:

Test result	master count	PR count	difference
Total	78,460	78,460	0
Passed	18,964	18,958	-6
Ignored	15,559	15,559	0
Failed	43,937	43,943	+6
Panics	1,127	1,127	0
Conformance	24.17	24.16	-0.01%

[tofpie]

I have run the tests before and after the fix on Linux using WSL, and I have checked some of the newly failed tests. In fact, it seems that even when using Linux, some files from the test262 repository contains CRLF line endings. So, tests that were previously passed because they were considered empty are now failed because they contain not yet supported features (such as eval).
On the other side, some tests that explicitly test the single line comment with CRLF, and are now passed.