BOM as whitespace test case

[subzey]

Byte order mark (zero-width non-breaking space) chars should be trimmed in tags

[dasilvacontin]

We should add browser testing in order to check this.

[dasilvacontin]

Thanks @subzey!

[bobthecow]

Why?

[dasilvacontin]

I think we should treat BOM (aka zero width no-break space) as whitespace. After what I've read about it so far it looks like it's the logical decision and it's also the trend.

Besides, it's something we were already supporting: !!'\ufeff'.match(/\s/) is true for node v0.10.33, iojs v2.0.1 and Chrome Version 42.0.2311.135 (64-bit) (at least it works on these versions, which are the ones I have currently installed). Therefore, a fixed version of this test passes on the Travis build: #448.

Related: https://code.google.com/p/v8/issues/detail?id=3109

[dasilvacontin]

What are/were your thoughts, @bobthecow?

[bobthecow]

My thoughts are that:

1. There are a ton of (read: 25) "whitespace" characters in Unicode, and so far, none of 'em are special in mustache.
2. No other mustache implementation (to my knowledge) treats anything besides space, tab, and the various newline/carriage return characters as the kind of "whitespace" that's allowed inside tags.

So this makes Mustache.js inconsistent, for (as far as i can tell) little to no benefit.

[dasilvacontin]

1. There are a ton of (read: 25) "whitespace" characters in Unicode, and so far, none of 'em are special in mustache.

We are not treating BOM as special right now, it just works due to the /\s/ definition in js.

1. No other mustache implementation (to my knowledge) treats anything besides space, tab, and the various newline/carriage return characters as the kind of "whitespace" that's allowed inside tags.

So this makes Mustache.js inconsistent, for (as far as i can tell) little to no benefit.

That's something I didn't know. But then, if desirable, we should work into creating a cross-language definition for 'whitespace' that we'd use in Mustache, because there are probably other whitespacish characters that are treated differently.