Word diff "ignores" newline and extra spaces

[matanox]

Not sure if this is a feature or part of design, but whereas a line diff will treat a new line character (\n at least) as a potential difference, word diff will not. Indeed it's not a word, and I can see the inherent abstract consistency of this treatment. But it may come to mind providing control over what characters delimit words and what characters don't, for word diff. Anyway please don't break the API, the default should behave as it does now..

Having said that, the problem is more severe than new line handling, e.g. any non-word delimiter such as a space character, is just swallowed during the word diff process, and does not show as a difference in the word diff result. I think this aspect should be reconsidered for improvement (while keeping default behavior as is).

[papandreou]

Sounds like something I needed a while ago, and luckily I sent a PR (#22 :).

Try diffWordsWithSpace.

[matanox]

@papandreou don't WordDiff and WordWithSpaceDiff just use the same tokenization?
I must have missed something there when reviewing the source yesterday, and still am.

[papandreou]

Yeah, it's a little obscure. They use the same tokenization regexp, but WordDiff is based on a Diff instance where ignoreWhitespace is set to true, which WordWithSpaceDiff leaves out:

https://github.com/kpdecker/jsdiff/blob/master/diff.js#L155-L156

The Diff constructor sets it on the instance here: https://github.com/kpdecker/jsdiff/blob/master/diff.js#L41-L43

And uses it when deciding whether to consider two all whitespace fragments equal here: https://github.com/kpdecker/jsdiff/blob/master/diff.js#L137-L143

[matanox]

Oh thanks, I ended up switching to https://github.com/cubicdaiya/node-dtl, which lets me do my own tokenization without touching the code, but it's great to know this would also work!

[matanox]

Could be nice adding this to the readme documentation..