Improve tokenization policy

[manuyavuz]

As discussed in CocoaPods/Core#265, we can improve tokenizer while generating inverted index for search.

Current policy performs a naive and simple split approach which splits texts from whitespaces using String#split method. As suggested by @floere, we may implement a better tokenizer approach which may handle some typos in texts or clearing special characters like ! and . while preserving term specific usages like - character in Objective-C.

This needs to be evaluated in detail, opening issue as a reminder.

We may tag this as enhancement if appropriate.