Brainstorming

A basic description of the application can be found at: http://goo.gl/jscwZ

As described in the link above, HC differ from usual search engine by grouping related searches, promoting content over the source, using relevance feedback to refine queries and producing structured documentation.

Below are some interesting algorithms, concepts and other resources that could be useful.

Concepts & Techniques

• http://en.wikipedia.org/wiki/Precision_and_recall
• http://en.wikipedia.org/wiki/Focused_crawler
• http://en.wikipedia.org/wiki/URL_normalization
• http://en.wikipedia.org/wiki/Locality_sensitive_hashing
• http://en.wikipedia.org/wiki/Treebank

Algorithms

• http://en.wikipedia.org/wiki/MinHash
• http://en.wikipedia.org/wiki/Rocchio_Classification

Data

• The Web!!!
• http://en.wikipedia.org/wiki/Text_corpus
• Duck Duck Go A search engine with an easy API. It already somewhat organizes data into topics http://duckduckgo.com/?q=information+retrieval and categories http://duckduckgo.com/c/Information_retrieval

Libraries

NLP

• http://gate.ac.uk/
• http://alias-i.com/lingpipe/
• http://incubator.apache.org/opennlp/
  • https://github.com/dakrone/clojure-opennlp
• http://mallet.cs.umass.edu/
• http://uima.apache.org/