Text categorization as an information retrieval task

Abstract

A number of methods for feature reduction and feature selection in text classification and information retrieval systems are compared. These include feature sets that are constructed by Latent Semantic Indexing, 'local dictionaries' in the form of the words that score highest in frequency in positive class examples and feature sets that are constructed by relevance feedback strategies such as Rocchio's feedback algorithm or Genetic algorithms. Also, different derivations from the normal Recall and Precision performance indicators are discussed and compared. It was found that categorizers consisting of the words with highest tf .idf values scored best.