Text Compression Techniques

Abstract

The benefits associated with text compression include more efficient use of peripheral devices. faster information transfer rates and, in some cases, improved sorting speeds through the use of shorter sort keys. However. these advantages must be balanced against a slight increase in CPU-time and the extra storage required for the associated code tables. Information theory and statistics of English provide a background for the discussion of various text compression algorithms. The common objective of the methods described is to reduce the physical size of the text file while maintaining a complete representation of the information (reversible compression). The methods can be divided into two main classes: ( a) those that re-define the symbol codes to more accurate reflect the information content of each symbol, and (b) those that use special codes to represent commonly occurring groups of symbols, thereby reducing redundancy due to mutual information between symbols. Compression techniques not covered include those dealing with data files and with telemetry. Finally. another approach to compression is described. which also holds some promise for automatic indexing and simpler inverted file design in document retrieval systems.