What is text summarization?

Automatic text summarization (or document summarization) is a natural language processing (NLP) method that condenses information from one or more input text documents into an original output text. How much of the input text appears in the output is debated—some definitions state only 10%, others 50%.¹ Text summarization algorithms often use deep learning architectures—specifically, transformers—to parse documents and generate text summaries.

There are two principal types of summarization: extractive and abstractive.

Extractive summarization extracts unmodified sentences from the original text documents. A key difference between extractive algorithms is how they score sentence importance while reducing topical redundancy. Differences in sentence scoring determines which sentences to extract and which to retain.

Abstractive summarization generates original summaries using sentences not found in the original text documents. Such generation requires neural networks and large language models (LLMs) to produce semantically meaningful text sequences.

As one may guess, abstractive text summarization is more computationally expensive then extractive, requiring a more specialized understanding of artificial intelligence and generative systems. Of course, extractive text summarization may also utilize neural networks transformers—such as GPT, BERT, and BART—to create summaries. Nevertheless, extractive approaches do not require neural networks.²

Comparative evaluations of extractive and abstractive techniques show mixed results. For instance, while some research suggests that abstractive summarization is more prone to hallucinations—that is, misleading or factually false information.³ Additional research, however, suggests that abstractive hallucinations actually align with world knowledge, being derived from the summarization source material itself.⁴ Other comparisons of extractive and abstractive techniques show that each have their comparative benefits. While human users view abstractive summaries as more coherent, they also consider extractive summaries more informative and relevant.⁵ Research also suggests that the controversiality of text subject matter affects how users view respective summary types.⁶ Thus, there may not be a direct one-to-one evaluative comparison between these summarization types.

As with other NLP tasks, text summarization requires text data first undergo preprocessing. This includes tokenization, stopword removal, and stemming or lemmatization in order to make the dataset readable by a machine learning model. After preprocessing, all extractive text summarization methods follow three general, independent steps: representation, sentence scoring, and sentence selection.

In the representation stage, an algorithm segments and represents preprocessed text data for comparison. Many of these representations build from bag of words models, which represent text segments—such as words or sentences—as datapoints in a vector space. Large, multi-document datasets my use term frequency-inverse document frequency (TF-IDF), a variant of bag of words that weights each term to reflect its importance within a text set. Topic modeling tools such as latent semantic analysis (LSA) are another representation method that produce groups of summary keywords weighted across documents. Other algorithms, such as LexRank and TextRank, use graphs. These graph-based approaches represent sentences as nodes (or vertices) that are connected by lines according to semantic similarity scores. How do algorithms gauge semantic similarity?⁷

Sentence scoring, per its name, scores each sentence in a text according to their importance to that text. Different representations implement different scoring methods. For example, topic representation approaches score each sentence according to the degree that they individually express or combine key topics. More specifically, this may involve weighting sentences according to the co-frequency of topic keywords. Graph-based approaches, compute sentence centrality. These algorithms determine centrality using TF-IDF to calculate how far a given sentence node may be from a document’s centroid in vector space.⁸

The final general step in extractive algorithms is sentence selection. Having weighted sentences by importance, algorithms select the n most important sentences for a document or collection thereof. These sentences comprise the generated summary. But what if there is semantic and thematic overlap in these sentences? The sentence selection step aims to reduce redundancy in the final summaries. Maximal marginal relevance methods employ an iterative approach. Specifically, they recompute sentence importance scores in accordance with that sentence’s similarity to already selected sentences. Global selection methods select a subset of the most importance sentences to maximize overall importance and reduce redundancy.⁹

As this overview illustrates, extractive text summarization is ultimately a text (and most often, sentence) ranking issue. Extractive text summarization techniques rank documents and their test strings (for sample, sentences) in order or produce a summary that best matches the central topics identified in the given texts. In this way, extractive summarization may be understood as a form of information retrieval.¹⁰

As covered, abstractive text summarization techniques employ neural networks to generate original text that summarizes one or more documents. While there are numerous types of abstractive text summarization methods, literature does not use any one overarching classification system for describing these methods.¹¹ Nevertheless, it is possible to overview the general aims of these various methods.

As with many artificial intelligence applications, abstractive text summarization ultimately aims to mimic human-generated summaries. One key feature of the latter is sentence compression—humans summarize longer texts and sentences by shortening them. There are two general approaches to sentence compression: rule-based and statistical methods.

The former leverage syntactical knowledge to parse grammatical segments. These use keywords, syntactic clues, or even part-of-speech labels to extract text snippets that are then merged, often according to a pre-defined template. This template can be lifted from additional automated text analysis or user-defined rules.²

In statistical approaches, a model (whether obtained from pretraining or finetuning) learns which sentence segments to remove. For example, a tree parser may identify similar sentences from an input text and populate comparable sentences across a tree structure. A dependency tree is one such structure that models sentences according to the perceived relation between words, aligning with subject-predicate arrangements. A sentence in this structure may have the verb as its central node, with subjects and objects (that is, nouns) and conjunctions branching off. Additional verbs will then branch off from the nouns to which they are attached. Once text is represented in a tree structure, the algorithm then selects common words or phrases for use by a generative network in creating a new summary.¹²

As this brief overview of sentence compression hints, information fusion is another key aspect of abstractive summarization. People summarize documents by concatenating information from multiple passages into a single sentence or phrase.² One proposed approach to mimic this is sentence fusion across a multi-document set. This approach identifies commonly occurring phrases across a set of documents and fuses them through a technique called lattice computation to produce a grammatically coherent English summary.¹³ Another proposed method uses neural topic models to generate key terms that in turn guide summary generation. In this approach, commonly occurring keywords covering main points across multiple documents are combined into a single sentence or group thereof.¹⁴

A final concern in abstractive text summarization is the order of information. Summarized information does not necessarily follow the same order as that of the initial source document. When people write summaries, for instance, they may often organize information thematically. One method used for thematic organization is clusters. Specifically, extracted sentences are organized in clusters according to topical content (as determined by co-occurring keywords). Along these lines, neural topic models are another potential approach ordered information topically.²

Developers use a number of evaluation metrics for text summarization. Differences in metrics generally depend on the type of summary as well as which feature of the summary one wants to measure.

BLEU (bilingual evaluation understudy) is an evaluation metric commonly used in machine translation. It measures similarity between ground truth and model output for a sequence of n words, known as n-grams. In text summarization, BLEU measures how often, and to what extent, n-grams in an automatic summary overlap with those in a human-generated summary, accounting for erroneous word repetitions in the former. It then uses these precision scores for individual n-grams to calculate an overall text precision, known as the geometric mean precision. This final value is between 0 and 1, the latter indicating perfect alignment between the machine and human generated text summaries.¹⁵

ROUGE (recall-oriented understudy for gisting evaluation) is derived from BLEU specifically for evaluating summarization tasks. Like BLEU, it compares machine summaries to human-generated summaries using n-grams. But while BLEU measures machine precision, ROUGE measures machine recall. In other words, ROUGE computes the accuracy of an automatic summary according to the number of n-grams from the human-generated summarization found in the automatic summary. The ROUGE score, like BLEU, is any value between 0 and 1, the latter indicating perfect alignment between the machine and human generated text summaries.¹⁶

Note that these metrics evaluate the final summarized text output. They are distinct from the myriad sentence scoring methods used within text summarization algorithms that select suitable sentences and keywords from which to produce the final summarized output.

A number of libraries allow users to readily implement text summarization tools in Python. For instance, the HuggingFace Transformers Library comes loaded with BART, an encoder-decoder transformer architecture, for generating text summaries. OneAI’s Language Skills API also provides tools for readily generating text summaries.

Text summarization’s most obvious application is expedited research. This has potential uses for a variety of fields, such as legal, academic, and marketing. Researchers also show how text summarization transformers can advance additional tasks, however.

News News articles are a common dataset for testing and comparing text summarization techniques. Summarization is not always the end goal, however. A handful of studies investigates the role of transformer-derived text summaries as a mode of feature extraction for powering fake news detection models.¹⁷ This research shows promising potential and illustrates how text summaries can be adopted for more wide-reaching uses than merely saving time in reading multiple texts.

Translation Cross-lingual summarization is a branch of text summarization that overlaps with machine translation. Admittedly, this is not as large a research field as summarization or translation themselves. Nevertheless, the aim of summarizing a source language text or text collection in a different target language poses an array of new challenges.¹⁸ One published explores cross-lingual summarization with historical texts. In this task, historical language variants (for example, ancient Chinese versus modern Chinese, or Attic Greek to modern Greek) are treated as distinct languages. The specific experiment uses word embeddings alongside extractive and abstractive summarization and transfer learning methods to produce modern summarizations of ancient-language documents.¹⁹