Chain of Density Prompting: A New Way to Generate Better Summaries with Generative AI

Gabriel Botsie
Mar 9, 2024
5 min read

Summarising content is a difficult task. The post explores the:

Chain of Density prompt , a new LLM summarisation technique
Original research and modified (Claude) prompts are provided
There’s a practical application, analysing a news article
A verdict, followed by recommended actions

Links to the research paper, Hugging Face repository (referenced in the paper), and a supporting YouTube video are included at the end.

What’s a Chain of Density Prompt?

Chain of Density (CoD) is a new prompting technique for generating detailed, entity-rich summaries. The chain refers to the linked steps — identifying new entities and incorporating them into the summary; leading to denser entity rich summaries.

In their research, the authors used GPT-4, the CoD prompt is below.

Article: {{ARTICLE}}
You will generate increasingly concise, entity-dense summaries of the attached Article. 
Repeat the following 2 steps 5 times. 

Step 1. Identify 1- 3 informative Entities (" ; " delimited) from the Article which are missing from the previously generated summary. 
Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. 

A Missing Entity is: 
- Relevant:to the main story.
- Specific:descriptive yet concise (5 words or fewer).
- Novel:not in the previous summary.
- Faithful:present in the Article. 
- Anywhere:located anywhere in the Article. 

Guidelines: 
- The first summary should be long (4- 5 sentences, ~80 words) yet highly non-specific, containing little information beyond the
entities marked as missing. Use overly verbose language and fillers
(e.g., "this article discusses") to reach ~80 words. 
- Make every word count: re- write the previous summary to improve flow and make space for additional entities.
- Make space with fusion, compression, and removal of uninformative phrases like " the article discusses" .
- The summaries should become highly dense and concise yet self-contained, e. g., easily understood without the Article. 
- Missing entities can appear anywhere in the new summary.
- Never drop entities from the previous summary. If space cannot be made, add fewer new entities. Remember, use the exact same number of words for each summary.

Paste the article within the Article brackets, the entity definitions and guideline instructions follow.

Modifications

As a Claude user I optimised the code, changing “Article” to pasted text. A Guardian news article — not paywalled at time of writing- was used for text.

The Claude Prompt

You will generate increasingly concise, entity-dense summaries of the attached text. 
Repeat the following 2 steps 5 times. 

Step 1. Identify 1-3 informative Entities (" ; " delimited) from the Article which are missing from the previously generated summary. 
Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. 

A Missing Entity is: 
- Relevant:to the main story.
- Specific:descriptive yet concise (5 words or fewer).
- Novel:not in the previous summary.
- Faithful:present in the Article. 
- Anywhere:located anywhere in the Article. 

Guidelines: 
- The first summary should be long (4- 5 sentences, ~80 words) yet highly non-specific, containing little information beyond the
entities marked as missing. Use overly verbose language and fillers
(e.g., "this article discusses") to reach ~80 words. 
- Make every word count: re- write the previous summary to improve flow and make space for additional entities.
- Make space with fusion, compression, and removal of uninformative phrases like " the article discusses" .
- The summaries should become highly dense and concise yet self-contained, e. g., easily understood without the Article. 
- Missing entities can appear anywhere in the new summary.
- Never drop entities from the previous summary. If space cannot be made, add fewer new entities. Remember, use the exact same number of words for each summary.

Summary 1

This article discusses fashion trend forecasting and the use of AI technology in the industry.

Entities; fashion forecasters

Summary 2

This article discusses how fashion trend forecasters observe runway shows and street fashion to predict upcoming styles. AI technology is being used to analyze large datasets and detect patterns, helping forecasters spot trends more quickly.

Entities; runway shows; AI technology

Summary 3

This article discusses how fashion trend forecasters observe runway shows to predict styles. AI technology analyzes runway images to detect prevalent colors and patterns, helping forecasters spot trends quickly.

Entities; runway shows; AI technology; colors; patterns

Summary 4

This article discusses how fashion forecasters observe runway shows, street fashion and pop culture to predict upcoming styles. AI technology analyzes runway images to detect prevalent colors, patterns and attributes to help forecast trends quickly. Retaining qualitative expertise is still crucial.

Entities; runway shows; street fashion; pop culture; AI technology; colors; patterns; attributes

Summary 5

This article discusses how fashion forecasters observe runway shows, street fashion and pop culture to predict styles. AI technology analyzes runway images to detect prevalent colors, patterns and attributes, helping forecast trends quickly. Retaining qualitative expertise is crucial, as AI lacks aesthetic perspective.

Entities; runway shows; street fashion; pop culture; AI technology; colors; patterns; attributes; aesthetic perspective

Verdict

Claude follows the prompt. It identifies entities and utilises them in the summaries (steps 1 & 2). The guidelines are followed and the summaries make sense.

There’s a counter argument that entities were missed for example:

Paris Fashion Week: feels relevant, Paris is mentioned multiple times in the article
Industry worth $2 trillion: relevance argument, points to industry worth and AI impact
Social Media: Influencers, Consumer and Social Media are all mentioned but not expressly stated in the summaries.

Updating the entities and guidelines, length of the summary will resolve this. As a school grade mark, it’s good not great — a solid “B”.

Summary

In their conclusion, the research paper authors found that “a degree of densification is preferred”. They acknowledged the limitations in only studying news article summaries and using a closed GPT-4 model where they cannot share model weights.

Without using CoD, Claude provided the below 2–3 sentence summary:

Fashion forecasters are now using AI tools to analyze large datasets and detect emerging style trends more quickly and accurately, which helps reduce waste and environmental impact. However, human creativity and qualitative insights remain essential to interpret these data-driven predictions within the social context and satisfy desires for novelty. AI and human expertise together allow for the most comprehensive fashion trend forecasts.

It uses 20 more words, but it’s perfectly adequate. The value with CoD is in the chain, the entity definitions and guidelines. Tweak these to your preference for a more specific, tailored summary.

Actions

Add this technique to your prompt library
Using the technique in a business setting-requirement for more condensed detail- feels like the best use case
Remember that entities and guidelines can be changed, so
Mix it up! refine the prompt to your needs