Headline advice is usually anecdote dressed as law. Use numbers. Open a curiosity gap. Trigger an emotion. What makes the last few years unusual is that a publisher ran millions of randomised headline tests, released the raw archive, and researchers went through it. The results are more specific than the advice, and in two places they point the other way.
Key takeaways
- Each extra negative word raised click-through by about 2.3%; each positive word cost about 1%.
- The dataset behind that: roughly 105,000 headline variants and over 370 million impressions.
- A 2025 meta-analysis of 8,977 headline A/B tests found a curvilinear, not linear, concreteness effect.
- Click-through is a selection metric. It says nothing about what the click was worth.
What 370 million impressions actually showed
The evidence base is the Upworthy Research Archive, a public record of headline experiments run by a publisher that A/B tested almost everything it published. Claire Robertson and colleagues analysed it in Nature Human Behaviour in 2023 under the title “Negativity drives online news consumption”, working across roughly 105,000 headline variations, about 5.7 million clicks and more than 370 million impressions.
For a headline of average length, each additional negative word raised the click-through rate by around 2.3%. Each additional positive word reduced it by around 1%. Because the variants were randomised against the same audience at the same moment, this is a causal estimate rather than a correlation dug out of past performance, which is what separates it from most of the headline advice in circulation.
Two caveats travel with it. The corpus is news-style content on a single platform, so the size of the effect should not be assumed to transfer to a B2B subject line or a product page. And a 2.3% relative lift on a small base is a small absolute gain: it is a tiebreaker between two acceptable headlines, not a strategy.
The curiosity gap has an optimum, not a direction
The idea underneath the curiosity gap is older than the internet. George Loewenstein set it out in Psychological Bulletin in 1994: curiosity arises when attention is drawn to a gap in one’s own knowledge, and the gap is experienced as a deprivation the person is motivated to close. Withhold the answer and you create the itch.
What the field could never settle was whether vague, gap-opening headlines actually beat headlines that summarise the article. Marianne Aubin Le Quéré and J. Nathan Matias resolved a good deal of that in Scientific Reports in 2025 by scoring headlines on a continuous concreteness scale and running a meta-analysis of 8,977 headline experiments. The relationship turned out to be curvilinear. Where the baseline headline was too vague, adding concreteness raised click-through. Where it was already concrete, adding more lowered it. Performance peaks at a middling level of information.
That reframes the practical question. It is not “should I tease or summarise”, it is “where on that scale does this particular headline currently sit”, and the answer differs by publication, section and audience. Which is a considerably less quotable rule, and a considerably more useful one.
Numbers in headlines: a habit, not a finding
Listicle headlines are the most confidently repeated rule in the category, and we could not find controlled evidence of the same quality behind them. That is worth saying plainly rather than filling the gap with a plausible-sounding statistic.
What can be argued honestly is structural. A number sets an expectation about length and format before the click, and a reader who knows they are getting eight items can decide whether they have time for it. That is a legitimate reason to use one. It is not the same claim as “numbers increase clicks”, and treating a formatting convention as a psychological law is how the advice in this field ossifies.
Running the test so the result means something
Most in-house headline testing produces numbers that cannot support the decisions taken from them. Five constraints fix most of it.
- Change one thing. A variant that alters tone, length and specificity at once cannot tell you which of the three moved the metric.
- Randomise within the same window. Comparing Tuesday’s headline against Thursday’s compares audiences and news cycles, not headlines.
- Decide the sample size before you look. Stopping a test the moment one version pulls ahead is the single most common way to manufacture a result that does not repeat.
- Carry a second metric. Scroll depth, completion, reply rate, anything that registers what happened after the click. Click-through optimised alone will drift towards headlines the article cannot honour.
- Write the result down, including the losses. A team that only records winners rebuilds the same folklore every eighteen months.
The audience question matters here too. Effect sizes measured on a general news audience in the 2010s should not be assumed to hold for readers who have since been trained to recognise and discount the format, a shift we looked at in our piece on why Gen Z ignores your ads.
Questions we get about headline testing
Does this mean I should write negative headlines?
It means negativity is a measured lever with a measured cost. The published effect is small per word, was observed on news content, and says nothing about whether a negatively framed headline suits your brand or your subject. Sectors where trust is the product have good reasons to leave that lever alone.
How many impressions do I need before a headline test is meaningful?
More than most publishers have. The findings above rest on hundreds of millions of impressions precisely because single-digit percentage effects are invisible at small volume. Below a few thousand impressions per variant, a headline test is mostly measuring noise.
Is clickbait still effective?
As a selection mechanism, parts of it demonstrably work. The problem is that click-through is the only thing it optimises, and the promise a headline makes is redeemed or broken a few seconds later on the page. Nothing in the archive data captures that second event, which is exactly why it should not be the only metric on the dashboard.
Do the same rules apply to email subject lines?
Assume not, until tested. A subject line is read by people who already opted in, alongside a sender name that carries its own history, and inbox providers weigh engagement signals in ways a news feed does not. The method transfers. The effect sizes should not be borrowed.
Working on the visual half of the same problem?
What stops a thumb in a feed follows different rules from what earns a click on a line of text.
Sources: Robertson, C. E. and colleagues (2023), “Negativity drives online news consumption”, Nature Human Behaviour, 7, 812 to 822, based on the Upworthy Research Archive of roughly 105,000 randomised headline variations, about 5.7 million clicks and more than 370 million impressions, reporting a 2.3% increase in click-through rate per additional negative word for a headline of average length and a decrease of about 1% per positive word; Aubin Le Quéré, M. and Matias, J. N. (2025), “When curiosity gaps backfire: effects of headline concreteness on information selection decisions”, Scientific Reports, meta-analysis of 8,977 headline A/B tests, reporting a curvilinear relationship between concreteness and click-through; Loewenstein, G. (1994), “The Psychology of Curiosity: A Review and Reinterpretation”, Psychological Bulletin, 116, 75 to 98, for information gap theory. Both experimental datasets come from general-interest news publishing and the reported effect sizes should not be assumed to transfer to other formats or audiences. We found no controlled evidence of comparable quality for the widely repeated claim that numerals in headlines increase clicks, and have said so rather than substituting an unsourced figure. Updated August 2026.
