Masses vs. Margins: Differences in National and Battleground Persuasion Testing
Are voters in presidential battleground states persuaded by different messages than voters nationally? To what extent is a national audience a suitable proxy for a battleground audience? Grow Progress conducted two Rapid Message Tests in late June to answer these common questions.
Both tests – one among a national audience of 3,000 people and one among a presidential battleground state audience of 3,000 people – evaluated the effectiveness of two pieces of content (Burn and Terminate) against a placebo (a Morton Salt ad). After viewing one of the three ads, all respondents answered the same three questions:
- Vote likelihood
- Presidential vote choice
- Joe Biden favorability
Key Findings
Neither Burn nor Terminate affected respondents’ likelihood to vote.
Both ads effectively moved vote choice toward President Biden, but the magnitude of effect differed by audience. In both cases, the ads were more effective at moving vote choice toward Biden nationally than in battleground states. The difference here is meaningful. A strategist deciding how much money to put behind an ad in a battleground state might overestimate its effectiveness in that geography based on national results.
The differences between audiences are even more stark when it comes to improving President Biden’s favorability ratings. The national test saw +3pp directional movement on the Burn message and +6pp statistically significant movement on the Terminate message. In the battleground test, however, neither of the ads produced a measurable effect. The same battleground state strategist would make very different decisions based on the testing audience.
Subgroup results also demonstrate differences in the national test versus the battleground test. For example, on vote choice, both messages had no measurable effect amongst respondents ages 18-34 in the national audience, but had positive persuasive effects among the same age group in the battleground test.
Detailed Results
Persuasive Effects: For the national test, there was statistically significant or directionally positive movement across vote choice and Biden favorability for both messages:
However, when looking at the same exact test design but amongst a battleground state audience, there is no measurable effect on Biden favorability:
When looking at subgroup results, there are effect size differences between the two audiences by key demographic groups. For example, on vote choice Burn moved Black adults toward Biden by 10 points in the national test but away from Biden by one point in the battleground test. Similarly, Burn moved Latino adults toward Biden by three points nationally and by 13 points in battleground states.
Subgroup results are always more variable given the smaller sample sizes, but in this case, the large overall sample size is sufficient to measure differences, especially at this magnitude.
Takeaway
It’s important to test your messages and creative content among the audience you want to persuade. Sometimes testing among a proxy audience is acceptable, such as when it is necessary for budget, speed, or feasibility. In those instances, it is critical to understand the limitations of conclusions you can draw.
