Case Study: Flavored Tobacco Ban Ballot Measure
Summary: A message highlighting the tobacco companies that are funding the campaign to oppose California’s Proposition 31 was highly effective at increasing support for the measure.
Research outcomes and key findings:
We used Grow Progress’s Rapid Message Testing tool to evaluate the effects of Facebook posts in support of California’s Prop 31, which would ban the sale of flavored tobacco products in the state.
All four messages were highly effective at increasing support for the measure. The Rapid Message Testing Tool’s “Change Results Display” feature helped identify that “Funders” was the clear winner among the four.
Overnight delivery: This test was launched at 4:15 pm on Wednesday, October 5, and returned results at 9:49 am the next morning.
Affordability: For this test, we doubled the sample size for more precision and the ability to confidently observe subgroup effects. We evaluated four distinct image messages. This test cost 20 credits to run — about $4,000 for most customers.
Question
What works to change minds on California’s flavored tobacco sales ban?
The Yes on 31: Protect California Kids campaign has shared a series of Facebook posts with a wide range of arguments in support of the ballot proposition. We tested messages that highlighted a personal story of loss, the LA Times endorsement of the measure, flavored tobacco’s effect on youth, and the funders of the No side of the measure. The test was designed simply to observe the effects of the messages in changing vote choice on the measure.
What worked?
All messages significantly increased support for the ballot measure, with effects ranging from 14 to 18 percentage points. “Funders” appeared to be the most effective, increasing the “Yes” vote by 18 percentage points.
So should the campaign choose “Funders”? The other messages were nearly as persuasive, at first glance. Further analysis, below, shows that “Funders” is the strongest message.
The Rapid Message Testing tool allows users to instantly re-analyze the data to only show results for a particular subgroup, or to display movement toward or away from answer options other than “Yes.” In this case, we wanted to also look at which message was the most effective at reducing the “No” vote.
We can see that “Funders” and “La Times” both reduced opposition to the measure by 10 percentage points. So, the “net effect” of “Funders” was clearly the largest, increasing net support by 28 points.
Next steps
When you have a message that increases net support by 28 points in testing, you should just get it in front of as many people as possible! Further analysis showed that “Funders” worked among every subgroup, and should be highly effective at changing minds.
If the campaign were to widely deploy this message, it could make sense to test again once the message has had the chance to “burn in” — after most people have been exposed to the message, it’s possible that another message with information that’s new to the electorate would become more effective.
Appendix: Messages
Funders
La Times
Personal
Youth
