A powerful approach to analyze message testing data and avoid the pitfalls of traditional analytic techniques.

Helping campaigns test and craft messages is one of the principle responsibilities of a pollster.  Unfortunately, many pollsters use message testing techniques that rely entirely on self-reporting by respondents of how they would respond to messages.  This approach causes two important problems that can cause campaigns to adopt misguided messaging strategies:

1)      Social-desirability bias can cause respondents to over-report the effectiveness of messages that they think they should respond to and under-report the effectiveness of messages they believe they should not respond to.

2)     Because this self-reported approach does not include an importance measure it can lead to overestimating the effectiveness of messages that respondents agree with strongly but which do not really play a role in determining their vote.

Because of these concerns with the traditional approach to message testing, WRS has developed its Message Mapping ™ analytical tool. 

 

Message Mapping â„¢ uses the same message testing sequence that traditional polling uses with two key differences:

1)     We construct multiple sub-samples each of which hear different combinations of messages.  This allows us to mirror the way that researchers test medicines, by observing the difference in response (ballot movement) between those who heard a particular message and those who do not.

2)     By asking a structured recall question after the messaging we can evaluate which messages are the most memorable.

 

Analysis of the data set from the survey allows us to test each message on three attributes:

 

Effectiveness

• Effectiveness is measured by the simplest of metrics and answers the most important question: does this message alter voter behavior?
• Effectiveness of messages is tested exactly the way that other researchers test the effectiveness of medicines: by measuring the difference in response (in this case ballot movement) between those who heard a message and those who did not.

Stickiness

• Stickiness is a cutting edge variable designed to test how successful a message is at punching through a crowded media environment, grabbing a voters attention, and remaining with that voter.
•  In today’s complex media environment, voters, especially approaching an election cycle, are bombarded with information.  Making sure that a campaign is spending money on messages that voters will notice during a crowded commercial break or when sorting through many pieces of mail is the difference between a message that works in the controlled environment of a survey and one that really helps win a campaign.

Believability

• Believability is a critical, but sadly under-used, element of any message.  It doesn’t matter how much voters will agree with your message, if they don’t believe it, it won’t move them in the ballot box.

By analyzing all three of these dimensions together we can understand which messages will really help change behavior and help you win a campaign.  Below is an example of a Message Map™.

 

In this message map we see two messages which score very well on effectiveness—health care and an energy message. 

 

These are clear “winners� among the messages.  Immediate tax relief scores almost as well as those two on the effectiveness scale but is not as memorable and also may not be as believable (voters don’t think it will happen). 

 

Medical care for veterans is a classic “trap� message that would show up well in a traditional message analysis—it is memorable and voters will say they like it, but they just don’t care enough about it to respond to it on the ballot.

 

At WRS we remain committed every day to developing and improving tools that give our candidates and clients every advantage to win their campaigns by leveraging the latest developments in behavioral science and statistical analysis.  Message Mapping ™ is one of many ways we do so.