Frank Luntz, president of the polling and
communications firm Luntz Research, has served as an adviser to the U.S. House Republican
leadership, New York Mayor Rudolph Giuliani, and numerous candidates in this country and
abroad.
These articles appeared in the May 16 and
May 30, 1994, editions of The
Polling Report.
Voices of Victory, Part I
Focus Group Research
in American Politics
by Frank I. Luntz
Political pollsters are
drowning in numbers. Media organizations like CNN are polling on a weekly basis, if not
more. Congressional elections are six months away, yet nearly every incumbent and a good
number of challengers already have at least one telephone poll under their belt.
Why Quantitative
Research Isn’t Enough
Unfortunately, while we have all the numbers we can possibly crunch, we are severely
lacking in insight. "Not seeing the forest for the trees," as the saying goes,
most pollsters know what voters think, but too few understand how voters feel. If
understanding "why" is the objective, traditional telephone polling is
simply not enough. Qualitative research is the answer.
Quantitative research
(telephone polling) has become so sophisticated that it is possible to predict with
significant accuracy who will win most elections. But times -- and the American electorate
-- are changing.
If conventional wisdom and
national telephone polls were accurate, Ross Perot should have barely scraped into double
digits in 1992. All those Perovians should have fallen behind Bush or Clinton, or just
stayed home -- so it was said. Yet when all the ballots were counted, Perot ended up a
whisker below 20%.
And if post-election telephone
surveys, conventional wisdom, and 100 years of electoral history were any indication,
Perot’s support should have collapsed in the weeks following the election. Instead,
it expanded. Traditional quantitative polling inaccurately screened out likely Perot
voters prior to the election, and misgauged Perot voter commitment after the election.
The fact is, traditional
polling methods are increasingly ill-equipped to measure public emotions and motivations
as they exist today. Americans don’t want to respond "yes" or
"no" to alternatives that are either unacceptable or require clarification.
Asking voters to choose among fighting crime, reforming welfare, and improving health care
is an illegitimate choice for those who believe the government must accomplish all three.
If telephone polling is so
clear-cut and conclusive, why is there a tremendous discrepancy between polling firms in
their reported data on abortion? What do "pro-choice" and "pro-life"
mean anyway? Telephone polling can’t answer that question because voters themselves
can’t explain it in 30 seconds. We simply need to know more.
In today’s post-partisan
politics, there are too many shades of gray, too many "yes, but what I really think
is ..." attitudes, too many voter priorities that cannot be prioritized. With the
rise of talk radio and 24-hour television news channels, not to mention C-Span and public
access cable, there is a rapidly increasing number of semi-informed voters out there with
only half-formed political views. The elements that make up public opinion have changed;
so must its measurement.
The key to understanding why
qualitative research in general, and focus groups in particular, are so important in the
realm of today’s politics, can be summarized in a single sentence: Unlike traditional
quantitative research, focus groups are centrally concerned with understanding
attitudes rather than measuring them.
In an academic sense, the goal
of a focus group is to gain access to private, non-communicable, unconscious feelings and
emotions. In a real sense, focus group research is a direct, sensitive, and interactive
method of assessing public opinion, accomplishing what telephone studies cannot. It
approaches attitudes and priorities tangentially by allowing respondents to talk freely
and to choose descriptive categories significant to them (rather than to the
pollster, or even to the client).
A History of
Success
The focus group concept is about 50 years old, and like many modern innovations, its roots
date back to World War II. A group of sociologists were asked to investigate how the
military’s propaganda films were being received by their audiences. They learned
that, with proper prodding, people can identify the exact reason certain scenes, lines, or
phrases make them think or act in a certain way.
The consumer culture was next
to use focus group technology, turning to academically trained market researchers to
determine everything from packaging and pricing to advertising and marketing. Today,
roughly 70% of all consumer research dollars are earmarked for qualitative research, and
it is nearly impossible to find a Fortune 500 company that does not use focus groups to
develop its corporate image and/or marketing strategy.
By comparison, only 10% of all
political research is devoted to qualitative formats, and less than a fourth of all House
and Senate candidates have had any experience with the techniques. However, when they
learn how far behind the research curve they are, and what they are missing, that will
soon change.
Focus groups may have a
low-tech feel, but more often than not, it is a series of focus groups rather than
traditional telephone polling data that snatches victory from the jaws of defeat.
Historically, quantitative data has helped set themes and issues, but focus groups have
determined strategic communication and implementation. The evidence is staggering:
• It was a 1984 Georgia
focus group that gave Walter Mondale’s sputtering presidential campaign the
ammunition to fight back against a rapidly charging Gary Hart. A gaggle of Hart supporters
were brought together for the purpose of finding a weakness, any weakness, in Hart’s
political armor. At that focus group, the Mondale campaign found what they needed -- an
underlying concern about Hart’s ability to handle an international crisis. Two hours
of discussions with 15 potential voters yielded only one important finding, but it was the
magic bullet that stopped Hart dead in his tracks.
• The 1988 Republican
focus group in Paramus, N.J., has reached legendary stature, and deservedly so, for that
single gathering may have changed American history. George Bush was trailing Michael
Dukakis by double digits, with the critical target group, so-called Reagan Democrats,
trending toward the Massachusetts Governor. Assembled was just such a group, and they were
fed a litany of Dukakis negatives, from Willie Horton to Boston Harbor. Individually, the
negatives did not have a significant impact (although the prison furlough program evoked
considerable unease). However, the cumulative effect of the information provided to the
participants peeled them away from Dukakis one by one -- and made it clear to the Bush
camp exactly what had to be done to win.
• No one in political
history has had a greater commitment to focus group research than Bill Clinton. The
strategies for dealing with Gennifer Flowers, the draft dodging charge, and the other
moral challenges that faced his campaign in the primaries were developed through focus
group research. The technique has followed him into the Oval Office. In President
Clinton’s first year alone, his pollster conducted more focus groups than were
conducted in all four years of the Bush presidency.
Some of the traditional
campaign decisions -- issues, targeting and scheduling -- are not determined by
qualitative research, yet focus group results find their way into campaign strategy
through speeches, television and radio ads, and press opportunities. The reason is
obvious: qualitative research provides deep insight into behavioral and emotional
responses that you cannot capture in telephone studies.
Testing
Television Spots
The testing of television commercials is also perfectly suited to focus group research.
While firms like ours can play radio ads and the audio portion of a television commercial
directly through the telephone line, it is impossible to show respondents the visual
component through this medium.
Since the goal in any
advertising effort is to activate "mental imagery" -- the mental images that the
messages create in consumer minds -- visual analysis is far more effective than auditory
processing. Furthermore, scientific studies completed in the past decade using focus group
methodology have proven that "how it is said" and "what is heard" is
more important than "what is said." For example, focus groups have alerted media
consultants to the importance of "auditory stimuli" (i.e., background music and
sounds) to increase attention, recall, and persuasiveness.
On the other hand, focus
groups have occasional difficulty measuring the "sleeper effect," a gradual
acceptance of a particularly hostile stimulus. Americans generally have a negative
reaction to negative advertising. However, the palpability, and so the persuasiveness,
of the information often increases as the viewer sees it multiple times.
Limitations of
Focus Groups
Despite their star-studded history, the accuracy and legitimacy of qualitative research in
general, and focus groups in particular, are still raised by a small but vocal group in
the polling community. Some attacks are legitimate, but most are not.
To some extent, the problem
lies with the consumer. Candidates have never been the most sophisticated consumers of
political technology, and it took years before they were prepared to accept the
proposition that telephone studies are scientific, reliable, and valid. Qualitative
research can be just as empirical and objective, but skepticism still persists. A
perceived absence of formal structure and "hard numbers" does not make
qualitative research unscientific, but few candidates are academically trained behavioral
scientists, and they are intimidated by what they don’t understand.
Privately, political pollsters
have been known to highlight the benefits of telephone surveys at the expense of focus
groups for two simple reasons: either they are not qualified to moderate a group, or
because the profit margin for telephone surveys is much greater. Others, particularly
first and second generation media consultants, oppose focus groups because they cannot
control the outcome, and they don’t like surprises. (As Art Linkletter might quip,
"voters say the darndest things.") This is ironic. Better to be surprised by a
focus group conclusion than to have that surprise delivered on election day.
Focus groups do have their
limitations. The participants are chosen scientifically but, as a group of 10 or 12
people, the findings cannot be projected onto the entire population. The results are
dependent upon the interaction between the respondents and the moderator, and
unprofessional moderating can lead to inaccurate conclusions.
But scientifically derived
quantitative data can also misinform and mislead. In the mid-1980s, a significant number
of working class white Democrats were abandoning their party in favor of the Republican
Party of Ronald Reagan. However, telephone polls at that time suggested a growing
tolerance by these white voters toward the increasing political power of blacks. In fact,
the exact opposite was the case. It took a series of focus groups, by Democratic pollster
Stan Greenberg in suburban Detroit, to bring those telephone study errors to light.
Voices of Victory, Part II
The Makings of a
Good Focus Group
by Frank I. Luntz
A well-run focus group is a
laboratory for social interaction. A good focus group requires four simple
characteristics: the proper composition, an open environment, a probing moderator, and
in-depth analysis.
The composition of the focus
group must be selected strategically, with homogeneity as the key to a successful session.
Human behavioral studies have consistently proven that people will reveal their innermost
thoughts only to those they believe share a common bond.
For example, if your goal is
to study the real, in-depth feelings of whites and blacks toward affirmative
action, welfare, or crime, you cannot have an integrated focus group. Similarly, women
will not talk freely and emotionally about abortion if men (including a male moderator)
are present. This is just a fact of life.
The mood of the group is also
critical. A single dominant voice can cripple open, honest discussion by intimidating the
other participants. Also, keep food outside the focus group room. This has nothing to do
with the potential for a food fight. Continuing participant attention to food is an
unnecessary and ill-advised distraction.
But the single greatest
component of a successful focus group is the moderator. Academics have been justifiably
critical of many focus group practitioners because they lack one or more of the following
characteristics:
• a creative mind
• analytical skills
• verbal skills
• intellectual ability
• an eye for detail
• a tolerance for disorder
• listening skills
• a capacity for empathy
Being a "good
listener" is not enough to moderate a focus group properly. Remarkably few political
focus group moderators have been academically or professionally trained to stimulate
thorough but balanced discussion in an unbiased fashion.
Similarly, all too often,
focus group moderators put pressure on respondents to give information that they just do
not have. The fact is, voters are ill-informed about the intricate details of public
policy, and the loudest and most emotional respondent often knows the least about what he
or she is talking about. A professional focus group moderator knows how to keep such an
individual from intimidating and biasing the other participants.
Even with the
"right" participants, a good environment, and a trained moderator, the eventual
success of focus group research in developing political strategy is fully dependent on the
analysis. The question every focus group user needs to ask is: Who analyzes the
transcripts? Too often, the dialogue is poured over not by behavioral scientists or by
experts in sociology but by low level political types who know tactics but not people.
This can lead to misinterpretation of comments, false conclusions and, eventually, flawed
recommendations and strategy.
State-Of-The-Art
Qualitative Techniques
Imagine the power of being able to measure instantly and specifically the exact
reaction to a political theme, message, or messenger—second-by-second, by target
population subgroups. That power now exists.
In the post-Reagan era, most
politicians have understood the importance of harnessing verbal and visual imagery in
their effort to affect voter attitudes and opinions. Roughly one-half of President
Clinton’s annual $2 million polling budget is targeted toward communication, and it
shows with every speech and public appearance. Bill Clinton "feels your pain"
because he actually knows what your pain is.
Clinton’s not-so-secret
(and not-so-new) weapon is a technology called "instant response," which
combines the most important components of quantitative, qualitative, and in-depth public
opinion research to test message delivery, understanding, believability, and impact. A
computer-based system, the instant response technology specializes in the immediate,
second-to-second measurement of voter reaction to a speech, debate, or political
advertisement.
Here’s how it works:
Participants are gathered in a
single room for a two- to three-hour session. Each participant uses a button- or
dial-operated hand-held computer, roughly the size and weight of a small paperback book,
to relay his or her immediate reaction to a video or televised appearance. A
portable PC collects and records these responses in real time, along with demographic
information and customized quantitative close-ended opinion.
During the presentation, a
line graph is displayed continuously on a monitor adjacent to the PC. Audience reactions
are gathered literally second-by-second, enabling the pollster to determine exactly
which words, phrases, gestures, and other visuals enhance the communication effort, and
which should be altered or abandoned.
For example, George Bush used
an instant response system to test "ad-libs" prior to his 1988 debate
appearances, and Bill Clinton learned that his mannerisms and statements tested better
before a live audience than they did straight to camera.
The intensive focus groups
following the session answer the question "why" and "how," thus
providing confirmation and strategic guidance.
Focus group research is the
least financially profitable tool of the polling trade, but it may be the most powerful.
Political types may have been the last to discover its power, but nothing breeds attention
more than success. Every candidate wants to win, and as they become aware of the
qualitative option, its usage will continue to grow. Focus group influence is undeniable
even today. As media guru Roger Ailes concluded, "When I die, I want to come back
with real power. I want to come back as a member of a focus group."
Motivational Factors
Focus groups are best used to
explain "why" the public feels the way it does. A properly constructed and
administered focus group will draw out the "motivational factors" behind the
"top of mind" opinions -- which is critical to understanding what is driving
public opinion.
This figure illustrates the current motivational factors for the simple question:
"What is the most important problem facing America today?" In telephone polls,
rarely do more than 2% or 3% of those surveyed ever cite any of the three factors
themselves. However, using traditional statistical analysis and an adjusted conjoint
interviewing technique in focus group research, these three attitudes explain the
fundamental motivational factors of more than 80% of Americans.
• "Declining quality
of life" explains public concern about health care, the economy, the deficit, taxes,
and unemployment.
• "Disintegration of
morality in society" is the primary motivational factor behind concern about crime,
drugs, welfare, and immigration.
• "The break-up of
the American family" is the fear most associated with the problems in education, and
to a lesser extent, crime, drugs, and welfare.
— Frank I. Luntz
|
