Home ] Up ] Perot, Bush, Clinton ] Withstanding the Wave ] Rise of Robopolling ] Marriage and the Courts ] Delegates and the Democrats ] How to Forecast an Election ] Case for Online Polls ] Presidency on Life Support ] Iowa Caucuses ] 2000: Clinton's Coattails ] NAES 2000 ] Support for Clinton ] Sampling Error ] Incumbent Rule ] [ Focus Groups ] Internet Polling ] Questions About Polling ]

Frank Luntz, president of the polling and communications firm Luntz Research, has served as an adviser to the U.S. House Republican leadership, New York Mayor Rudolph Giuliani, and numerous candidates in this country and abroad.

These articles appeared in the May 16 and May 30, 1994, editions of The Polling Report.

Voices of Victory, Part I

Focus Group Research
in American Politics

by Frank I. Luntz

Political pollsters are drowning in numbers. Media organizations like CNN are polling on a weekly basis, if not more. Congressional elections are six months away, yet nearly every incumbent and a good number of challengers already have at least one telephone poll under their belt.

Why Quantitative Research Isn’t Enough
Unfortunately, while we have all the numbers we can possibly crunch, we are severely lacking in insight. "Not seeing the forest for the trees," as the saying goes, most pollsters know what voters think, but too few understand how voters feel. If understanding "why" is the objective, traditional telephone polling is simply not enough. Qualitative research is the answer.

Quantitative research (telephone polling) has become so sophisticated that it is possible to predict with significant accuracy who will win most elections. But times -- and the American electorate -- are changing.

If conventional wisdom and national telephone polls were accurate, Ross Perot should have barely scraped into double digits in 1992. All those Perovians should have fallen behind Bush or Clinton, or just stayed home -- so it was said. Yet when all the ballots were counted, Perot ended up a whisker below 20%.

And if post-election telephone surveys, conventional wisdom, and 100 years of electoral history were any indication, Perot’s support should have collapsed in the weeks following the election. Instead, it expanded. Traditional quantitative polling inaccurately screened out likely Perot voters prior to the election, and misgauged Perot voter commitment after the election.

The fact is, traditional polling methods are increasingly ill-equipped to measure public emotions and motivations as they exist today. Americans don’t want to respond "yes" or "no" to alternatives that are either unacceptable or require clarification. Asking voters to choose among fighting crime, reforming welfare, and improving health care is an illegitimate choice for those who believe the government must accomplish all three.

If telephone polling is so clear-cut and conclusive, why is there a tremendous discrepancy between polling firms in their reported data on abortion? What do "pro-choice" and "pro-life" mean anyway? Telephone polling can’t answer that question because voters themselves can’t explain it in 30 seconds. We simply need to know more.

In today’s post-partisan politics, there are too many shades of gray, too many "yes, but what I really think is ..." attitudes, too many voter priorities that cannot be prioritized. With the rise of talk radio and 24-hour television news channels, not to mention C-Span and public access cable, there is a rapidly increasing number of semi-informed voters out there with only half-formed political views. The elements that make up public opinion have changed; so must its measurement.

The key to understanding why qualitative research in general, and focus groups in particular, are so important in the realm of today’s politics, can be summarized in a single sentence: Unlike traditional quantitative research, focus groups are centrally concerned with understanding attitudes rather than measuring them.

In an academic sense, the goal of a focus group is to gain access to private, non-communicable, unconscious feelings and emotions. In a real sense, focus group research is a direct, sensitive, and interactive method of assessing public opinion, accomplishing what telephone studies cannot. It approaches attitudes and priorities tangentially by allowing respondents to talk freely and to choose descriptive categories significant to them (rather than to the pollster, or even to the client).

A History of Success
The focus group concept is about 50 years old, and like many modern innovations, its roots date back to World War II. A group of sociologists were asked to investigate how the military’s propaganda films were being received by their audiences. They learned that, with proper prodding, people can identify the exact reason certain scenes, lines, or phrases make them think or act in a certain way.

The consumer culture was next to use focus group technology, turning to academically trained market researchers to determine everything from packaging and pricing to advertising and marketing. Today, roughly 70% of all consumer research dollars are earmarked for qualitative research, and it is nearly impossible to find a Fortune 500 company that does not use focus groups to develop its corporate image and/or marketing strategy.

By comparison, only 10% of all political research is devoted to qualitative formats, and less than a fourth of all House and Senate candidates have had any experience with the techniques. However, when they learn how far behind the research curve they are, and what they are missing, that will soon change.

Focus groups may have a low-tech feel, but more often than not, it is a series of focus groups rather than traditional telephone polling data that snatches victory from the jaws of defeat. Historically, quantitative data has helped set themes and issues, but focus groups have determined strategic communication and implementation. The evidence is staggering:

• It was a 1984 Georgia focus group that gave Walter Mondale’s sputtering presidential campaign the ammunition to fight back against a rapidly charging Gary Hart. A gaggle of Hart supporters were brought together for the purpose of finding a weakness, any weakness, in Hart’s political armor. At that focus group, the Mondale campaign found what they needed -- an underlying concern about Hart’s ability to handle an international crisis. Two hours of discussions with 15 potential voters yielded only one important finding, but it was the magic bullet that stopped Hart dead in his tracks.

• The 1988 Republican focus group in Paramus, N.J., has reached legendary stature, and deservedly so, for that single gathering may have changed American history. George Bush was trailing Michael Dukakis by double digits, with the critical target group, so-called Reagan Democrats, trending toward the Massachusetts Governor. Assembled was just such a group, and they were fed a litany of Dukakis negatives, from Willie Horton to Boston Harbor. Individually, the negatives did not have a significant impact (although the prison furlough program evoked considerable unease). However, the cumulative effect of the information provided to the participants peeled them away from Dukakis one by one -- and made it clear to the Bush camp exactly what had to be done to win.

• No one in political history has had a greater commitment to focus group research than Bill Clinton. The strategies for dealing with Gennifer Flowers, the draft dodging charge, and the other moral challenges that faced his campaign in the primaries were developed through focus group research. The technique has followed him into the Oval Office. In President Clinton’s first year alone, his pollster conducted more focus groups than were conducted in all four years of the Bush presidency.

Some of the traditional campaign decisions -- issues, targeting and scheduling -- are not determined by qualitative research, yet focus group results find their way into campaign strategy through speeches, television and radio ads, and press opportunities. The reason is obvious: qualitative research provides deep insight into behavioral and emotional responses that you cannot capture in telephone studies.

Testing Television Spots
The testing of television commercials is also perfectly suited to focus group research. While firms like ours can play radio ads and the audio portion of a television commercial directly through the telephone line, it is impossible to show respondents the visual component through this medium.

Since the goal in any advertising effort is to activate "mental imagery" -- the mental images that the messages create in consumer minds -- visual analysis is far more effective than auditory processing. Furthermore, scientific studies completed in the past decade using focus group methodology have proven that "how it is said" and "what is heard" is more important than "what is said." For example, focus groups have alerted media consultants to the importance of "auditory stimuli" (i.e., background music and sounds) to increase attention, recall, and persuasiveness.

On the other hand, focus groups have occasional difficulty measuring the "sleeper effect," a gradual acceptance of a particularly hostile stimulus. Americans generally have a negative reaction to negative advertising. However, the palpability, and so the persuasiveness, of the information often increases as the viewer sees it multiple times.

Limitations of Focus Groups
Despite their star-studded history, the accuracy and legitimacy of qualitative research in general, and focus groups in particular, are still raised by a small but vocal group in the polling community. Some attacks are legitimate, but most are not.

To some extent, the problem lies with the consumer. Candidates have never been the most sophisticated consumers of political technology, and it took years before they were prepared to accept the proposition that telephone studies are scientific, reliable, and valid. Qualitative research can be just as empirical and objective, but skepticism still persists. A perceived absence of formal structure and "hard numbers" does not make qualitative research unscientific, but few candidates are academically trained behavioral scientists, and they are intimidated by what they don’t understand.

Privately, political pollsters have been known to highlight the benefits of telephone surveys at the expense of focus groups for two simple reasons: either they are not qualified to moderate a group, or because the profit margin for telephone surveys is much greater. Others, particularly first and second generation media consultants, oppose focus groups because they cannot control the outcome, and they don’t like surprises. (As Art Linkletter might quip, "voters say the darndest things.") This is ironic. Better to be surprised by a focus group conclusion than to have that surprise delivered on election day.

Focus groups do have their limitations. The participants are chosen scientifically but, as a group of 10 or 12 people, the findings cannot be projected onto the entire population. The results are dependent upon the interaction between the respondents and the moderator, and unprofessional moderating can lead to inaccurate conclusions.

But scientifically derived quantitative data can also misinform and mislead. In the mid-1980s, a significant number of working class white Democrats were abandoning their party in favor of the Republican Party of Ronald Reagan. However, telephone polls at that time suggested a growing tolerance by these white voters toward the increasing political power of blacks. In fact, the exact opposite was the case. It took a series of focus groups, by Democratic pollster Stan Greenberg in suburban Detroit, to bring those telephone study errors to light.

Voices of Victory, Part II

The Makings of a
Good Focus Group

by Frank I. Luntz

A well-run focus group is a laboratory for social interaction. A good focus group requires four simple characteristics: the proper composition, an open environment, a probing moderator, and in-depth analysis.

The composition of the focus group must be selected strategically, with homogeneity as the key to a successful session. Human behavioral studies have consistently proven that people will reveal their innermost thoughts only to those they believe share a common bond.

For example, if your goal is to study the real, in-depth feelings of whites and blacks toward affirmative action, welfare, or crime, you cannot have an integrated focus group. Similarly, women will not talk freely and emotionally about abortion if men (including a male moderator) are present. This is just a fact of life.

The mood of the group is also critical. A single dominant voice can cripple open, honest discussion by intimidating the other participants. Also, keep food outside the focus group room. This has nothing to do with the potential for a food fight. Continuing participant attention to food is an unnecessary and ill-advised distraction.

But the single greatest component of a successful focus group is the moderator. Academics have been justifiably critical of many focus group practitioners because they lack one or more of the following characteristics:

• a creative mind
• analytical skills
• verbal skills
• intellectual ability
• an eye for detail
• a tolerance for disorder
• listening skills
• a capacity for empathy

Being a "good listener" is not enough to moderate a focus group properly. Remarkably few political focus group moderators have been academically or professionally trained to stimulate thorough but balanced discussion in an unbiased fashion.

Similarly, all too often, focus group moderators put pressure on respondents to give information that they just do not have. The fact is, voters are ill-informed about the intricate details of public policy, and the loudest and most emotional respondent often knows the least about what he or she is talking about. A professional focus group moderator knows how to keep such an individual from intimidating and biasing the other participants.

Even with the "right" participants, a good environment, and a trained moderator, the eventual success of focus group research in developing political strategy is fully dependent on the analysis. The question every focus group user needs to ask is: Who analyzes the transcripts? Too often, the dialogue is poured over not by behavioral scientists or by experts in sociology but by low level political types who know tactics but not people. This can lead to misinterpretation of comments, false conclusions and, eventually, flawed recommendations and strategy.

State-Of-The-Art Qualitative Techniques
Imagine the power of being able to measure instantly and specifically the exact reaction to a political theme, message, or messenger—second-by-second, by target population subgroups. That power now exists.

In the post-Reagan era, most politicians have understood the importance of harnessing verbal and visual imagery in their effort to affect voter attitudes and opinions. Roughly one-half of President Clinton’s annual $2 million polling budget is targeted toward communication, and it shows with every speech and public appearance. Bill Clinton "feels your pain" because he actually knows what your pain is.

Clinton’s not-so-secret (and not-so-new) weapon is a technology called "instant response," which combines the most important components of quantitative, qualitative, and in-depth public opinion research to test message delivery, understanding, believability, and impact. A computer-based system, the instant response technology specializes in the immediate, second-to-second measurement of voter reaction to a speech, debate, or political advertisement.

Here’s how it works:

Participants are gathered in a single room for a two- to three-hour session. Each participant uses a button- or dial-operated hand-held computer, roughly the size and weight of a small paperback book, to relay his or her immediate reaction to a video or televised appearance. A portable PC collects and records these responses in real time, along with demographic information and customized quantitative close-ended opinion.

During the presentation, a line graph is displayed continuously on a monitor adjacent to the PC. Audience reactions are gathered literally second-by-second, enabling the pollster to determine exactly which words, phrases, gestures, and other visuals enhance the communication effort, and which should be altered or abandoned.

For example, George Bush used an instant response system to test "ad-libs" prior to his 1988 debate appearances, and Bill Clinton learned that his mannerisms and statements tested better before a live audience than they did straight to camera.

The intensive focus groups following the session answer the question "why" and "how," thus providing confirmation and strategic guidance.

Focus group research is the least financially profitable tool of the polling trade, but it may be the most powerful. Political types may have been the last to discover its power, but nothing breeds attention more than success. Every candidate wants to win, and as they become aware of the qualitative option, its usage will continue to grow. Focus group influence is undeniable even today. As media guru Roger Ailes concluded, "When I die, I want to come back with real power. I want to come back as a member of a focus group."


Motivational Factors

Focus groups are best used to explain "why" the public feels the way it does. A properly constructed and administered focus group will draw out the "motivational factors" behind the "top of mind" opinions -- which is critical to understanding what is driving public opinion.


The Real Problems

This figure illustrates the current motivational factors for the simple question: "What is the most important problem facing America today?" In telephone polls, rarely do more than 2% or 3% of those surveyed ever cite any of the three factors themselves. However, using traditional statistical analysis and an adjusted conjoint interviewing technique in focus group research, these three attitudes explain the fundamental motivational factors of more than 80% of Americans.

• "Declining quality of life" explains public concern about health care, the economy, the deficit, taxes, and unemployment.

• "Disintegration of morality in society" is the primary motivational factor behind concern about crime, drugs, welfare, and immigration.

• "The break-up of the American family" is the fear most associated with the problems in education, and to a lesser extent, crime, drugs, and welfare.

— Frank I. Luntz



. . . focus groups are centrally concerned with understanding attitudes rather than measuring them.


Copyright 2014 POLLING REPORT, INC., and polling/sponsoring organizations