Have Focus Groups Had Their Day?

"They just ensure that you don’t offend anyone, and produce bland inoffensive products." Jonathan Ive

I'll admit to having never really been fond of focus groups. For quite some time I've been troubled by the question of whether they yield quite the value that companies seem to invest in them. Or rather, I should perhaps more accurately say troubled by the question of whether the way in which focus groups are used justifies that value. Which is a different, but no less significant question.

It seems I'm not the only one. Last year Behavioural Economist and author Dan Ariely elucidated against the way in which they often seem to be used, noting what a strange idea it is that: "we take 10 people who know basically nothing about your project and you put them in the room and you let them talk for a while and then you take...whatever they came up with, as a consequence of these two hours of random thinking and you base your strategy on it, to a large degree". His point, is that too often businesses rely on small 'focus groups' to answer the really big questions.

This, I should say, I have witnessed. His suggested reason for it has a distinct ring of plausibility about it: that focus groups are extremely good at giving business folk and marketers nuggets around which they might construct believable stories. Saying “Joe, focus group 17, said this,” creates a story that is a lot more human, more filled with relatable desire, one that is more exciting than one that begins "87% of the people said X". The focus group is, therefore, "incredibly useful as a persuasive attempt to tell people what to do, but as a way to find out information, it’s not as useful as people think it is."

Similarly, Douglas Rushkoff has argued that focus groups frequently cause more difficulties than they are meant to solve, with data often cherry picked (to support a pre-formulated hypothesis) from participants who are eager to say the right thing.

The cynical part of me also questions whether the way in which we subconsciuosly shape our behaviour and account for it plays an under-recognised role in the outcome. Ariely says that whilst we are very good at explaining our behaviour, that explanation is often a carefully packaged story that has little to do with the real causes of our behaviours. Instead, our actions are often guided by the inner, primitive parts of our brains that we can't consciously access. If, for example, we are predisposed to copy others, and this is one of the inherent but often unacknowledged characteristics that influences our behaviour, do we really account for that in the way that we should?

There is (perhaps) an interesting parallel to draw here between focus groups and that other situation where people are confined to a windowless room in a relatively unnatural environment with other people and a plate of biscuits - the brainstorm. Some studies have suggested that brainstorming sessions involving groups of people might actually generate a narrower focus than if those people brainstormed individually because of a "collaborative fixation" on particular ideas due to the fact that we mirror ideas in meetings, until we all become fixated on the same thing. If this can happen in a brainstorm, might focus groups be (however subconsciously and inconspicuously) victim of the same problem?

Sure, I recognise that used in the right way, as part of the right process (and probably alongside other methodologies), focus groups may have value. And there are many (no doubt highly skilled) moderators who's job it is to avoid bias and extract real insight. But too often I think, focus groups end up (however subconsciously and inconspicuously) being used in a prophetic way to set a strategic direction as part of a process of innovation or product development and so, as Tom Fishburn puts it, afford "staggering authority to eight strangers gathered on the other side of a one-way mirror". 

Consider how feasible it is that a focus group might be used to justify a certain course of action in a product development cycle that has implications far beyond the product itself (as many product iterations do). The problem, is that groups are singularly ill-equipped to originate stand-out ideas. A focus group will understand new ideas and products within the context of their past experience. Lord knows it's pretty difficult for any of us to understand how we might react to something in the future, how we might realise when something is a breakthrough idea when it has little reference to our understanding of the world right now. I wouldn't be the first to question whether the greatest ideas in advertising, the really standout product ideas, would have happened if the companies involved had listened to focus groups. I somehow doubt it. And the trouble with incrementalism is that that way lies the looming death spiral of mediocrity. 

At the last Google Firestarters event, Tom Hulme from IDEO talked about the importance in Design Thinking of actual experience and observation in developing understanding and empathy, and the hazards of introducing ideas out of context (in a focus group for example). Designers at IDEO believe in the value of real, personal customer experience - in immersing themselves in the world of their client's customer.

When I was working at a large magazine company, there was a time when the Editors of a few different women's magazines went to live with some of their readers for a week. They stayed in their houses, ate dinner and watched telly with the family. They went out on a girls night out with the Mum and her friends, and at the weekend with her to the supermarket to do the weekly shop. Out of all the focus groups, Quant research, and reader surveys that were being done at the time, I suspect that this was likely the single most instructive thing those Editors had done to better understand their readers.

As more products and services become digitised, the opportunities for direct interaction and feedback from loyal and new customers alike are mushrooming, so whilst all of these things have to be used judiciously of-course, it strikes me that we're not short of interesting alternatives. I'm no trained researcher so this is a lay man's view. But I think it's an important question to consider. So, have focus groups had their day? Are they good or bad? Useful or useless? I'm intrigued to know what you might think.

Image courtesy

Comments

[gemma]

I don't think that groups *themselves* have had their day - but badly thought out / run ones have.

I think groups need a few caveats:

- you need to do quite a lot of them (to avoid basing the strategy on a dominant respondent leading group-think in Newcastle)

- the same person needs to plan the research, moderate the groups, do the analysis and write the report (or how can you be sure that anything has been correctly interpreted/analysed?)

- that same person should also ideally be involved in the wider project that the groups are there to influence or at the very least properly familiar with the brand and its category

- I don't think you can use groups as a forum to come up with *new stuff* (see the Henry Ford quote about a faster horse that probably wasn't his, but still...). But they're brilliant if you need to, say, refine and sense check a number of possible proposition territories or work out how far a brand could stretch into new areas.

I suppose I'm saying use groups with caution and in the right context, but don't stop using them.

[Alistair Vince]

Focus groups may well have had their day. Observation of the consumer definitely hasn't. What people use the findings for is something that has to be challenged.

I do think this ties in nicely with Martin's post of the month about insight.

We long lost belief in the focus group and that's why we set up watchmethink (we hope we're one of those 'interesting alternatives'). Giving the consumer the chance to upload video of them using everyday products and services, or create video diaries or test new products - all in their own environment seemed a lot more real. No camera crews, no people staring at them, not based on formulas. We just wanted to be useful and interesting.

An alternative.

Thanks Neil

[Dan Weingrod]

I suspect many fellow readers have seen this video from the 2007 Hatch Awards: http://www.youtube.com/watch?v=624FxhJlVM0 . It pretty much expresses my feeling about the efficacy, and efficiency, of focus groups.

The answer is, as you point out, to get out of the building among real customers and to use the metrics available from digital media as an opportunity to learn and iterate.

[Jim Williams]

I have noticed when putting together a focus group people have a tendency to think a lot about HOW and not enough about WHY. A focus group is not a solution to every problem but when used in conjunction with other research methods can provide important insight that is hard to get in other ways.

In her white paper on focus groups Carol-Ann Morgan suggests some useful guidelines for deciding when to use a focus group:

"Situations when focus groups are used are:

To unravel complex processes from the basics e.g. a complicated buying process
To identify customer needs i.e. where there is a complex interaction of factors influencing motives
To identify working practices e.g. how a particular product is used
To test new products i.e. where something needs showing to people
To explore and identify issues of satisfaction for customers, staff or suppliers
To explore perceptions of brand and service elements associated with the brand

In general, focus groups are NOT the preferred option where:

Measurement of size and distribution is required
The sample base is widespread and small
There is the need to protect the respondent from possible bias introduced by others
The topic area is sensitive e.g. requiring disclosure of production techniques or identification of customers
Respondents require preparation to answer knowledgably"

http://www.b2binternational.com/publications/white-papers/market-research-focus-group/

[Andy Crysell]

Given the ever increasing number of alternative methods available - online research communities, video based work, co-creation, smarter forms of ethnography and accompanied activity - the number of occasions in which the focus group is the best answer to the problem is diminishing and will continue to do so. There remains a perceived efficiency in them as a research method, however, particularly for clients for whom attending focus groups is a longstanding part of company culture.

Every research method suffers from a bias/need to compromise of some kind and the primary one for focus group's is of course that cramming eight strangers in a sterile space with a few bowls of Twiglets doesn't mirror real life (unless, as mentioned in the original post, you're a seasoned brainstormer!). However, when used in the right way, to explore appropriate types of subject matter, and with careful recruitment and moderation, it's nonetheless possible to create lively sessions which produce telling results.

In terms of product development, focus groups offer a suitable means to gain a read on sentiment around something comparatively familiar, such as a new FMCG, but, by comparison, would be appalling at doing the same when the product is less familiar - game changing media/technology etc. Careful, skilled research project design (and indeed making a call on whether formal research is required at all) is as vital as ever.

So, on the wane then, but not dead and buried just yet...

[Phil Adams]

The most important thing you learn every time you attend a focus group is how unimportant your client's brand and your agency's ideas are to normal people. As we're all prone to forgetting this, the reminder is probably worth the recruitment fees on its own.

It's also pretty much impossible to find a neutral researcher. They're bright and therefore opinionated. They also want to impress with their insight and thinking to secure future projects. (Don't we all?) But a lot of debriefs therefore use carefully selected and carefully emphasised research feedback in support of pet theories. Sometimes useful pet theories, sometimes not.

Emphasis is my final bugbear for this comment. You can tell that a lot of debrief presentations have been hastily written on the train by an overworked researcher. Text heavy charts that therefore tend to give equal emphasis to all findings. I always though that, were I a researcher, I'd design my debrief charts in such a way that the surface area devoted to each finding would be in direct proportion to how prominently and/or how vehemently it was expressed during the fieldwork.

[Johnnie Moore]

Agree with much of the above, but will add a couple of things from my own experience.

Back in the day, I ran lots of focus groups. What I realised was after about 4 groups I reached my conclusions and it was very hard to learn anything new from any more of them. Because my brain had figured out its narrative and wasn't going to let diversity spoil it. So much for the idea of the talented, all-seeing "trained" researcher! I suspect almost any researcher would be subject to the same flaw, though they may not want to admit it.

OFC there was a huge financial incentive to do loads of groups in the name of reliability but I think a lot of money was wasted.

The final turning point was when I found myself being recruited to a group as a participant. Normally, I'd be filtered out as a contaminated marketeer, but maybe this time they were desperate.

Sitting in the room, I was startled at the extent to which I adjusted what I said to fit in with the dynamics of the group, rather than say what I really thought. Me, this supposedly trained researcher!

Sure, they're a technique you can use to generate hypotheses, but these days there are loads of cheaper ways to do that.

[K Bordwell]

These 'is the so-and-so dead' arguments are a little bit misleading and simplistic - I agree with the comments above that focus groups have their place within the wider set of customer research methods.

I particularly agree that the same person should own the research as the analysis, and come from the school of planning where planners should (wherever possible) design, moderate, and analyse their own research. Maybe this isn't very objective, but if it's balanced with other stuff it is by far the most 'effective' because the planner has the big picture in mind, and understands what's relevant.

Research of whatever kind is only as good as the person who briefs it, the people who run it, the participants, the analysis, and so on. There's a lot of variables to get wrong - or right. Ethnographic research has its place but I've seen a lot of bollocks ethnographic research that doesn't tell you anything you don't know about what people do in their own time - it's not the panacea.

IMO it boils down to doing whatever the best thing is to get to know your audience better. If you don't do that how will you know your communications are going to work?

[Simon]

To repeat what you and many of the other commenters have said, focus groups still have their place and usefulness. But only as one of many techniques that can be used to inspire and evaluate ideas. They can't - and shouldn't - be used to produce fully formed strategies. Equally, they shouldn't be used to iterate or tamper with pre-existing (coherent) strategies.

Observational and online qualitative techniques have their uses, but focus groups can still be a valuable, cost-efficient way of speaking to an audience, and referencing their opinions against your own pre-conceptions.

[John]

I have done a Johnnie Moore in the past and infiltrated a couple of groups to see what it was like on the other side.

I can tell you that most participants are there for the money, have little loyalty to their stated "opinions" and think the whole process to be trivial at best.

[neilperkin]

Really interesting comment thread. I guess, without repeating myself, my continued cynicism around focus groups surrounds the fact that they are just so...artificial. An artificial situation, in an artificial environment, and often with an artificial context. Is it really possible to get genuine insight from that?

[Tim H]

Philip Graves has plenty to say on focus groups in his 2010 book Consumerology: The Market Research Myth, the Truth about Consumer Behaviour and the Psychology of Shopping http://j.mp/rdegAL

Philip recently spoke at a Food & Drink Innovation Network seminar on Packaging NPD. I challenged him that in the end someone has to make a decision - oh for the entrepreneurial spirit of risk set against a corporate attempt to minimise risk by covering ones ****!

I remember an article in the FT once which quoted Malcolm MP as saying focus groups were for those scared of making decisions.

[Bob Corwin]

I think Bill Hicks had the best take on this subject : http://www.youtube.com/watch?v=gDW_Hj2K0wo

[neilbazza]

We wrestle with this issue a lot at Zag, BBH's ventures division, when evaluatin new product & brand ideas.

The most fruitful approach I've been finding is to use them to try and improve and optimise concepts/product ideas rather than 'test' them. So the premise for the people is "We're doing this and we want to spend the next 90 minutes working with you on ways to make it better". This tends to lend itself to working with groups of 3 or 4 or, if there's more budget, working with larger groups initially and then picking off the most interesting people and following up with them individually.

[Judith Herron Arango]

The key with focus groups is using them for the right purpose. They can't replace quantitative research. They do however give you insights into how consumers / end-users talk about your products. The vocabulary that appears in the groups can help marketers avoid mis-communicating when constructing survey questions. The way a company talks about its products is not necessarily how consumers think about them. Not understanding that can be fatal to expensive quantitative work if you haven't done the qualitative study up front.

[Lily@idriveyourcar]

Expanding on the current focus group could make them more efficient. For example, I don't think that putting 10 random strangers into a room yields any sort of normal environment for the individuals. I would imagine not all of them fell comfortable to speak freely, are followers rather than leaders. By taking the awkwardness out of the focus group, they would be more successful.

[Mash Bonigala]

I don't think focus groups have had their day. In fact with the emergence of social media, getting access to focus groups is even easier - almost any one can be part of the focus group. I still believe focus groups, when properly approached (as suggested in this article: http://goo.gl/l1sXc) are always effective.

Of course, one should not use focus groups to flesh out seed ideas etc. Because the focus group would be unfamiliar with the idea, they may give biased opinions or may even shoot the idea down.