Only Dead Fish

Have Focus Groups Had Their Day?

"They just ensure that you don’t offend anyone, and produce bland inoffensive products." Jonathan Ive

I'll admit to having never really been fond of focus groups. For quite some time I've been troubled by the question of whether they yield quite the value that companies seem to invest in them. Or rather, I should perhaps more accurately say troubled by the question of whether the way in which focus groups are used justifies that value. Which is a different, but no less significant question.

It seems I'm not the only one. Last year Behavioural Economist and author Dan Ariely elucidated against the way in which they often seem to be used, noting what a strange idea it is that: "we take 10 people who know basically nothing about your project and you put them in the room and you let them talk for a while and then you take...whatever they came up with, as a consequence of these two hours of random thinking and you base your strategy on it, to a large degree". His point, is that too often businesses rely on small 'focus groups' to answer the really big questions.

This, I should say, I have witnessed. His suggested reason for it has a distinct ring of plausibility about it: that focus groups are extremely good at giving business folk and marketers nuggets around which they might construct believable stories. Saying “Joe, focus group 17, said this,” creates a story that is a lot more human, more filled with relatable desire, one that is more exciting than one that begins "87% of the people said X". The focus group is, therefore, "incredibly useful as a persuasive attempt to tell people what to do, but as a way to find out information, it’s not as useful as people think it is."

Similarly, Douglas Rushkoff has argued that focus groups frequently cause more difficulties than they are meant to solve, with data often cherry picked (to support a pre-formulated hypothesis) from participants who are eager to say the right thing.

The cynical part of me also questions whether the way in which we subconsciuosly shape our behaviour and account for it plays an under-recognised role in the outcome. Ariely says that whilst we are very good at explaining our behaviour, that explanation is often a carefully packaged story that has little to do with the real causes of our behaviours. Instead, our actions are often guided by the inner, primitive parts of our brains that we can't consciously access. If, for example, we are predisposed to copy others, and this is one of the inherent but often unacknowledged characteristics that influences our behaviour, do we really account for that in the way that we should?

There is (perhaps) an interesting parallel to draw here between focus groups and that other situation where people are confined to a windowless room in a relatively unnatural environment with other people and a plate of biscuits - the brainstorm. Some studies have suggested that brainstorming sessions involving groups of people might actually generate a narrower focus than if those people brainstormed individually because of a "collaborative fixation" on particular ideas due to the fact that we mirror ideas in meetings, until we all become fixated on the same thing. If this can happen in a brainstorm, might focus groups be (however subconsciously and inconspicuously) victim of the same problem?

Sure, I recognise that used in the right way, as part of the right process (and probably alongside other methodologies), focus groups may have value. And there are many (no doubt highly skilled) moderators who's job it is to avoid bias and extract real insight. But too often I think, focus groups end up (however subconsciously and inconspicuously) being used in a prophetic way to set a strategic direction as part of a process of innovation or product development and so, as Tom Fishburn puts it, afford "staggering authority to eight strangers gathered on the other side of a one-way mirror". 

Consider how feasible it is that a focus group might be used to justify a certain course of action in a product development cycle that has implications far beyond the product itself (as many product iterations do). The problem, is that groups are singularly ill-equipped to originate stand-out ideas. A focus group will understand new ideas and products within the context of their past experience. Lord knows it's pretty difficult for any of us to understand how we might react to something in the future, how we might realise when something is a breakthrough idea when it has little reference to our understanding of the world right now. I wouldn't be the first to question whether the greatest ideas in advertising, the really standout product ideas, would have happened if the companies involved had listened to focus groups. I somehow doubt it. And the trouble with incrementalism is that that way lies the looming death spiral of mediocrity. 

At the last Google Firestarters event, Tom Hulme from IDEO talked about the importance in Design Thinking of actual experience and observation in developing understanding and empathy, and the hazards of introducing ideas out of context (in a focus group for example). Designers at IDEO believe in the value of real, personal customer experience - in immersing themselves in the world of their client's customer.

When I was working at a large magazine company, there was a time when the Editors of a few different women's magazines went to live with some of their readers for a week. They stayed in their houses, ate dinner and watched telly with the family. They went out on a girls night out with the Mum and her friends, and at the weekend with her to the supermarket to do the weekly shop. Out of all the focus groups, Quant research, and reader surveys that were being done at the time, I suspect that this was likely the single most instructive thing those Editors had done to better understand their readers.

As more products and services become digitised, the opportunities for direct interaction and feedback from loyal and new customers alike are mushrooming, so whilst all of these things have to be used judiciously of-course, it strikes me that we're not short of interesting alternatives. I'm no trained researcher so this is a lay man's view. But I think it's an important question to consider. So, have focus groups had their day? Are they good or bad? Useful or useless? I'm intrigued to know what you might think.

Image courtesy

On Anxiety

This weekend I spent a bit of time with Andrew Zuckerman's work and (courtesy of Maria Popova) listened to this talk he gave at this year's 99% conference which contains a series of fantastic insights into his view on the creative process, wonderfully illustrated by soundbites from his work, most notably the Wisdom and Music projects.

Zuckerman talks about how it is the mix of curiosity and rigor that in his view gets creative projects done (he starts with Chuck Close talking about the myth of inspiration: "I always thought that inspiration is for amateurs. The rest of us just show up and get to work. If you wait around for the clouds to part and bolt of lightning to strike you...you're not gonna make an awful lot of work. But if you just get to work, something will occur to you, and something else will occur to you, and something else that you reject will put you in another direction."). And I really liked what he had to say about the importance of honesty and hard work in achieving success (which can sometimes seemingly be completely subsumed by a view of the mythical power of god-given talent IMHO).

He also talks about something which seems seldom explored at a personal level but I'd suggest is a major factor in the strangling of great ideas before they've even begun to breath: the anxiety that comes with the new. For Zuckerman, as it does for many of us I suspect, this manifests itself in the doubts that can arrive once a project has begun, the self-sabotaging voices, the anxiety about originality and finding ones true voice. I can certainly relate to that.

It's true enough that the way in which we often think about our work can be dangerous. The striving for perfection, the time we spend worrying about being the expert, and being the absolute best right from the start. When in reality prefection isn't what's required, and it is actually doing the work, and learning from it, that really matters. To quote Frank Gehry: "Your best work is your expression of yourself. You may not be the greatest at it, but when you do it, you're the only expert in it."

And then I listen to Milton Glaser (also courtesy of Maria) talking about how the only way to deal with the issue of success and failure is to discover what we're really capable of ourselves - to not only find out what we might be truly good at, but to confront the reality of what we're not so good at:

Milton Glaser – on the fear of failure. from Berghs' Exhibition '11 on Vimeo.

As Ian MacKaye of Fugazi says at the end of Zuckerman's talk: "Success is not a goal...success is in the doing, always. The question is, did you make what you were trying to make...did you try."

Never Ending Stories

I read this Frog Design piece a while back and it's stuck in my mind ever since. It's an interview with artist Jonathan Harris (he of We Feel Fine fame) about his (not yet live) new project that is about encouraging people to tell long-form stories (or integrated memories) using photos, videos, timelines, sound maps, and then connecting those stories up by automatically identifying commonalities (people, places, times, themes) and weaving them together into a kind of 'meta-story'. It is, it seems, motivated at least in part by a desire to provide a counterpoint to the apparently fragmentary nature of digital content streams.  

It's an intriguing idea and I'm sympathetic to the desire to slow down, and take time outside the constantly moving flow of content and upates. But even with the most rapidly moving flows, the river still takes you to still pools, deep waters, and little eddies that suck you in. Even the most disconnected streams still seem to be able to join up in ways that you can't foresee to become something bigger. I rather liked Marcus's excellent presentation on Character Design in the Digital Age and how he says that the internet is a "huge empathy machine connecting millions of streams to each other", and how digital characters live in these streams.

And then I come across HistoryPin, a partnership between Google and social movement We Are What We Do to build 'a digital time machine that creates a new way for the world to see and share history'. It's been up a year or so, and won a webby this year in the non-profit category, and it's a rather lovely idea (that I hadn't seen it before). The site invites people to upload and 'pin' old photos, films and audio onto a Google map and tell the stories behind them. You can make collections of pins, and create tours (you can see some of the best content here, and I like the way that local communities and schools seem to be getting involved), but it is the overlaying of these old images and films onto the modern setting via Steetview that makes it so compelling. It's a bit like Streetmuseum, that augmented reality app from the Museum of London that allowed people to hold up their phone and see a location in London as it looked in days past.

I have no idea what point I'm trying to make with this post, beyond the rather obvious observation, once again, that it is the way that stories connect together online which is almost as interesting as the stories themselves. 

Thanks to Adam for the History Pin link

Digital Curation - SXSWi 2012

The uber-smart Len Kendall, Alicia Kan, and the not-so uber smart me, have joined forces on a panel proposal for SXSWi 2012 on the subject of Digital Curation. It's subtitled "Why Value Over Volume Wins", and we're attempting to provide some useful perspective on the thorny subjects of information overload and digital content curation which I think provide some pretty fertile ground.

I've written before of my view on the future of content curation, but there can be little doubt that with the quantum shift that's taking place in the amount of information being created, this is an area which is changing rapidly and can only get more, not less, important. 

As JP Rangaswami has noted, the production, distribution and consumption of all forms of digital information have been democratised, and curation seems rapidly to be going the same way leading to a change in the very concept of "the expert, the professional, the editor, the moderator of all that is great and good". Professional curation, whilst not dying, is undoubtedly undergoing a significant degree of change to take account of new tools, and new forms of collaboration (exemplified by some of the excellent work The Guardian have done in this area).

Meanwhile, apps like Flipboard successfully combine professional with social curation, Charles Leadbetter's adage that "we are what we share" is becoming increasingly pertinent, and many of us are getting increasingly adept at creating ways to filter signal from noise through social filtering using tools such as Tweetdeck, Twitter lists, Google+ to segment groups of people and the content they share.

Meanwhile services like Noah's Percolate use a combination of social and algorithmic curation to surface relevant content and in a wider sense, the alogorithm is playing an ever increasing role in how we find and access content every day. From apps that scrobble content consumption patterns over time to refine content recommendation, to the wider personalisation through data of news, search and information leading to some pretty searching questions about the data that is being collected and on the implications of such a rise in machine driven curation. “There’s a shift in how information is flowing online,” says Eli Pariser in The Filter Bubble (it's worth watching his TED talk on the subject), “It’s invisible. And if we don’t pay attention to it, it could be a real problem.”

So, we're not short of things to talk about. You can see more about the panel here, and of-course any comments and votes that might help us get selected are greatly appreciated.

Porous Paywalls

This piece by Felix Salmon is the smartest post on paywalls I've read in a long time. In it, he talks about how the porousness of the NY Times paywall is a feature, not a bug:

"It allows anybody, anywhere, to read any NYT article they like. That makes the NYT open and inviting — and means that I continue to be very happy to link to NYT stories."

The NYT paywall of-course, does not attempt to be an impermeable barrier in the way those of the WSJ or the Times of London do. People who don't subscribe can still read a certain number of articles before the wall comes up. If you arrive via a link shared on Facebook, Twitter or a blog you won't hit a paywall. And as many quickly worked out, there seems to be plenty of none-too-complex ways to work around it.

Salmon compares it to a polite "Keep Off The Grass' notice, rather than a tall fence with razor wire on the top. It is, as Dan Gillmor puts it, more of a 'suggestion wall' than a paywall. Yet the early signs indicate that it appears to be working.

So in this, and in many other ways (as is so often the case with digital models), the NY Times paywall is counterintuitive to traditional media practice. And that's what makes it interesting.

Last year, Jeff Jarvis questioned the 'cockeyed economics of metering reading', and the logic of apparently charging (and, he suggested, sending away) "the readers who are worth the most while serving free those who are worth least". In a similar vein, typical print subscription models reward the most loyal readers with lower cover prices. Those who read the most are the most valuable, so get to pay the least.

Conventional wisdom also tells us that people who get something for free will never pay for it. So if you have something of value, and you want to charge for it, you have to isolate it and protect it at all costs. Build a big wall around it. Make it as impenetrable as possible. Then charge people to come through the gate.

Yet as Salmon points out, it seems that inspite of the ability to gain (albeit) limited free access, and (for many) the ability to work around those limitations, a large enough segment of the population seem to feel that they're getting sufficient value in NYT content that it's only proper to pay. And in being open and still allowing people access the NY Times are maximising the number of people who might feel the same.

At the other end of the scale, The Times of London paywall takes an approach that it is the invconvenience of not being able to read their content that will force you to pay. You might argue that at one level they are succeeding, but it is an unsubtle, closed door policy.

As Jeff Sonderman puts it, motivations such as convenience, duty or appreciation are more compelling than coercion, and this is especially important when talking about intangible goods, like information. He also points out that newspapers (and I would add magazines, and indeed any print publication) have always had a 'leaky payment system'. Industry research has long proven that newspapers (and magazines) have an average of more than one reader per copy.

If I was running a closed operation, my fear would be around not only whether it would work in the short-term, but what my long-term future looked like. How will people who are new to my content find it? What will the experience be like when and if they do find it and can't access it? How am I going to attract new, younger readers if they can't see what I do? How can I continue to be part of the wider debate around the issues that really matter when only a tiny proportion of the people involved in that debate can read my analysis? And when so few of them are likely to link to my analysis because they know others won't be able to read it?

A wholly closed approach removes you from the news content sharing ecosystem, and from the way in which people find content they want to consume on the web. In a recent Pew survey of more than 2,200 Americans, 75% of online news consumers said they get news forwarded through email or posts on social networking sites, and 52% said they share links to news with others via those means. I can't remember the last time I saw a Times Of London link being shared.

The results of these paywall experiments could not be more important for the future of news and of journalism. As Clay Shirky and others have long observed, the economics of ‘analog dollars to digital dimes’ (or the ratio of revenue per reader/user) seems to be not a problem, but a feature of reality. Like him, I believe the future of news is likely to be chaotic, and involve a multiplicity of business models. But like Salmon, it feels to me like the NYT model utilises more of the kind of subtle, human approach that often works best on the web. This Autumn, Shirky returns to teaching undergraduates for the first time in 12 years. I'll leave the last word to him:

"The thing I really want to impress on my students is that the commercial case for news only matters if the profits are used to subsidize reporting the public can see".

Image courtesy

Innovation Inertia

Why do some businesses just seem to stop innovating? Or at least open themselves up to being out-innovated by smart, agile competitors who are still hungry and foolish (in a good way) enough to prioritise new ideas highly?

I'm not one for Lazy End-ism, but this proclamation of the demise of Flickr makes some good points about an ongoing lack of product innovation and external focus at the site. Launched in 2004 and bought by Yahoo in 2005, the site now has over 5 billion photos on it but thinking about it, I find hard to remember the last big Flickr innovation I heard about. In fact I'm finding it hard to remember any big Flickr innovation.

Meanwhile, Facebook has overtaken it to become the world's biggest photo site, photo-sharing apps like Instagram are bringing new dimensions (and a lot of people in a short space of time) to photo-sharing, Google+ is rapidly becoming a serious potential competitor (the author draws a startling comparison between the look and feel of one his albums on Flickr (ostensibly unchanged since 2004), and a similar one on G+ which is already implementing new tools), and more questions seem to be focused around how to migrate photos to Picasa than the latest new Flickr feature.

The author points at an interesting scientific study (ironically tweeted out by Blake Irving who is in charge of Flickr at Yahoo) which concluded that when just 10% of the population holds an unshakable belief, their belief will always be adopted by the majority of the society. If the most resolved 10% have already moved on, he says, then Flickr is already dead.

It's a little early to be calling the undertakers IMO, but we've all been witness to the rapid declines of once pioneering services acquired by larger corporates (MySpace, FriendsReunited, Bebo, Dopplr). It's easy for them to get caught up the system. A system that can stifle the kind of inventive culture they need to keep innovating and stay ahead. A system that, with a short-term focus on hitting the right quarterly number, can often lead to reactive rather than smart planning. Perhaps it's right that more businesses should be run by product-driven executives.

I like Flickr. I really hope they sort it out. But there is a lesson here about innovation inertia, and the need to stay hungry, and to stay foolish.

Thanks to Paloma for the original link.

Post Of The Month - July 2011 - The Winner

An even spread of voting for much of this months vote, which then became a contest between Edward Boches' excellent Social Media Trends post and Martin Weigel's challenge to Stop Fetishising The Insight. But in the end it was the latter that won through, so well done Martin. As always, you get the props of your blogging peers and are entered into the Hall of Fame. Thanks everyone for taking part, and don't forget to bookmark your good reads for nomination in next month's vote.

Post Of The Month - July 2011 - The Vote

Thanks as always for some good nominations. So this month's vote is between:

If Your Website's Full Of Assholes, It's Your Fault by Anil Dash

Stop Fetishising The Insight from Martin Weigel

If You Want To Succeed, You've Got To Assume No-one Cares by Martin Weigel

Rigidly Defined Areas Of Doubt by Andy Whitlock

Approaches To Ideas and a Proposed Metaphor by Thomas Wagner

The Next Five Social Media Trends And Their Impact On Marketing from Edward Boches

 And you can vote below:

Playing With Ratios

Fred Wilson writes an interesting post about what he calls the common 'web/mobile laws of physics', making the observation that many of the tech services and companies they see as VCs exhibit similar ratios between the number of registered user or downloads (if it's a mobile app) the service has, the number of monthly and daily active users, and the maximum number of concurrent users (for those with a real time component). The 30/10/10 ratio runs:

• 30% of the registered users or number of downloads will use the service each month
• 10% of the registered users or number of downloads will use the service each day
• The maximum number of concurrent users of a real-time service will be 10% of the number of daily users

Fred goes on to talk about how often these ratios are incorrectly interpreted as poor performance, and how smart tech services and apps change these ratios more in their favour through judicious use of email and mobile notifications (such as Facebook photo tagging or Instagram 'Like' notifications) which can increase visit frequency and materially increase the proportion of registered users that use the service daily or monthly. It's possible to turn off these notifications on many services of-course but from personal experience, it's notable that whilst I've made efforts to avoid notification overload, I haven't turned them off on every service. There are some things it seems, that I want to be notified about.

Of-course, perhaps the best known ratio of this type is the 90-9-1 principle of participation inequality amongst online communities - the idea that (accepting slight variations due to subject matter) such communities typically have somewhere around 1% of people who create content (creators), 9% who edit or modify that content (contributors), and 90% who view the content without contributing (lurkers).

At last weeks Wikipedia conference, Jimmy Wales said he was concerned at the steady decline in the number of contributors to the site (cue sensationalist Gawker headlines about Wikipedia 'slowly dying'). With almost 90,000 active Wikimedia editors and 381 million unique users to Wikimedia projects the participation inequality ratio is (perhaps inevitably given the utility of the content) skewed lower, though in fairness the definition of an 'active' editor used is one that demands 5+ edits a month so the real number of editors is likely to be a lot higher. Still, the total number of active editors has gone down by 2.6% y-o-y.

There may be many reasons for this. Wales himself has suggested that the profile of editors is slowly changing as the priorities of the typical longstanding contributor change over time and that, given how established the site now is, there are simply fewer new entries to add. But it is the decline in the number of new editors per month (at 13% y-o-y) that is notable, which is perhaps what Wales is referring to when he says that they are not replenishing their ranks ('It is not a crisis' he says, 'but I consider it to be important'). So Wikimedia have set about addressing the problem through simplification of editorial guidelines which have become convoluted over time, providing new tools to the community, and partnerships with Universities.

I don't think for a minute that Wikipedia is slowly dying. But it's a reminder of a few things: that unique users are not created equal; of just how important a relatively small proportion of users can be to any company, service or application that relies on the regular interaction, partcipation and contribution of its users; and that, as indicators of the health of such services, ratios of these kinds matter.

Post Of The Month - July 2011 - Nominations

Slightly later than usual due to holidays, but nominations are now open for Post Of The Month. So please nominate the best posts that you've read and which were posted in the month of July in the comments below. As usual, I've listed a few starters (you may notice there's a double nomination for two excellent posts from Martin Weigel). Once I have a good selection I'll stick them all up for a vote. OK, so my starting three are:

If Your Website's Full Of Assholes, It's Your Fault by Anil Dash

Stop Fetishising The Insight from Martin Weigel

If You Want To Succeed, You've Got To Assume No-one Cares by Martin Weigel

Please add your own nominations in the comments below.