Leading “Gurus” Respond to Their Critics – Including Us – in Boston Globe “Luck Inc.” Article

There was a great story in The Boston Globe on the role of luck in business success, and I’m happy to note that this research was prominently mentioned. (http://www.boston.com/bostonglobe/ideas/articles/2009/04/12/luck_inc/?page=full).

What got my attention in “Luck Inc.: The 7 secrets of really, really lucky companies” was that, for the first time I’m aware of, Tom Peters and Jim Collins engaged their critics (sort of), including us (sort of). I’d like to comment on three points in particular.

First, Peters seemed to take the view that some of the criticisms of the field research reported in success studies are missing the point. Peters was quoted in the article: “‘It's not the same kind of data that you would use to refute or confirm Einstein's theory of relativity,’ says Peters. ‘It's exploratory social science research.’”

It’s been a while since I’ve read Search cover to cover, but that level of circumspection seems to be a recent addition to the conversation surrounding this type of work. Seems to me you can’t have it both ways: making bold claims about deep causal structures, and when the models don’t appear quite as predictive as you might have hoped, invoking post hoc disclaimers.

Second, in a magnanimous defense of the underlying utility of the most popular success studies, an outspoken critic of the genre, Dr. Robert Sutton, Professor, Management Science and Engineering Faculty at the Stanford Technology Ventures Program, notes that “There's value in mastering the obvious…If Jim Collins' impact is to get people to do stuff that they know they should do already - facing the hard truths or being selfless or whatever - I certainly don't think that's a bad thing.”

It’s difficult to disagree with that observation, though to me it seems very different from what people think they are buying, and it’s certainly not what they’re told they’re getting. It’s de rigueur in this kind of work to claim that one’s findings are enormously counter-intuitive – in other words, that the advice does NOT amount to what everyone already knows they are doing (see page 10 of the 2001 edition of Good to Great, for example).

Third, in what is framed in the article as a direct response to our observations about “luck”, Collins points to unpublished work he is doing that shows that luck does not, in fact, explain the difference between the winners and losers in his model. “There is, however, a significant difference in how the winners and losers view the role of luck - and therein will lie an absolutely fascinating chapter!” he writes in an e-mail.”

It is a perfectly reasonable hypothesis that highly successful companies view luck differently, or take better advantage of their lucky breaks, or create more lucky breaks, than less successful companies. If one were able to explain how that all works, so much the better. But, as Drake Bennett of The Boston Globe rightly points out, our objection is not that luck creates success, but that it masks it. We do not claim that “successful companies were just lucky.” Rather, our view is that until you sort out the role of luck, you can’t know whether you’re looking at successful companies or merely lucky ones.

In other words, unless Collins’ new sample of successful companies can be shown to have achieved performance beyond what would have been expected due to levels of luck (good or bad) that are attributable to the system itself (our “common cause variation”), then the findings will remain highly suspect because there will be no good reason to believe that there are any differences in the underlying attributes of the two firms. We’ll remain stuck in a Kipling-esque just-so world building generalizations on the post hoc identification of proximate causes. This sort of research necessarily starts with sampling on the dependent variable, and that imposes some very exigent demands on what constitutes a proper sample.

(As an aside, Collins, in a comment that seems of a kind with Peters’ “exploratory social science” back-peddling, objects to the characterization of his work as “success studies”. According to The Boston Globe article, “it is more concerned, [Collins] argues, with discovering why companies do not become great than with why they do.” Really?!? Good to Great’s subtitle is “Why Some Companies Make the Leap…and Others Don’t.” At a minimum, that seems to seek to explain both why some companies become great and why others don’t. In addition, much of the advice seems to be “positive” (“do this to be successful”):have a hedgehog concept, first who then what, and so on. That’s the sort of thing that amounts to an explanation of why some companies do “make the leap.” It seems to me that Collins’ major works are legitimately characterized as studies of the causes of success, not its impediments.)

I don’t claim to understand Collins’ views on these issues based a couple of sentences in one article. But even so, I’m increasingly of a mind that the point we hope to make in the monograph might actually be part of a bone fide paradigm shift. That is, a change in the “rules of the game” when it comes to understanding the managerial determinants of corporate success. To date, the archetype has been Search and Good to Great, and much of what has followed in that tradition feels very much like Kuhnian “normal science”. Subsequent efforts have sought to extend the explanatory power of this basic approach and to tidy up various puzzles that remain in the wake of the defining studies. These projects buy into the basic assumptions of what constitutes a phenomenon worthy of study, the schema of concepts required to understand that phenomenon, and the rules of evidence and reasoning to be applied when drawing conclusions. For example, The Breakthrough Company is billed as Good to Great for small companies.”

Paradigms break when the anomalies pile up to the point that a new set of frameworks is needed to understand what’s going on. The approach we’re taking just might be exploring the constituents of a “new set of frameworks” for research into corporate performance. We’re working on a paper for the Ivey Business Journal (www.iveybusinessjournal.com) that will be coming out in the summer that will try to make this point. Watch for it.

Michael E. Raynor, of Deloitte Consulting LLP, is a Distinguished Fellow with Deloitte Research. In his client projects and research, he explores the challenges of corporate strategy, innovation and growth.

Comments

[James]

That didn't take long. Good show!

It's a sure sign that a pot is being well stirred when the harsh aroma of defensiveness bubbles to its surface.

It'll be interesting to work through the challenge of teasing "luck" out of the success equation. Looking at root causes, for example, does luck play a role when a company lands a top-notch innovator to lead its product development strategy?

I find this whole exploration fascinating, and consider myself lucky (and brilliant!) for having found this blog.

:)

[Bob Sutton]

Michael,

I think that you are generally on target with your point. Indeed, although I did make the comment that Good to Great and other books do help managers at times.... in part because the advice is backed by research they don't mention -- my main concern (On my blog listed below ) is that Good to Great and other books make such excessive claims about the quality of the research and the implications for what it takes to manage effectively. I think Good to Great is a mediocre exploratory study, with many of the ideas backed by other more rigorous research -- and others less so (such as claims about the importance of leadership, which are overblown). My main concern is that I would like to future advice to managers based in the weight of prior studies rather than claims of a single breakthrough study AND I think that authors and advisers need to come clean about the limits and drawbacks of their opinions. Indeed, the term I am going to start using to explain my suggestions to managers is "evidence-based opinion," as I draw heavily on the best peer reviewed studies I can find, but my advice is also shaped by my experiences, biases, and values. So, for example, given my values, there is no such thing as a great tobacco company -- but if you just used stock price as a measure of excellence, it would be possible.

Keep up the good work, you are stirring the right pot and stirring it well.

Here is my related post:

http://bobsutton.typepad.com/my_weblog/2009/04/a-wellcrafted-critique-of-business-success-books-and-my-ambivalence-about-good-to-great-.html

[Michael L. Rynne]

Phew! At last....!
That was my response when I read your article in HBR. Then I chased the link to here - and it just got better.
Fundamentally, I don't necessarily believe successful companies are only "lucky" (but then, neither do you, I suspect), but I have always had a serious problem with the thesis of Peters and Collins. Frankly, I hated the book. It had that whiff of business school evangelicism, where it was implicit that "if only you have faith" you too can have a successful company. But, as for evidence-based analysis...forget it. It's difficult for anyone brought up in the hard sciences to adapt to or accept the faith-based, lightly-once-over, fluffy work that passes as research in the management academic world. Of course it's a difficult area - there are so many variables, so much interaction. But that's a reason to be cautious, not credulous.
As for luck as the source of success? Ye..s. Frequently perhaps. I'm sure though there are nuggets of great truth buried somewhere, and yet to be found...but not in "Good to Great"!
Thanks for the refreshment.