The Talent

November 19, 2010 11:30 AM

Lies, Damn Lies, and Statistics

Posted by Ed Shanahan

By Steven Harper

ALM editor-in-chief Aric Press recently wrote about Indiana Law Professor Bill Henderson's new for-profit venture on recruiting, retention, and promotion. The Wall Street Journal's Law Blog and ABA Journal covered it, too.

I was troubled after reading about Henderson's venture.

The first thing that came to mind was the venerable computer programming maxim, "garbage in, garbage out." That's because Henderson is asking Am Law 200 partners to identify values and traits they want in their lawyers. And he's assuming they'll tell him the truth. But will they admit to seeking bright, ambitious associates wearing blinders in pursuit of elusive equity partnerships typically awarded to fewer than 10 percent of large firm entering classes? Or that such low "success" rates are inherent in the predominant large law firm business model that requires attrition and limits equity entrants to preserve staggering profits?

Then I considered Mark Twain's reflections on the three kinds of falsehoods: "Lies, damn lies, and statistics." It came to mind because Henderson's researchers, as Press wrote, "pour over the resumes and evaluations of associates and partners trying to identify characteristics shared by those who have become 'franchise players' and those who haven't." Here's what those resumes and evaluations won't reveal: the internal politics driving decisions.

Most equity partners are talented, but equally deserving candidates fail to advance for reasons unrelated to their abilities. Rather, as the business model incentivizes senior partners to hoard billings that justify personal economic positions, those at the top wield power that makes or breaks young careers--and everybody knows it. Doing a superior job is important, but working for the "right" people is outcome determinative. Merit sometimes loses out to idiosyncrasy that is impervious to Henderson's data collection methods.

But perhaps the biggest problem with Henderson's plan is it's goal: identifying factors correlating with individual success. Does the magic formula include, as Press writes, "a few years in the military, a few years in the job force, or a few years as a law review editor?"

If managers warm to Henderson's conclusions (after paying his company to develop them), they'll leap from correlation to causation, develop checklists of supposed characteristics common to superstars like themselves, and hire accordingly. Law schools pandering to the sliver of the profession that large firms occupy (it's less than 15 percent of all attorneys) could take such criteria even more seriously. Before long, prospective students will incorporate the acquisition of "success" credentials into their life plans.

The difficulty is that today's law firm partners already favor like-minded proteges. That tendency inhibits diversity as typically measured--gender, race, ethnicity, sexual orientation, and the like--along with equally important diversity of views and a willingness to entertain them. Even today, concerned insiders are reluctant to voice dissent from big law's prevailing raison d'etre--maximizing short-term profits at the expense competing professional values and longer-term institutional vitality. Won't meaningful diversity--of backgrounds, life experiences, and resulting attitudes about professional mission--suffer as groupthink makes firms even more insular? Meanwhile, trying to improve overall "success" rates is a futile goal; they won't budge until the leveraged pyramid disappears.

I don't fault Henderson, who bypassed a large firm practice for academia after his 2001 graduation, for his efforts. And I've accepted his invitation to take a closer look at what he's doing. But it's important to heed Press's warning: "To some extent, it doesn't matter what Henderson and Co. discover. What matters is that the inquiries have begun. If we've learned anything from the last decade, it's that we can't predict the consequences of new information beyond acknowledging its power to disrupt."

Consider two unfortunate examples. The flawed methodology behind U.S. News' law school rankings hasn't deterred most students from blindly choosing the highest-rated one that accepts them. (Exorbitant tuition and limited job prospects may be changing that.) Likewise, most of big law remains attached to equity partner profits as the principal (and sometimes exclusive) measure of a firm's success.

The most important things that happened to me--in work and in life--were fortuitous. No statistical model could have predicted them. Still, I hope Henderson's study answers an important question: Would his likely-to-succeed factors have led any firm to hire me?


Steven J. Harper is an adjunct professor at Northwestern University. He recently retired as a partner at Kirkland & Ellis, after 30 years in private practice. His blog about the legal profession, The Belly of the Beast, can be found at A version of the column above was first published on The Belly of the Beast.

Make a comment

Comments (2)
Save & Share: Facebook | Del.ic.ious | | Email |

Reprints & Permissions


Report offensive comments to The Am Law Daily.

Steven, what a fascinating topic. I am sincerely surprised that it hasn’t generated a large stream of comments . . . so, maybe we can provoke some.

Two questions occur to me. One, can anyone discern success factors from interviewing for values, characteristics and traits, or reviewing candidates resumes? I guess time will tell.

It reminds me of experiences I had back in the late 80’s and early 90’s. At that time I was involved in actively training young attorneys (senior associates and newer partners) in client relations and business development skills. And please note that I labeled it as “skills’ not knowledge. It involved teaching lawyers how to convert a call from a prospect into a one-on-one meeting, how to ask for a retainer, how to effectively delegate a client matter to another attorney, and so forth; twelve skill topics in total. I eventually packaged this intellectual knowledge into a series of video-based training modules entitled Rainmaking® that were used in over 300 law firms world-wide. The relevance of all this was that I had the opportunity to personally meet with many of the older name partners that started significant firms as well as the senior and highly gifted business originators who were perceived within their firms to have some kind of magical powers. One of the important similarities that I “accidentally” discovered that most of these gifted attorneys shared, was that at some point early in their careers, often well before law school, they sold encyclopedias (anybody remember them?) door-to-door, were commission-based retail merchants, or tried their hand at life-insurance sales – any kind of similar experience that taught them that just because some prospective client said: “No” it wasn’t the end of the world. And, they had bloody well better get a few prospective clients to say, “Yes” otherwise they would starve. Now I’ve mentioned this discovery to more than a few managing partners over the years, but I’m not aware of any firm that has yet to formally factor in this kind of criteria.

Secondly, if you are able to discern any meaningful success factors, will anyone pay attention?

Fast forward to the late-90’s and I’m organized a one-day conference on law firm leadership. Among the speakers that I’ve brought to Chicago is the managing partner of a highly successful litigation boutique from Canada. This lawyer explains to the audience (made up of primarily US law firm managing partners) how his firm has utilized a personality profile to discern the psychometric profile of the kind of litigator who does extremely well within his firm. He takes great pains to explain that this is not the same profile that may work in just any law firm or even in any litigation boutique. He also shows the audience examples of profiles wherein his firm had taken a chance on hiring some candidate outside of the norm, only later to confirm that they need to stick to the profile that worked for them. The reaction of the audience? That may work in Canada, but we would never do anything like that in our firms!

Right on the money. At least for many litigators, much of the mystique has moved from big firms to boutiques. Savvy litigation clients have become skeptical of big firms claiming a monopoly over deciding the next generation of who represents them in the courtroom.

The comments to this entry are closed.

By: TwitterButtons.com

[email protected]

From the Newswire

Sign up to receive Legal Blog Watch by email
View a Sample