November 19, 2010 11:30 AM

Lies, Damn Lies, and Statistics

Posted by Ed Shanahan

By Steven Harper

ALM editor-in-chief Aric Press recently wrote about Indiana Law Professor Bill Henderson's new for-profit venture on recruiting, retention, and promotion. The Wall Street Journal's Law Blog and ABA Journal covered it, too.

I was troubled after reading about Henderson's venture.

The first thing that came to mind was the venerable computer programming maxim, "garbage in, garbage out." That's because Henderson is asking Am Law 200 partners to identify values and traits they want in their lawyers. And he's assuming they'll tell him the truth. But will they admit to seeking bright, ambitious associates wearing blinders in pursuit of elusive equity partnerships typically awarded to fewer than 10 percent of large firm entering classes? Or that such low "success" rates are inherent in the predominant large law firm business model that requires attrition and limits equity entrants to preserve staggering profits?

Then I considered Mark Twain's reflections on the three kinds of falsehoods: "Lies, damn lies, and statistics." It came to mind because Henderson's researchers, as Press wrote, "pour over the resumes and evaluations of associates and partners trying to identify characteristics shared by those who have become 'franchise players' and those who haven't." Here's what those resumes and evaluations won't reveal: the internal politics driving decisions.

Most equity partners are talented, but equally deserving candidates fail to advance for reasons unrelated to their abilities. Rather, as the business model incentivizes senior partners to hoard billings that justify personal economic positions, those at the top wield power that makes or breaks young careers--and everybody knows it. Doing a superior job is important, but working for the "right" people is outcome determinative. Merit sometimes loses out to idiosyncrasy that is impervious to Henderson's data collection methods.

But perhaps the biggest problem with Henderson's plan is it's goal: identifying factors correlating with individual success. Does the magic formula include, as Press writes, "a few years in the military, a few years in the job force, or a few years as a law review editor?"

If managers warm to Henderson's conclusions (after paying his company to develop them), they'll leap from correlation to causation, develop checklists of supposed characteristics common to superstars like themselves, and hire accordingly. Law schools pandering to the sliver of the profession that large firms occupy (it's less than 15 percent of all attorneys) could take such criteria even more seriously. Before long, prospective students will incorporate the acquisition of "success" credentials into their life plans.

The difficulty is that today's law firm partners already favor like-minded proteges. That tendency inhibits diversity as typically measured--gender, race, ethnicity, sexual orientation, and the like--along with equally important diversity of views and a willingness to entertain them. Even today, concerned insiders are reluctant to voice dissent from big law's prevailing raison d'etre--maximizing short-term profits at the expense competing professional values and longer-term institutional vitality. Won't meaningful diversity--of backgrounds, life experiences, and resulting attitudes about professional mission--suffer as groupthink makes firms even more insular? Meanwhile, trying to improve overall "success" rates is a futile goal; they won't budge until the leveraged pyramid disappears.

I don't fault Henderson, who bypassed a large firm practice for academia after his 2001 graduation, for his efforts. And I've accepted his invitation to take a closer look at what he's doing. But it's important to heed Press's warning: "To some extent, it doesn't matter what Henderson and Co. discover. What matters is that the inquiries have begun. If we've learned anything from the last decade, it's that we can't predict the consequences of new information beyond acknowledging its power to disrupt."

Consider two unfortunate examples. The flawed methodology behind U.S. News' law school rankings hasn't deterred most students from blindly choosing the highest-rated one that accepts them. (Exorbitant tuition and limited job prospects may be changing that.) Likewise, most of big law remains attached to equity partner profits as the principal (and sometimes exclusive) measure of a firm's success.

The most important things that happened to me--in work and in life--were fortuitous. No statistical model could have predicted them. Still, I hope Henderson's study answers an important question: Would his likely-to-succeed factors have led any firm to hire me?

 

Steven J. Harper is an adjunct professor at Northwestern University. He recently retired as a partner at Kirkland & Ellis, after 30 years in private practice. His blog about the legal profession, The Belly of the Beast, can be found at www.thebellyofthebeast.wordpress.com. A version of the column above was first published on The Belly of the Beast.

Make a comment

Comments (2)
Save & Share: Facebook | Del.ic.ious | | Email |

Reprints & Permissions

The comments to this entry are closed.