LSAT Cultural Bias Interview

I interviewed Stephen Harris, former LSAT question-writer and author of Mastering Logic Games, about whether the LSAT is culturally biased.

(He’s written hundreds of the questions that appear in your books of LSAT PrepTests.)

Our discussion follows.

***

Do you think the LSAT is culturally biased? If so, what steps can be taken to correct for that? In writing questions, did you take measures to reduce your own cultural bias or does the question selection process correct for that? Is the LSAT a test of acculturation?

“Biased” is a word that is used in different senses. In some sense, the whole point of a test is to discriminate. A good test discriminates on the basis of reasonable considerations; a bad test not so much. For most tests, the basis for this discrimination will be cultural factors. Is a spelling test biased against bad spellers? In some sense, the answer is clearly yes. And that’s the point, even though spelling is a cultural phenomenon.

Likewise, the whole point of the LSAT is to help determine the extent to which test takers possess certain abilities, and these abilities are clearly cultural in some sense, like reading and drawing verbal inferences. But does this mean that either a spelling test or the LSAT is culturally biased in a troubling way? Not necessarily. They might be, by choosing idiosyncratic words to spell, for instance, or allowing extraneous factors that privilege one group over another to play a role on the LSAT. We can, and will, argue over which skills are the important ones, and whether a tool tests these skills in idiosyncratic ways that disadvantage otherwise qualified students inappropriately.

By any reasonable measure, it seems to me, the LSAT tests generally relevant skills in a manner that rarely prevents students who are likely to perform well in law school from gaining admission to some school or other. But bias in this pejorative sense is more like crime – you can never eliminate it, only combat it, especially since the standards by which inappropriate bias is judged are subject to dispute and undergoing constant change as a result of larger cultural conversations.

As for socioeconomic bias specifically, undoubtedly LSAT performance is correlated with socioeconomic status; life expectancy, quality of health care and education, and virtually everything else that people think is good is correlated with socioeconomic status. It would be really surprising if LSAT scores weren’t. But it seems to me that the problem with LSAT bias in this sense isn’t the LSAT per se as much as these other factors, and I doubt that there is an LSAT-specific remedy for this issue.

The most reliable way to limit one’s own inappropriate biases, I think, is to choose item topics carefully. Topics that one group might be more familiar with than another are generally poorly suited for test items. As an example, questions about sports very often risk the possibility of gender bias, since for some sports whether people play or follow them is pretty strongly correlated with gender.

*** This is part 3 of the series of interviews. You can also get them all in a free book I put together.