Read Question Stem or Stimulus First?

I interviewed Stephen Harris, former LSAT question-writer who has written hundreds of the questions that appear in your books of LSAT PrepTests.

Our discussion follows.

***

In Logical Reasoning, do you recommend reading the stimulus first or the question stem first? 

Always, always, always read the stem first. To begin with, the stem typically tells you what the task is that’s associated with the item, so you don’t know what you’re supposed to do (or even if you want to try) until you’ve read the stem. This is especially important since what you want to pay attention to in the stimulus depends on the task at hand. For example, you should approach the stimulus of a conclusion question completely differently than that of a flaw question or an inference question.

And even within a particular item type, say inference items, subtle differences in wording can suggest different things to look for in the stimulus. For example, if an inference stem says “…must be true that …” you have a strong hint that formal logic (“P —> Q” stuff) may be involved and will look for it in the stimulus; but if instead the stem reads “…provides the most support for…” you would expect that the process of elimination will be involved, and you would pay attention to the relative strength of the answer choices as you worked through them.

I’d love to know what dos and don’ts they tell the writers. What will I absolutely never see on an LSAT?

LSAC does provide a writer’s manual that covers the basics, especially things to avoid. Most of the “don’t”s I remember involve style/topic issues. LSAC tries very hard to avoid items that appear to favor one subgroup of test takers over another, and wants to stay away from controversial issues entirely. The rationale they offer is that material that elicited a strong emotional response in at least some test takers gives less reliable results than does material that test takers are able to approach calmly. That explains why the questions strike many folks as being so boring!

How could I get a job authoring questions for the LSAT? 

Well, the first step is to get a PhD, preferably in philosophy or linguistics. Once you’ve done this hopefully you will have better employment options available than subsistence-level freelance question writing, but step 2 is to contact the company that LSAC contracts with to supply questions (who is still ACT, if I’m not mistaken) and enquire about openings. But given the points mentioned above, the best way to think of the job is as a fun hobby rather than as an employment opportunity.

How is question difficulty initially assessed or decided?

I don’t pretend to know the details, but they look at a couple of things (at least) when evaluating questions: the proportion of test takers who get a question right, and how well performance on this question correlates with performance on other questions (you don’t want the only people getting a question right to be people who generally miss lots of other questions).

*** This is part 5 of the series of interviews. You can also get them all in a free book I put together.