Quarter of birth is not a good instrument! Hallo?

17 August 2009

Arrrgh! After 15 years, it’s hard to believe that John Bound’s and my critique of Angrist and Krueger’s quarter of birth instrument (and the whole weak instrument literature that followed, plus a bunch of papers that specifically adress the AK 1991 QJE paper) has yet to make a dent. How can the Economist claim that

In an influential early example of this sort of study, Joshua Angrist of the Massachusetts Institute of Technology (MIT) and Alan Krueger of Princeton University used America’s education laws to create an instrumental variable based on years of schooling. These laws mean that children born earlier in the year are older when they start school than those born later in the year, which means they have received less schooling by the time they reach the legal leaving-age. Since a child’s birth date is unrelated to intrinsic ability, it is a good instrument for teasing out schooling’s true effect on wages.

WRONG!!! Quarter of birth meets neither the relevance (at least in AK’s preferred specification) nor the exclusion restriction assumption required for IV. See Kasey Buckle’s and Dan Hungerman’s excellent paper that, one hopes, is the death knell for any paper that claims quarter of birth is unrelated to outcomes except through the compulsory schooling/school age starting law mechanism. There’s also a very good paper by Rashmi Barua and Kevin Lang that shows that the compulsory schooling/school age starting law mechanism doesn’t meet the monotonicity requirements for a LATE interpretation of the IV estimate of the effects of school entry on outcomes.

Despite all this, Angrist and Pischke in their otherwise excellent Mostly Harmless Econometrics once again belabor the AK results and present them as if they are sensible estimates of the returns to education.

Why is this? How can a result which has for many reasons and by many authors been shown to have problems persist in being held up as a shining example of the usefulness of the IV technique? Part of the answer is that more than any other IV story (and every good IV paper has a story), AK’s story is really good. Incredibly clever. Easy to see in graphs. Believable. So, we (even I) want to think that AK’s instrument is sensible and good. Because if the AK story doesn’t hold, then lots of other IV stories are probably invalid. And with them goes the whole natural experiment movement.

Update:
(I posted a slightly different version of this to The Economist website.)

Advertisements

3 Responses to “Quarter of birth is not a good instrument! Hallo?”

  1. Jan Says:

    I noticed too that the Angrist and Kruger paper often gets cited in an exclusively positive way.

    Part of the reason might be that professors in grad classes will usually first present the original Angrist and Krueger’s paper extensively before they go on to add any critical thoughts. Furthermore, often professors are quite modest when criticizing “established” papers, even when the counter-arguments are very strong, maybe because they do not want to force their point of view on the students. For students, the original arguments might therefore keep partially sticking.


  2. […] So we’ve known for a while that quarter of birth might not be a great instrument. But it would still get used. Bound, Jaeger and Baker complained about it in 1995; I think we went through their critique in our 1999 metrics class. Jaeger got angry in 2009 when quarter of birth was still being cited as a decent instrument. […]


  3. […] So we’ve known for a while that quarter of birth might not be a great instrument. But it would still get used. Bound, Jaeger and Baker complained about it in 1995; I think we went through their critique in our 1999 metrics class. Jaeger got angry in 2009 when quarter of birth was still being cited as a decent instrument. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: