Helpful Reminders From the (Recent) Past

While Prof. Lawless' (Ill.) essay, What Empirical Legal Scholars Do Best, is over one decade old, it is surviving the test of time well. At its core, the essay advances the claim that one comparative advantage for legally-trained ELS scholars is that they can more easily lever "in-depth knowledge of fine-grained institutional detail that can unlock patterns that otherwise might remain hidden." At the same time, Bob is also quick (and correct) to note that formal legal training is far from a necessary precondition for generating the helpful in-depth knowledge base. Relatedly, even if legal scholars benefit from comparative advantage when it comes to studying legal institutions or doctrines, many law-trained scholars (those without quantitative Ph.D.s) must overcome other traditional comparative disadvantages, typically involving methodology and empirical execution. The essay's final claim also resonates: "empirical legal scholars should be developing new methodologies that are particularly suited to the study of the legal system."

Even if the essay reminds many of what is already well-understood, periodic reminders can be helpful as well.

An Updated (v.2.2) Stata Command for Structural Breaks in Time Series and Panel Data

In time series or panel data structural breaks (or change points) in relations between (or among) key variables can occur. If they do occur, researchers need to be aware of and adjust to such breaks. To make an obvious point clear, the applications to ELS are significant (e.g., Financial Crisis, Brexit Referendum, COVID19, to name but a few).

An updated version (v.2.2) of a user-written command, xtbreak, was recently released to the Stata platform that helps estimate and test for known and unknown structural breaks. For known breakpoints, xtbreak can assess whether the break occurs at a specific point in time. For unknown breaks, xtbreak tests three different hypotheses. First, no break against the alternative of s breaks; second, no breaks against a lower and upper limit of breaks; third, the null of s breaks against the alternative of one more break (s+1).

For those interested, information on the xtbreak program (v.2.2), as well as download information, is here. Additional info accompanied by a brief tutorial is found here.

More Post-Estimation Commands: Contrast

For those who include factor variables on the right-side of a regression equation, a more robust set of post-estimation commands in Stata unlocks additional information on, e.g., how various factor variable outcomes differ from, e.g., the factor variable's designated reference category. Stata's contrast command provides a set of contrast operators that make it easy to specify named contrasts such as reference-level contrasts, adjacent contrasts, Helmert contrasts, and orthogonal polynomial contrasts. For anyone interested, a Stata instructional video walks you through the basics (with a helpful on-line example, here) and additional information in included at UCLA's resource (click here).

Selection Effects: One Cautionary Tale

While the specific context discussed below is not law-related, per se, the degree to which it illustrates the potentially corrosive effects of selection effects remains germane to ELS scholars. Indeed, in much (if not most) of research on legal systems or institutions, empirical or other, it remains difficult to over-emphasize the threat posed by selection effects.

Recent media and scholarly attention has focused on a (so-called) "Mississippi education miracle." The purported "miracle" involves National Assessment of Educational Progress (NAEP) reading test score results for Mississippi 4th graders. Notably (and, perhaps, incredibly), between 2013 and 2024 Mississippi's ranking moved from 49th to a tie for 8th place among 53 US states and territories. From a purely psychometric perspective such a dramatic turnaround clearly marks a sharp deviation from what one would expect given the "laws of nature" as well as more than a century of empirical experience in the education setting.

While most of the initial public and scholarly attention was celebratory, more recent and emerging attention evidences more skepticism and focuses on the likely influence of a key change in Mississippi education policy. Specifically, since 2013 (when Mississippi's test score ascent began), only those third-graders who demonstrated acceptable reading skills were permitted to progress to the 4th grade and sit for the NAEP tests in Mississippi. Thus, Mississippi selected an outcome variable to help measure its education progress (NAEP reading score) and then instituted a compound treatment (deciding which 3rd grades students would move onto 4th grade) on the basis of that outcome variable (a student's reading skills).

While the entire episode is a bit more complicated (click here for a lengthy description), as Andrew Gelman observes, there are "lots of moving parts." Complications notwithstanding, one perspective (Gelman's) is that: "On statistical grounds, it would seem undeniable that some large chunk of the improved test scores in Mississippi come from the selection effect of delaying the students who were going to perform the worst, but it seems hard to put a number on this."