Research
Working Papers
-
Schools across the U.S. have sharply restricted student use of phones during the school day. We evaluate one type of restriction—lockable phone pouches—using nationwide data combining large-scale surveys, GPS pings, standardized test scores, and school administrative records, along with sales records from the largest pouch provider. Using a staggered difference-in-differences design, we find that pouch adoption substantially reduces phone use as measured by GPS pings and teacher reports. In the first year after adoption, disciplinary incidents increase and student subjective well-being falls, consistent with short-term disruption. However, effects on well-being become positive in later years and disciplinary effects fade. For academic achievement, average effects on test scores are consistently close to zero. High schools see modest positive effects, particularly in math, while middle schools see small negative effects. We find little evidence of effects on school attendance, self-reported classroom attention, or perceived online bullying.
-
The immediate impacts of COVID-19 on K12 schooling are well known. Over nearly 18 months, students' academic performance and mental health deteriorated dramatically. This study aims to identify if and how the pandemic led to longer-term changes in core aspects of schooling. Drawing on semi-structured interviews with 31 teachers and administrators across 12 districts in two states, we find that schools today look quite different in several areas including the availability and use of instructional technology, instructional practice, parent-teacher communication, and the balance between academics and social-emotional well-being. We interpret these findings through the lens of institutional theory, and discuss implications of the changes for practitioners and policymakers.R&R at the Journal of Educational Change
-
This paper presents the first evidence of how students make career technical education (CTE) course-taking decisions. Among the universe of Michigan high-schoolers we find large disparities in CTE access and participation by gender, race, and income. We decompose participation gaps between supply (access) and demand (preferences) with a simple discrete choice model. We find that student preferences for CTE content drive participation gaps by gender, inequities in access drive gaps by income, and school-level supply and demand factors combine to create the gaps by race. Policy simulations highlight the importance of accessible CTE delivery models within comprehensive high schools.R&R at the Journal of Human Resources
Peer-Reviewed Journal Articles
-
Transitional Kindergarten (TK) is a relatively new and under-researched model of early childhood education. Using data from Michigan TK, this study examines impacts for children (50% female, 78% White, 9% Black, 7% Hispanic) who enrolled at age 4 in 2014–2015 and 2018–2019 using a regression discontinuity design. Michigan TK improved kindergarten readiness (0.9 SD, N = 1,943) and third-grade math scores (0.2–0.3 SD, N = 15,680). It had no impact on ever being placed in special education from kindergarten through second grade, but it did cause earlier entry into special education in kindergarten (N = 15,704). Our findings add to the larger evidence base on early education programs and contribute substantially to the evidence base on TK specifically.
-
Media accounts suggest that American school board meetings have transformed from bureaucratic affairs into arenas of conflict, particularly around race. Yet systematic evidence remains limited. Analyzing over 40,000 meeting minutes from 2018 to 2022, we examine the prevalence and predictors of race-related discourse. Using large language models and natural language processing, we find that race-related content appears in 24% of meetings, predominately voiced by officials. Opposition related to racial issues occurs in only 1% of meetings, coming from both officials and the public. Race-related discussions—both affirmative and oppositional—are most common in suburban, high-income, highly educated districts in the Northeast and West, where local discourse tracks national race-related discourse. Oppositional content also peaks in politically competitive areas.
-
State laws that mandate in-grade retention for struggling readers are widespread in the U.S., and retention at the secondary school level is common in many countries. Researchers often use regression discontinuity (RD) methods to study such policies, leveraging strict performance cutoffs as an instrument to estimate the Local Average Treatment Effect (LATE) of retention on student achievement. In this paper, we document a likely threat to the internal validity of these studies. Examining two cohorts of Michigan students, we find that being flagged for retention increases reading performance by roughly 0.05 SD, a modest but meaningful impact. However, because being flagged increases the likelihood of actually being retained by only 3.4 percentage points, the implied effect of retention itself under standard assumptions would be an implausibly large 1.3 SD. Survey evidence suggests that flagged students receive more intensive reading support even if they are not retained, a violation of the exclusion restriction. Moreover, we estimate similar effects in districts that did not retain any students. These results raise concerns about potential bias in previously estimated retention effects and highlight the importance of carefully considering exclusion assumptions in analyses of multifaceted education interventions.
-
In 2015, Michigan increased its Career and Technical Education (CTE) funding and changed its funding formula to reimburse programs based on student progression through program curricula. Although this change nearly doubled program completion rates, student enrollment and persistence were unaffected; instead, administrators accelerated student progress by reorganizing course curricula around notches in the new funding formula. As a result of response heterogeneity, 30% of the funding increase was transferred away from high-poverty districts to more affluent ones, underscoring how supply-side responses to loopholes shape the incidence of public services.
-
In recent years, several states have expanded a new publicly funded learning option: Transitional Kindergarten (TK). TK bridges prekindergarten and kindergarten in its eligibility, requirements, and design. Using Michigan as a case study, we examine TK's fit in the early learning landscape. Broadly, we find TK in Michigan reduces some socioeconomic gaps in early program enrollment while exacerbating others. Specifically, districts with more White and fewer economically disadvantaged students are more likely to offer TK. Among preschool-age children, noneconomically disadvantaged children are more likely to enroll in TK; among kindergarten-age children, take-up is similar by family income. Finally, some children enroll in TK instead of other public options, but there is no evidence that public slots decline overall.
-
Detroit students who obtain a college degree overcome many obstacles to do so. This article reports the results of a randomized evaluation of a program meant to provide support to low-income community college students. The Detroit Promise Path program was designed to complement an existing College Promise scholarship, providing students with coaching, summer engagement, and financial incentives. The evaluation found that students offered the program enrolled in more semesters and earned more credits compared with those offered the scholarship alone. However, at the 3-year mark, there were no discernable impacts on degrees earned. This article examines systemic barriers to degree completion and offers lessons for the design of interventions to increase equity in postsecondary attainment.
-
Media reports suggest that parent frustration with COVID school policies and the growing politicization of education have increased community engagement with local public schools. However, there is no evidence to date on whether these factors have translated into greater engagement at the ballot box. This paper uses a novel data set to explore how school board elections changed following the start of the COVID-19 pandemic. I find that school board elections following the onset of the COVID-19 pandemic were more likely to be contested and that voter turnout in contested elections increased. These changes were large in magnitude and varied with several district characteristics.
-
Recent controversies have highlighted the importance of local school district governance, but little empirical evidence exists evaluating the quality of district policy makers or policies. In this paper, we take a novel approach to assessing school district decision making. We posit a model of rational decision making under uncertainty that emphasizes districts learning over time. We test the predictions from the model using data on a set of highly visible and consequential decisions facing school district leaders—the choice of learning mode during the 2020–21 school year. We find that district behavior is consistent with a Bayesian learning process in several key respects. Districts respond on the margin to health risks: All else equal, a marginal increase in new COVID-19 cases reduces the probability that a district offers in-person instruction the next week. This negative response is magnified when the district was in-person the prior week and attenuates in magnitude over the school year, suggesting that districts learn from experience about the effect of in-person learning on disease transmission in schools. We also find evidence that districts are influenced by the learning mode decisions of peer districts, but not their peers' experiences with in-person instruction and disease transmission, which implies that some important frictions exist.
-
Transitional kindergarten (TK) is a relatively recent entrant into the US early education landscape, combining features of public pre-K and regular kindergarten. We provide the first estimates of the impact of Michigan's TK program on third-grade test scores. Using an augmented regression discontinuity design, we find that TK improves third-grade test scores by 0.29 (math) and 0.19 (English language arts) standard deviations relative to a counterfactual that includes other formal and informal early learning options. These impacts are notably large relative to the prior pre-K literature.
-
In 2014, the municipal water source in Flint, Michigan was switched, causing lead from aging pipes to leach into the city's drinking water. While lead exposure in Flint children increased modestly on average, some children were exposed to high lead levels. Surveys of Flint residents show the water crisis was also associated with increased levels of stress, anxiety, and depression. We use Michigan's administrative education data and utilize synthetic control methods to examine the impact of the crisis on Flint's school-age children. We find decreases in math achievement and increases in special needs classification, even among children living in homes with copper (rather than lead) water service lines. Low socioeconomic status students and younger students experienced the largest effects on math achievement, and boys experienced the largest effects on special needs classification. Our results point toward the broad negative effects of the crisis on children and suggest that existing estimates may substantially underestimate the overall societal cost of the crisis.
-
Lottery-based identification strategies offer potential for generating the next generation of evidence on U.S. early education programs. The authors' collaborative network of five research teams applying this design in early education settings and methods experts has identified six challenges that need to be carefully considered in this next context: (a) available baseline covariates that may not be very rich; (b) limited data on the counterfactual; (c) limited and inconsistent outcome data; (d) weakened internal validity due to attrition; (e) constrained external validity due to who competes for oversubscribed programs; and (f) difficulties answering site-level questions with child-level randomization. The authors offer potential solutions to these six challenges and concrete recommendations for the design of future lottery-based early education studies.
-
We explore how much borrowers value student debt relief in the setting of the federal Teacher Loan Forgiveness program, which cancels between $5,000 and $17,500 in debt for teachers at high-need schools. Using both quasi-experimental evidence and a randomized controlled trial, we find that neither eligibility nor a targeted information intervention affects employment decisions. Information was found to increase application and receipt rates for teachers who had achieved eligibility. Evidence from contingent valuation surveys suggests that teachers do in general value debt relief. Incorporating qualitative evidence, we conclude that take-up may be constrained by program complexity and administrative barriers.
-
This paper examines whether school COVID-19 policies influenced enrollment differently by student age and race/ethnicity. Unlike much prior research, we i) analyze enrollments for virtually the entire U.S. public school population for both the 2020–2021 and 2021–2022 school years, ii) compare enrollment trends within districts in order to isolate subgroup heterogeneity from district characteristics, and iii) account for district selection into preferred learning modes. Analyzing data on over 9,000 districts that serve more than 90% of public school students in the United States, we find enrollment responses to COVID policies differed notably. We find that White enrollments declined more than Black, Hispanic, and Asian enrollments in districts that started the 2020–2021 school year virtually, but in districts that started in-person the reverse was true: Non-White enrollments declined more than White enrollments. Moreover, Black, Hispanic, and Asian families responded more than White families to higher COVID-19 death rates in the months preceding the start of the 2021 school year. In 2021–2022, enrollment differences by the previous year's learning mode persisted. Racial/ethnic differences did not vary by whether the district required masking in classrooms. These findings are consistent with the greater risk faced by communities of color during the pandemic and demonstrate an additional source of disparate impact from COVID policies.
-
Roughly one in four juveniles arrested in the U.S. spend time in a detention center prior to their court date. To study the consequences of this practice for youth, we link the universe of individual public school records in Michigan to juvenile and adult criminal justice records. Using a selection-on-observables design, we estimate that juvenile detention leads to a 38% decline in the likelihood of graduating high school and a 27% increase in the likelihood of being arrested as an adult by age 19. Falsification tests suggest the results are not driven by unobserved heterogeneity.
-
There is growing interest in the use of unconditional cash transfers as a means to alleviate poverty, yet little is known about the effects of such transfers in the U.S. This paper reports on the results of a randomized controlled study of a one-time $1,000 unconditional cash transfer in May 2020 to families with low incomes in 12 U.S. states. The families were receiving, or had recently received, Supplemental Nutrition Assistance Program benefits. We examine the impact of the cash transfer on five pre-registered outcomes (material hardship, mental health, parenting, child behavior, partner relationships) and several secondary outcomes (hardship avoidance, consumption, employment, benefit use). We find no statistically significant effects (powered to detect effects of 0.09 standard deviations) of the cash transfer on any outcomes for the full sample. In pre-specified exploratory analyses, we find significant reductions in material hardship (-0.17 standard deviations) among families with less than $500 of earnings in the previous month, roughly the bottom 50 percent of monthly earnings for the study sample.
-
A common rationale for offering online courses in K-12 schools is that they allow students to take courses not offered at their schools; however, there has been little research on how online courses are used to expand curricular options when operating at scale. We assess the extent to which students and schools use online courses for this purpose by analyzing statewide, student-course level data from high school students in Florida, which has the largest virtual sector in the nation. We introduce a "novel course" framework to address this question. We define a virtual course as "novel" if it is only available to a student virtually, not face-to-face through their own home high school. We find that 7% of high school students in 2013–14 enrolled in novel online courses. Novel courses were more commonly used by higher-achieving students, in rural schools, and in schools with relatively few Advanced Placement/International Baccalaureate offerings.
-
This paper reports findings from a randomized controlled trial of a one-time, $1,000 unconditional cash transfer to low-income households in October 2020. We use a combination of administrative and survey data collected six weeks posttreatment to examine four preregistered hypotheses: impacts on material hardship and mental health in the full study sample as well as among a very low-income sample. We find no effects of the cash transfer on any of the prespecified or other exploratory outcomes. We explore various explanations for these null results and discuss implications for future research on unconditional cash transfer programs.
-
Prior research suggests that gender differences in hours worked play an important role in the gender pay gap. Yet common estimates of the wage returns to hours worked are close to zero, implying that hours differences explain little of the gender wage gap, even though men work more hours than women on average. However, while the wage returns to hours worked within occupations are small, the authors document that the wage returns to average hours worked across occupations are large. They develop a conceptual framework that reconciles these facts. Findings show that, under some assumptions, gender differences in hours worked can account for a substantial portion of the gender wage gap and that increases in the returns to hours worked over the past four decades slowed progress in reducing the gender pay gap.
-
Foster care provides substitute living arrangements to protect maltreated children. The practice is remarkably common: it is estimated that 5 percent of children in the United States are placed in foster care at some point during childhood. This paper describes the main tradeoffs in child welfare policy and provides background on policy and practice most in need of rigorous evidence. Trends include efforts to prevent foster care on the demand side and to improve foster home recruitment on the supply side. With increasing data availability and a growing interest in evidence-based practices, there are opportunities for economic research to inform policies that protect vulnerable children.
-
In 2011, the U.S. Department of Education granted states the opportunity to apply for waivers from the core requirements of the No Child Left Behind Act. In exchange, many states implemented systems of differentiated accountability that included a focus on schools with the largest achievement gaps between subgroups of students. We use administrative data from Michigan in a series of regression-discontinuity analyses to study the effects of these school reforms on schools and students. We find some evidence that targeting schools for such reforms led to small, short-run reductions in the within-school math achievement gap. However, these reductions were driven by stagnant performance of lower-achieving students alongside declines in the performance of their higher-achieving peers. These findings serve as a cautionary tale for the capacity of the accountability provisions embedded in the recent reauthorization of No Child Left Behind, the Every Student Succeeds Act, to meaningfully improve student and school outcomes.
-
We show that the design and decentralized scoring of New York's high school exit exams—the Regents Examinations—led to systematic manipulation of test scores just below important proficiency cutoffs. Exploiting a series of reforms that eliminated score manipulation, we find heterogeneous effects of test score manipulation on academic outcomes. While inflating a score increases the probability of a student graduating from high school by about 17 percentage points, the probability of taking advanced coursework declines by roughly 10 percentage points. We argue that these results are consistent with test score manipulation helping less advanced students on the margin of dropping out but hurting more advanced students that are not pushed to gain a solid foundation in the introductory material.
-
This article uses fixed effects models to estimate differences in contemporaneous and downstream academic outcomes for students who take courses virtually and face-to-face—both for initial attempts and for credit recovery. We find that while contemporaneous outcomes are positive for virtual students in both settings, downstream outcomes vary by attempt type. For first-time course takers, virtual course taking is associated with decreases in the likelihood of taking and passing follow-on courses and in graduation readiness (based on a proxy measure). For credit recovery students, virtual course taking is associated with an increased likelihood of taking and passing follow-on courses and being in line for graduation. Supplemental analyses suggest that selection on unobservables would have to be substantial to render these results null.
-
Michigan Merit Curriculum (MMC) is a statewide college-preparatory policy that applies to the high school graduating class of 2011 and later. Using detailed Michigan high school transcript data, this article examines the effect of the MMC on various students' course-taking and achievement outcomes. Our analyses suggest that (a) post-MMC cohorts took and passed approximately 0.2 additional years' of math courses, and students at low socioeconomic status (SES) schools drove nearly all of these effects; (b) post-policy students also completed higher-level courses, with the largest increase among the least prepared students; (c) we did not find strong evidence on students' ACT math scores; and (d) we found an increase in college enrollment rates for post-MMC cohorts, and the increase is mostly driven by well-prepared students.
-
Selecting more productive employees among a pool of job applicants can be a cost-effective means of improving organizational performance and may be particularly important in the public sector. We study the relationship among applicant characteristics, hiring outcomes, and job performance for teachers in the Washington DC Public Schools. Applicants' academic background (e.g., undergraduate GPA) is essentially uncorrelated with hiring. Screening measures (written assessments, interviews, and sample lessons) help applicants get jobs by placing them on a list of recommended candidates, but they are only weakly associated with the likelihood of being hired conditional on making the list. Yet both academic background and screening measures strongly predict teacher job performance, suggesting considerable scope for improving schools via the selection process.
-
An important goal of community colleges is to prepare students for the labor market. But are students aware of the labor market outcomes in different majors? And how much do students weigh labor market outcomes when choosing a major? In this study we find that less than 15% of a sample of community college students in California rank broad categories of majors accurately in terms of labor market outcomes. Students believe that salaries are 13% higher than they actually are, on average, and students underestimate the probability of being employed by almost 25%. We find that the main determinants of major choice are beliefs about course enjoyment and grades, but expected labor market outcomes also matter. Experimental estimates of the impact of expected labor market outcomes are larger than OLS estimates and show that a 10% increase in salary is associated with a 14 to 18% increase in the probability of choosing a specific category of majors.
-
A substantial and rapidly growing literature has developed around estimating earnings gains from 2-year college degrees using administrative data. These papers almost universally employ a person-level fixed-effects strategy to estimate earnings premia net of fixed attributes. We note that the seminal piece on which these papers build—Jacobson, Lalonde, & Sullivan, Journal of Econometrics, 2005, 125(1–2), 271–304—provides theoretical and empirical evidence for the importance of additionally differencing out individual time trends. The subsequent literature has not followed suit. Through replication we ask whether this matters. We show that it does, and further that these person-level time trends need not be computationally burdensome in large administrative data. We recommend them as a unifying econometric standard for future work.
-
Early childhood trauma increases the risk of academic difficulties. The purpose of this study was to investigate the prevalence of early contact with child protection services (CPS) and to determine whether early exposure to maltreatment investigations was associated with important academic outcomes. The authors focused specifically on standardized test scores (math and reading), grade retention, and special education status in third grade. The sample was diverse and included all children born between 2000 and 2006 and enrolled in Michigan's public schools (N = 732,838). By the time these students reached third grade, approximately 18% were associated with a formal CPS investigation. In some school districts, more than 50% of third graders were associated with an investigation. African American and poor students were more likely to be investigated for maltreatment. Children associated with maltreatment investigations scored significantly lower on standardized math and reading tests, were more likely to be identified as needing special education, and were more likely to be held back at least one grade. These findings indicate that involvement with CPS is not an infrequent event in the lives of young children and that within some school districts, maltreatment investigations are the norm. Child welfare and educational systems must collaborate so that the early academic struggles experienced by victims of maltreatment do not mature into more complicated difficulties later in life.
-
This paper investigates whether demand-side market pressure explains colleges' decisions to provide consumption amenities to their students. Using a discrete choice model of college demand, we find that most students appear to value consumption amenities, such as operating spending on student activities, sports, and dormitories, while the taste for academic quality is confined to high-achieving students. Heterogeneity in student preferences creates variation in demand pressure across institutions, which we estimate can account for 11% of the total variation in the ratio of amenity to academic spending across 4-year colleges in the United States.
-
This article examines the impacts of the Michigan Merit Curriculum (MMC), a statewide college-preparatory curriculum that applies to the high school graduating class of 2011 and later. Our analyses suggest that the higher expectations embodied in the MMC had slight impact on student outcomes. Looking at student performance in the ACT, the only clear evidence of a change in academic performance comes in science. Our best estimates indicate that ACT science scores improved by 0.2 points (or roughly 0.04 SD) as a result of the MMC. Our estimates for high school completion are sensitive to the choice of specification, though some evidence suggests that the MMC reduced graduation for the least prepared students.
-
This paper reports the results of an experimental evaluation of Evidence-Based Literacy Instruction (EBLI). Developed over 15 years ago, EBLI aims to provide teachers with instructional strategies to improve reading accuracy, fluency and comprehension. Sixty-three teachers in grades 2–5 in seven Michigan charter schools were randomly assigned within school-grade blocks to receive EBLI training or a business-as-usual control condition. Comparing students in treatment and control classrooms during the 2014–15 school year, we find no significant impact on reading performance. Teacher survey responses and interviews with program staff suggest that several implementation challenges may have played a role in the null findings.
-
Economists often use test scores to measure a student's performance or an adult's human capital. These scores reflect nontrivial decisions about how to measure and scale student achievement, with important implications for secondary analyses. For example, the scores computed in several major testing regimes, including the National Assessment of Educational Progress (NAEP), depend not only on the examinees' responses to test items, but also on their background characteristics, including race and gender. As a consequence, if a black and white student respond identically to questions on the NAEP assessment, the reported ability for the black student will be lower than for the white student—reflecting the lower average performance of black students. This can bias many secondary analyses. Other assessments use different measurement models. This paper aims to familiarize applied economists with the construction and properties of common cognitive score measures and the implications for research using these measures.
-
One long-standing motivation for low-income housing programs is the possibility that housing affordability and housing conditions generate externalities, including on children's behavior and long-term life outcomes. We take advantage of a randomized housing voucher lottery in Chicago in 1997 to examine the long-term impact of housing assistance on a wide variety of child outcomes, including schooling, health, and criminal involvement. In contrast to most prior work focusing on families in public housing, we focus on families living in unsubsidized private housing at baseline, for whom voucher receipt generates large changes in both housing and nonhousing consumption. We find that the receipt of housing assistance has little, if any, impact on neighborhood or school quality or on a wide range of important child outcomes.
-
Over the past decade, most of the quantitative studies on teacher preparation have focused on comparisons between alternative and traditional routes. There has been relatively little quantitative research on specific features of teacher education that might cause certain pathways into teaching to be more effective than others. The vast majority of evidence on features of preservice preparation comes from qualitative case studies of single institutions that prepare teachers. Among the few large-scale cross-institution studies that exist, most provide only descriptive trends that fail to account for teacher and school characteristics that might explain apparent relationships in the data. Additionally, these studies typically look at state- or district-level data, providing little information on national trends.
-
Recent evidence on the large variance in teacher effectiveness has spurred interest in teacher labor markets. Research documents that better qualified teachers typically work in more advantaged schools but cannot determine the relative importance of supply versus demand. To isolate teacher preferences, we document which schools prospective teachers interviewed at during job fairs in Chicago. We find substantial variation in the number of applicants per school, ranging from under five to over 300. Schools serving more advantaged students have more applicants per vacancy, on average, and teacher preferences vary systematically by their own demographic characteristics. School geographic location is highly predictive of applications, even after controlling for distance from applicants' home addresses and a host of school and neighborhood characteristics.
-
In 2004, the Chicago Public Schools and the Chicago Teachers Union signed a new collective bargaining agreement that gave principals the flexibility to dismiss probationary teachers (those with fewer than 5 years of experience) for any reason and without the hearing process typical in many urban districts. Results suggest that the policy reduced annual teacher absences by roughly 10% and reduced the incidence of frequent absences by 25%. The majority of the effect was due to changes in the composition of teachers in the district, although there is evidence of modest incentive effects for young untenured teachers.
-
In this paper we estimate the causal effects on child mortality from moving into less distressed neighborhood environments. We match mortality data covering the period from 1997 to 2009 with information on every child in public housing that applied for a housing voucher in Chicago in 1997 (N = 11,680). Families were randomly assigned to the voucher wait list, and only some families were offered vouchers. The odds ratio for the effects of being offered a housing voucher on overall mortality rates is equal to 1.13 for all children (95% CI 0.73–1.70), 1.34 for boys (95% CI 0.85–2.05) and 0.21 for girls (95% CI 0.01–1.04).
-
A number of studies have examined the impact of school accountability policies, including No Child Left Behind (NCLB), on student achievement. However, there is relatively little evidence on how school accountability reforms and NCLB, in particular, have influenced education policies and practices. This study examines the effects of NCLB on multiple district, school, and teacher traits using district-year financial data and pooled cross sections of teacher and principal surveys. Our results indicate that NCLB increased per-pupil spending by nearly $600, which was funded primarily through increased state and local revenue. We find that NCLB increased teacher compensation and the share of elementary school teachers with advanced degrees but had no effects on class size. We also find that NCLB did not influence overall instructional time in core academic subjects but did lead schools to reallocate time away from science and social studies and toward the tested subject of reading.
-
Teacher and principal surveys are among the most common data collection techniques employed in education research. Yet there is remarkably little research on survey methods in education, or about the most cost-effective way to raise response rates among teachers and principals. In an effort to explore various methods for increasing survey response rates, we randomly assigned 1,177 high school principals in the state of Michigan to 1 of 4 experimental conditions. We varied the mode of survey delivery, the mode in which the prenotification letter was sent, and whether or not a $10 incentive was provided. The results indicate that providing a monetary incentive substantially increased response rates over the no incentive condition and that principals were more likely to respond to a paper-based survey than a web-based one.
-
Plagiarism appears to be a common problem among college students, yet there is little evidence on the effectiveness of interventions designed to minimize plagiarism. This study presents the results of a field experiment that evaluated the effects of a web-based educational tutorial in reducing plagiarism. We found that assignment to the treatment group substantially reduced the likelihood of plagiarism, particularly among student with lower SAT scores who had the highest rates of plagiarism. A followup survey suggests that the intervention reduced plagiarism by increasing student knowledge rather than by increasing the perceived probabilities of detection and punishment.
-
This study estimates the effects of means-tested housing programs on labor supply using data from a randomized housing voucher wait-list lottery in Chicago. Economic theory is ambiguous about the expected sign of any labor supply response. We find that among working-age, able-bodied adults, housing voucher use reduces labor force participation by around 4 percentage points (6 percent) and quarterly earnings by $329 (10 percent), and increases Temporary Assistance for Needy Families program participation by around 2 percentage points (15 percent). We find no evidence that the housing-specific mechanisms hypothesized to promote work, such as neighborhood quality or residential stability, are important empirically. (JEL I38, J22, R23, R38)
-
We examine the effect of more than 3,400 gun shows using data from Gun and Knife Show Calendar and vital statistics data from California and Texas. Considering the one month following each show and a surrounding area ranging from 80 to 2,000 square miles, we find no evidence that gun shows increase either gun homicides or suicides. The similarity of our estimates for California and Texas suggests that the much tighter California gun show regulations do not substantially reduce the number of firearms-related deaths in that state. Using incident-level crime data for Houston, Texas, we also find no evidence of an effect on other crime categories.
-
The No Child Left Behind (NCLB) Act compelled states to design school accountability systems based on annual student assessments. The effect of this federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of fourth graders (effect size 5 0.23 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in eighth-grade math achievement, particularly among traditionally low-achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased fourth-grade reading achievement. © 2011 by the Association for Public Policy Analysis and Management.
-
This article takes advantage of a unique policy change to examine how principals make decisions regarding teacher dismissal. In 2004, the Chicago Public Schools (CPS) and Chicago Teachers Union signed a new collective bargaining agreement that gave principals the flexibility to dismiss probationary teachers for any reason and without the documentation and hearing process that is typically required for such dismissals. With the cooperation of the CPS, I matched information on all teachers who were eligible for dismissal with records indicating which teachers were dismissed. With these data, I estimate the relative weight that school administrators place on a variety of teacher characteristics. I find evidence that principals do consider teacher absences and value-added measures, along with several demographic characteristics, in determining which teachers to dismiss.
-
In this paper, we estimate the impact of receiving an NIH postdoctoral training grant on subsequent publications and citations. Our sample consists of all applications for NIH postdoctoral training grants (unsuccessful as well as successful) from 1980 to 2000. Both ordinary least squares and regression discontinuity estimates show that receipt of an NIH postdoctoral fellowship leads to about one additional publication over the next five years, which reflects a 20% increase in research productivity.
-
When students are tracked into vocational and academic secondary schools, access to higher education is usually restricted to those who complete an academic track. Postponing such tracking may increase university attendance among disadvantaged students if additional time in school enables them to catch up with their more privileged counterparts. However, if ability and expectations are fairly well set by an early age, postponing tracking during adolescence may not have much effect. This paper exploits an educational reform in Romania to examine the impact of postponing tracking on the proportion of disadvantaged students graduating from university using a regression discontinuity (RD) design. We show that, although students from poor, rural areas and with less educated parents were significantly more likely to finish an academic track and become eligible to apply for university after the reform, this did not translate into an increase in university completion. Our findings indicate that simply postponing tracking, without increasing the slots available in university, is not sufficient to improve access to higher education for disadvantaged groups.
-
We examine the relationship between the formal ratings that principals give teachers and a variety of observable teacher characteristics, including proxies for productivity. Prior work has shown that principals can differentiate between more and less effective teachers, especially at the tails of the quality distribution, and that subjective evaluations of teachers are strongly correlated with subsequent student achievement. However, whereas prior work has relied on survey data, we consider formal ratings from a setting in which the stakes are reasonably high. We find that the ratings are correlated with an array of teacher qualities including experience for young teachers, education credentials, and teacher absenteeism. Our finding that principals reward qualities of teachers known to be related to student productivity provides reason to be optimistic about policies that would assign more weight to principal evaluations of teachers in career decisions and compensation.
-
Research on the relationship between teacher characteristics and teacher effectiveness has been underway for over a century, yet little progress has been made in linking teacher quality with factors observable at the time of hire. To extend this literature, we administered an in-depth survey to new math teachers in New York City and collected information on a number of nontraditional predictors of effectiveness, including teaching-specific content knowledge, cognitive ability, personality traits, feelings of self-efficacy, and scores on a commercially available teacher selection instrument. We find that only a few of these predictors have statistically significant relationships with student and teacher outcomes. However, the individual variables load onto two factors, which measure what one might describe as teachers' cognitive and noncognitive skills. We find that both factors have a moderately large and statistically significant relationship with student and teacher outcomes, particularly with student test scores.
-
The controversial No Child Left Behind Act (NCLB) brought test-based school accountability to scale across the United States. This study draws together results from multiple data sources to identify how the new accountability systems developed in response to NCLB have influenced student achievement, school-district finances, and measures of school and teacher practices. Our results indicate that NCLB brought about targeted gains in the mathematics achievement of younger students, particularly those from disadvantaged backgrounds. However, we find no evidence that NCLB improved student achievement in reading. School-district expenditure increased significantly in response to NCLB, and these increases were not matched by federal revenue. Our results suggest that NCLB led to increases in teacher compensation and the share of teachers with graduate degrees. We find evidence that NCLB shifted the allocation of instructional time toward math and reading, the subjects targeted by the new accountability systems.
-
This paper constructs a statistical model of learning that suggests a systematic way of measuring the persistence of treatment effects in education. This method is straightforward to implement, allows for comparisons across educational treatments, and can be related to intuitive benchmarks. We demonstrate the methodology using student-teacher linked administrative data for North Carolina to examine the persistence of teacher quality. We find that teacher-induced learning has low persistence, with three-quarters or more fading out within one year. Other measures of teacher quality produce similar or lower persistence estimates.
-
Low-achieving students in many school districts are retained in a grade to allow them to gain the academic or social skills that teachers believe are necessary to succeed academically. In this paper, we use plausibly exogenous variation in retention generated by a test-based promotion policy to assess the causal impact of grade retention on high school completion. We find that retention among younger students does not affect the likelihood of high school completion, but that retaining low-achieving eighth grade students in elementary school substantially increases the probability that these students will drop out of high school. (JEL I21, J13)
-
Peer interactions among children have long interested social scientists. Identifying causal peer effects is difficult, and a number of studies have used random assignment to produce evidence that peers affect each other's outcomes. This focus by sociologists and economists on whether peers affect each other has not been matched by direct evidence on how these effects operate. The authors argue that one reason for the small number of studies in sociology and economics on the mechanisms underlying peer effects is the difficulty of collecting data on microinteractions. They argue technology reduces data collection costs relative to direct observation and allows for realistic school activities with randomly assigned peers. The authors describe a novel strategy for collecting data on peer interactions and discuss how this approach might shed light on mechanisms underlying peer influence. The centerpiece of this strategy is the use of handheld computers by middle and high school students as part of interactive math and science lessons called the Discussion Game. The handhelds collect data on interactions between students and track how students' answers evolve as they interact with different peers.
-
We examine how well principals can distinguish between more and less effective teachers. To put principal evaluations in context, we compare them with the traditional determinants of teacher compensation—education and experience—as well as value‐added measures of teacher effectiveness based on student achievement gains. We present "out‐of‐sample" predictions that mitigate concerns that the teacher quality and student achievement measures are determined simultaneously. We find that principals can generally identify teachers who produce the largest and smallest standardized achievement gains but have far less ability to distinguish between teachers in the middle of this distribution.
-
This paper examines revealed preferences of parents for their children's education, using parent requests for individual elementary school teachers and information on teacher attributes, including principal reports of teacher characteristics that are typically unobservable. On average, parents strongly prefer teachers whom principals describe as good at promoting student satisfaction, though they also value teacher ability to raise academic achievement. These aggregate effects mask striking differences across schools. Families in higher poverty schools strongly value student achievement and appear indifferent to the principal's report of a teacher's ability to promote student satisfaction. The results are reversed for families in wealthier schools.
-
While the persistence of criminal activity is well documented, this may be due to persistence in the unobserved determinants of crime. There are good reasons to believe, however, that there may actually be a negative relationship between crime rates in a particular area due to temporal displacement. We exploit the correlation between weather and crime to examine the short-run dynamics of crime. Using variation in lagged crime rates due to weather shocks, we find that the positive serial correlation is reversed. These findings suggest that the long-run impact of temporary crime-prevention efforts may be smaller than the short-run effects.
-
School choice has become an increasingly prominent strategy for enhancing academic achievement. To evaluate the impact on participants, we exploit randomized lotteries that determine high school admission in the Chicago Public Schools. Compared to those students who lose lotteries, students who win attend high schools that are better in a number of dimensions, including peer achievement and attainment levels. Nonetheless, we find little evidence that winning a lottery provides any systematic benefit across a wide variety of traditional academic measures. Lottery winners do, however, experience improvements on a subset of nontraditional outcome measures, such as self-reported disciplinary incidents and arrest rates.
-
The recent federal education bill, No Child Left Behind, requires states to test students in grades 3 to 8 each year and to judge school performance on the basis of these test scores. While intended to maximize student learning, there is little empirical evidence about the effectiveness of such policies. This study examines the impact of an accountability policy implemented in the Chicago Public Schools in 1996–1997. Using a panel of student-level, administrative data, I find that math and reading achievement increased sharply following the introduction of the accountability policy, in comparison to both prior achievement trends in the district and to changes experienced by other large, urban districts in the mid-west. However, for younger students, the policy did not increase performance on a state-administered, low-stakes exam. An item-level analysis suggests that the observed achievement gains were driven by increases in test-specific skills and student effort. I also find that teachers responded strategically to the incentives along a variety of dimensions—by increasing special education placements, preemptively retaining students and substituting away from low-stakes subjects like science and social studies.
-
We explore the impact of school choice on student outcomes in the context of open enrollment within the Chicago Public Schools (CPS). Roughly half of the students opt out of their assigned high school to attend a different CPS school, and these students are much more likely than those who remain in their assigned schools to graduate. To determine the source of this apparent benefit, we compare outcomes across (i) similar students with differential access to schooling options and (ii) travelers and non-travelers within the same school. The results suggest that, other than for students who select career academies, the observed cross-sectional benefits are likely spurious.
-
This paper utilizes a plausibly exogenous source of variation in housing assistance generated by public housing demolitions in Chicago to examine the impact of high-rise public housing on student outcomes. I find that children in households affected by the demolitions do no better or worse than their peers on a wide variety of achievement measures. Because the majority of households that leave high-rise public housing in response to the demolitions move to neighborhoods and schools that closely resemble those they left, the zero effect of the demolitions may be interpreted as the independent impact of public housing.
-
As standards and accountability have become increasingly prominent features of the educational landscape, educators have relied more on remedial programs such as summer school and grade retention to help low-achieving students meet minimum academic standards. Yet the evidence on the effectiveness of such programs is mixed, and prior research suffers from selection bias. However, recent school reform efforts in Chicago provide an opportunity to examine the causal impact of these remedial education programs. In 1996, the Chicago Public Schools instituted an accountability policy that tied summer school and promotional decisions to performance on standardized tests, which resulted in a highly nonlinear relationship between current achievement and the probability of attending summer school or being retained. Using a regression discontinuity design, we find that the net effect of these programs was to substantially increase academic achievement among third-graders, but not sixth-graders. In addition, contrary to conventional wisdom and prior research, we find that retention increases achievement for third-grade students and has little effect on math achievement for sixth-grade students.
-
While there is a substantial literature on the relationship between general teacher characteristics and student learning, school districts and states often rely on in-service teacher training as a part of school reform efforts. Recent school reform efforts in Chicago provide an opportunity to examine in-service training using a quasi-experimental research design. In this paper, we use a regression discontinuity strategy to estimate the effect of teacher training on the math and reading performance of elementary students. We find that marginal increases in in-service training have no statistically or academically significant effect on either reading or math achievement, suggesting that modest investments in staff development may not be sufficient to increase the achievement of elementary school children in high-poverty schools.
-
This paper examines the short-term effect of school on juvenile crime. To do so, we bring together daily measures of criminal activity and detailed school calendar information from 29 jurisdictions across the country, and utilize the plausibly exogenous variation generated by teacher in-service days. We find that the level of property crime committed by juveniles decreases by 14 percent on days when school is in session, but the level of violent crime increases by 28 percent on such days. Our findings suggest that both incapacitation and concentration influence juvenile crime.
-
We develop an algorithm for detecting teacher cheating that combines information on unexpected test score fluctuations and suspicious patterns of answers for students in a classroom. Using data from the Chicago public schools, we estimate that serious cases of teacher or administrator cheating on standardized tests occur in a minimum of 4–5 percent of elementary school classrooms annually. The observed frequency of cheating appears to respond strongly to relatively minor changes in incentives. Our results highlight the fact that high-powered incentive systems, especially those with bright line rules, may induce unexpected behavioral distortions such as cheating. Statistical analysis, however, may provide a means of detecting illicit acts, despite the best attempts of perpetrators to keep them clandestine.
-
This article analyzes the impact of high-stakes testing in Chicago on student achievement in grades targeted for promotional decisions. Using a three-level Hierarchical Linear Model, we estimate achievement value added in gate grades (test-score increases over and above that predicted from a student's prior growth trajectory) for successive cohorts of students and derive policy effects by comparing value added pre- and postpolicy. Test scores in these grades increased substantially following the introduction of high-stakes testing. The effects are larger in the 6th and 8th grades and smaller in the 3rd grade in reading. Effects are also larger in previously low-achieving schools. In reading, students with low skills experienced the largest improvement in learning gains in the year prior to testing, while students with skills closer to their grade level experienced the greatest benefits in mathematics.
-
Nearly 60 percent of college students today are women. Using longitudinal data on a nationally representative cohort of eighth grade students in 1988, I examine two potential explanations for the differential attendance rates of men and women—returns to schooling and non-cognitive skills. The attendance gap is roughly five percentage points for all high school graduates. Conditional on attendance, however, there are few differences in type of college, enrollment status or selectivity of institution. The majority of the attendance gap can be explained by differences in the characteristics of men and women, despite some gender differences in the determinants of college attendance. I find that higher non-cognitive skills and college premiums among women account for nearly 90 percent of the gender gap in higher education. Interestingly, non-cognitive factors continue to influence college enrollment after controlling for high school achievement.
-
The impact of high school graduation exams on student achievement and dropout rates is examined. Using data from the National Educational Longitudinal Survey (NELS), this analysis is able to control for prior student achievement and a variety of other student, school, and state characteristics. It was found that graduation tests have no significant impact on 12th-grade math or reading achievement. These results are robust with a variety of specification checks. Although graduation tests have no appreciable effect on the probability of dropping out for the average student, they increase the probability of dropping out among the lowest ability students. These results suggest that policymakers would be well advised to rethink current graduation test policies.
-
The primary goal of this article is to examine how multicultural education can alter the learning environment of a school and thereby influence student relations, attitudes, and behaviors. The study generated three primary findings. (1) By creating an engaging and close-knit learning environment, activities that recognize and celebrate cultural diversity can increase motivation, effort, and school identification in minority students. (2) Despite good intentions to lessen divisions among groups, multicultural education may provide terrain for intergroup conflict, particularly among different minority groups. (3) The ways in which culture itself is discussed and understood can have a significant impact on student relations, attitudes, and behaviors. These findings inform current theories concerning learning environments, minority student achievement, and intergroup relations. Most important, educational theory and practice must pay greater attention to the relationship between "minority" groups rather than focus exclusively on the interaction between the traditionally dominant and subordinate groups.
Book Chapters & Other Publications
-
A Comment on Hoxby's 'Skill Desert' Chapter
-
The Changing Federal Role in School Accountability
-
The Potential and Limits of Federal Policy: A Response to Ladd
-
Measuring Instructor Effectiveness in Higher Education
-
Can Technology Help Promote Equality of Educational Opportunities?
-
Organizing Schools to Improve Student Achievement: Start Times, Grade Configurations and Teacher Assignments
-
Educational Expectations and Attainment
-
Family Income, Neighborhood Poverty and Crime
-
Educational Interventions: Their Effects on the Achievement of Poor Children
-
Improving Educational Outcomes for Poor Children
-
Is Gaining Access to a Selective Elementary School Gaining Ground? Evidence from Randomized Lotteries
-
The Challenges of Staffing Urban Schools with Effective Teachers
-
Do High School Exit Exams Influence Educational Attainment or Labor Market Performance?
-
The Effects of School Size on Parental Involvement and Social Capital
-
Can the Federal Government Improve Education Research?
-
Catching Cheating Teachers: The Results of an Unusual Experiment in Implementing Theory
-
Getting Inside Accountability: Lessons from Chicago
-
A Closer Look at Achievement Gains under High-Stakes Testing in Chicago
-
Summer in the City: Achievement Gains in Chicago's Summer Bridge Program
-
Evaluating Chicago's Efforts to End Social Promotion
Resting Papers
Papers no longer under active revision but available upon request.
-
This paper reports findings from a nationally representative survey of K-12 teachers in May 2023 that examines the potential long-term impacts of COVID-19 on public schooling. The findings suggest fundamental ways in which school operations, instructional practice and parent-teacher interaction have changed since the pandemic. Some changes seem promising; others suggest caution. While policymakers may not be able to directly influence some of the reported changes in the short run, monitoring the evolution of school practices (and their consequences for children) will position educational leaders to help teachers and students address the consequences of the COVID-19 pandemic going forward.
-
In this paper, we leverage randomized admissions lotteries to estimate the impact of attending a National Heritage Academy (NHA) charter school. NHA is the fourth largest for-profit charter operator in the country, enrolling more than 56,000 students in 86 schools across 9 states. Unlike several of the other large for-profit companies that operate virtual charters, NHA only has standard bricks-and-mortar schools. Our estimates indicate that attending a NHA charter school for one additional year is associated with a 0.04 standard deviation increase in math achievement. Effects on other outcomes are smaller and not statistically significant. In contrast to most prior charter school research that finds the largest benefits for low-income, underrepresented minorities in urban areas, the benefits of attending an NHA charter network are concentrated among non-poor students attending charter schools outside urban areas. Using data from a survey of school administrators in traditional public and charter schools, we document several aspects of school organization, culture and instructional practice that might explain these positive effects.
-
This paper examines the relative labor market gains experienced by first-time college students who enrolled in five community colleges in Michigan in 2003 and 2004. It tracks credentials, credits, earnings, and employment for these students through 2011 and compares the labor market outcomes of those who earned a credential (associate degree or certificate) and those who enrolled but did not earn a credential. The data sources consist of administrative records data from the colleges, Unemployment Insurance earnings data from the State of Michigan, and enrollment and graduation data from the National Student Clearinghouse. The analytic sample consists of 20,581 students. The authors find that students who were awarded a long-term certificate (referred to as a "diploma" in some states, including North Carolina) earned $2,500 to $3,600 more per year than did those without a credential, with the larger returns concentrated among men. For associate degrees, the estimated returns were $9,400 for women and $5,600 for men. Women saw little gain when awarded a short-term certificate, while men gained $5,200 per year. Estimated returns were highest in health-related and technical fields.
-
Pay for performance (PFP) is once again gaining popularity within education. This study examines teacher attitudes toward PFP policies, and how these views vary by teacher experience, subject area specialization, grade level(s) taught, educational background, personality characteristics, risk and time preferences, and feelings of efficacy. Data were collected through a voluntary, online survey instrument fielded over a two-week period at the end of the 2006-2007 school year. The sample comprised all full-time instructional personnel in 199 traditional public and magnet schools in a large, urban school district in Florida. Results suggest only modest support for PFP policies among teachers. We detect some association between teacher demographics and views on PFP policies. The most striking finding is how little teachers appear to understand how the two most recent PFP initiatives in Florida operate.
-
This paper explores the phenomenon referred to as test score inflation, which occurs when achievement gains on "high-stakes" exams outpace improvements on "low-stakes" tests. The first part of the paper documents the extent to which student performance trends on state assessments differ from those on the National Assessment of Educational Progress (NAEP). I find evidence of considerable test score inflation in several different states, including those with quite different state testing systems. The second part of the paper is a case study of Texas that uses detailed item-level data from the Texas Assessment of Academic Skills (TAAS) and the NAEP to explore why performance trends differed across these exams during the 1990s. I find that the differential improvement on the TAAS cannot be explained by several important differences across the exams (e.g., the NAEP includes open-response items, many NAEP multiple-choice items require/permit the use of calculators, rulers, protractors or other manipulative). I find that skill and format differences across exams explain the disproportionate improvement in the TAAS for fourth graders, although these differences cannot explain the time trends for eighth graders.