Scholarly Comments on Academic Economics

Invalid Methods and False Answers: Physics Education Research and the Use of GREs

by

Read this article

Access statistics
2,609 article downloads
1,196 complete issue downloads
Total: 3,805

Abstract

Finding good educational policies requires sound estimates of their potential effects. Methods for making such estimates, i.e., finding causal estimands, have made great progress in the last few decades. Nevertheless, serious errors in causal reasoning have been found previously in papers in a leading physics education journal, Physical Review Physics Education Research. Here I examine three papers from that journal that present explicit methods of causal inference. The methods offered involve major errors, including errors in identifying causal mediation, choosing variables to control for, and imputing missing data. The erroneous methods lead to major underestimation of the predictive power of Graduate Record Exams.

Podcast related to this article: Michael Weissman on GREs in Physics Education Research (EJW Audio, May 2022).