Invalid Methods and False Answers: Physics Education Research and the Use of GREs
by
Read this article
- Access statistics
- 2,792 article downloads
- 1,292 complete issue downloads
- Total: 4,084
Abstract
Finding good educational policies requires sound estimates of their potential effects. Methods for making such estimates, i.e., finding causal estimands, have made great progress in the last few decades. Nevertheless, serious errors in causal reasoning have been found previously in papers in a leading physics education journal, Physical Review Physics Education Research. Here I examine three papers from that journal that present explicit methods of causal inference. The methods offered involve major errors, including errors in identifying causal mediation, choosing variables to control for, and imputing missing data. The erroneous methods lead to major underestimation of the predictive power of Graduate Record Exams.
Podcast related to this article: Michael Weissman on GREs in Physics Education Research (EJW Audio, May 2022).