Research Repository

An Empirical Unraveling of Lord's Paradox

Xiao, ZhiMin and Higgins, Steve and Kasim, Adetayo (2019) 'An Empirical Unraveling of Lord's Paradox.' Journal of Experimental Education, 87 (1). pp. 17-32. ISSN 0022-0973

An Empirical Unravelling of Lords Paradox.pdf

Download (519kB) | Preview


Lord's Paradox occurs when a continuous covariate is statistically controlled for and the relationship between a continuous outcome and group status indicator changes in both magnitude and direction. This phenomenon poses a challenge to the notion of evidence-based policy, where data are supposed to be self-evident. We examined 50 effect size estimates from 34 large-scale educational interventions and found that impact estimates are affected in magnitude, with or without reversal in sign, when there is substantial baseline imbalance. We also demonstrated that multilevel modeling can ameliorate the divergence in sign and/or magnitude of effect estimation, which, together with project specific knowledge, promises to help those who are presented with conflicting or confusing evidence in decision-making.

Item Type: Article
Additional Information: 23 pages, 2 figures, 3 tables
Uncontrolled Keywords: Evaluation; evidence-based policy; Lord's Paradox; multilevel modelling; RCT
Divisions: Faculty of Science and Health
Faculty of Science and Health > Health and Social Care, School of
SWORD Depositor: Elements
Depositing User: Elements
Date Deposited: 05 Aug 2021 12:34
Last Modified: 18 Aug 2022 12:22

Actions (login required)

View Item View Item