How Different Types of Sampling Bias Impact the Validity and Generalizability of Research Findings Across Various Scientific Disciplines

Abstract

Different types of sampling bias affect research findings by distorting estimates, undermining internal validity, and limiting generalizability. Joyal-Desmarais et al. ¹ show that convenience samples in public health overestimate COVID-19–related behaviors, while Cheung et al. ² report that voluntary adolescent samples tend to underestimate risk behaviors, even if associations remain robust. Sykes et al. ³ note that misaligned sampling in social science distorts both qualitative and quantitative conclusions, and Fanelli et al. ⁴ report that small, early, or highly cited studies inflate effects through publication and selection biases. In ecology, Fourcade et al. ⁵ document that geographic bias produces inaccurate species distribution models, and in genomics, Foulkes et al. ^6,7 demonstrate that selection bias leads to large errors in effect estimates despite partial correction by inverse probability weighting. Similarly, studies in sexual minority health and paleontology indicate that nonprobability and preservational biases restrict valid cross-context inference.

Bias correction methods show mixed results. In public health, demographic covariate adjustment and logistic regression sometimes fail to fully counter nonresponse and selection biases. Systematic sampling in ecology and inverse probability weighting in genomics reduce bias in specific contexts, although not eliminating it entirely. Together, these studies imply that selection, geographic, analytical, preservational, temporal, and publication biases compromise both the internal and external validity of research, with the precise impact and mitigation efficacy varying by discipline and sampling design.