Model Specification in Mixed-Effects Models A Focus on Random Effects

Main Article Content

Keith Lohse
Allan J. Kozlowski
Michael J. Strube


Mixed-effect models are flexible tools for researchers in a myriad of fields, but that flexibility comes at the cost of complexity and if users are not careful in how their model is specified, they could be making faulty inferences from their data. We argue that there is significant confusion around appropriate random effects to be included in a model given the study design, with researchers generally being better at specifying the fixed effects of a model, which map onto to their research hypotheses. To that end, we present an instructive framework for evaluating the random effects of a model in three different situations: (1) longitudinal designs; (2) factorial repeated measures; and (3) when dealing with multiple sources of variance. We provide worked examples with open-access code and data in an online repository. We think this framework will be helpful for students and researchers who are new to mixed effect models, and to reviewers who may have to evaluate a novel model as part of their review.


Metrics Loading ...

Article Details

How to Cite
Lohse, K., Kozlowski, A., & Strube, M. J. (2023). Model Specification in Mixed-Effects Models: A Focus on Random Effects. Communications in Kinesiology, 1(5). (Original work published November 20, 2023)


Agresti, A. (2015). Foundations of Linear and Generalized Linear Models . Wiley.

Barr, D. J. (2013). Random effects structure for testing interactions in linear mixed-effects models. Frontiers in Psychology, 4.

Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious Mixed Models (Version 2). arXiv.

Bates, D., Machler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using {lme4} (Vol. 67, pp. 1--48).

Benjamin, D. J., & Berger, J. O. (2019). Three Recommendations for Improving the Use of p-Values. The American Statistician, 73(sup1), 186–191.

Bolker, B. (2023). GLMM FAQ.

Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009). Generalized linear mixed models: a practical guide for ecology and evolution. Trends in Ecology & Evolution, 24(3), 127–135.

Brown, V. A. (2021). An Introduction to Linear Mixed-Effects Modeling in R. Advances in Methods and Practices in Psychological Science, 4(1), 251524592096035.

Brown, V. A., & Strand, J. F. (2019, May 22). About face: Seeing the talker improves spoken word recognition but increases listening effort. Center for Open Science.

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum Associates Publishers.

Dunlop, D. D. (1994). Regression for Longitudinal Data: A Bridge from Least Squares Regression. The American Statistician, 48(4), 299.

Efron, B., & Tibshirani, R. J. (1993). Bootstrap standard errors: some examples. In An Introduction to the Bootstrap (pp. 60–85). Springer US.

Faraway, J. J. (2016). Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC Press.

Fisher, R. A. (1919). XV.—The Correlation between Relatives on the Supposition of Mendelian Inheritance. Transactions of the Royal Society of Edinburgh, 52(2), 399–433.

Fox, J. (2016). Applied regression analysis and generalized linear models. SAGE.

Fox, J., & Weisburg, S. (2011). An R companion to applied regression. SAGE.

Frossard, J., & Renaud, O. (2019). Choosing the correlation structure of mixed effect models for experiments with stimuli (Version 3). arXiv.

Garcia, T. P., & Marder, K. (2017). Statistical Approaches to Longitudinal Data Analysis in Neurodegenerative Diseases: Huntington’s Disease as a Model. Current Neurology and Neuroscience Reports, 17(2).

Gelman, A. (2005). Analysis of variance—why it is more important than ever. The Annals of Statistics, 33(1).

Goodman, S. N. (2019). Why is Getting Rid of P-Values So Hard? Musings on Science and Statistics. The American Statistician, 73(sup1), 26–30.

Gurka, M. J., Edwards, L. J., & Muller, K. E. (2011). Avoiding bias in mixed model inference for fixed effects. Statistics in Medicine, 30(22), 2696–2707.

Hodges, J. S. (2016). Richly Parameterized Linear Models. Chapman and Hall/CRC.

Imrie, R. (2004). Demystifying disability: a review of the International Classification of Functioning, Disability and Health. Sociology of Health & Illness, 26(3), 287–305.

Johnson, P. C. D. (2014). Extension of Nakagawa-Schielzeth’s R2GLMM to random slopes models. Methods in Ecology and Evolution, 5(9), 944–946.

Judd, C. M., McClelland, G. H., & Ryan, C. S. (2017). Data Analysis. Routledge.

Kauermann, G., & Carroll, R. J. (2001). A Note on the Efficiency of Sandwich Covariance Matrix Estimation. Journal of the American Statistical Association, 96(456), 1387–1396.

Kenny, D. A., Korchmaros, J. D., & Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8(2), 115–128.

Kenward, M. G., & Roger, J. H. (1997). Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood. Biometrics, 53(3), 983.

Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). {lmerTest} Package: Tests in Linear Mixed Effects Models (Vol. 82, pp. 1--26).

Lawrence, M. A. (2016). ez: Easy Analysis and Visualization of Factorial Experiments.

Lohse, K., Shen, J., & Kozlowski, A. J. (2020, January 29). Modeling Longitudinal Outcomes: A Contrast of Two Methods. Center for Open Science.

Long, J. D. (2012). Longitudinal Data Analysis for the Behavioral Sciences Using R . SAGE.

Mason, F., Cantoni, E., & Ghisletta, P. (2021). Parametric and semi-parametric bootstrap-based confidence intervals for robust linear mixed models. Methodology, 17(4), 271–295.

McLean, R. A., Sanders, W. L., & Stroup, W. W. (1991). A Unified Approach to Mixed Linear Models. The American Statistician, 45(1), 54.

McNeish, D. (2017). Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction. Multivariate Behavioral Research, 52(5), 661–670.

Nakagawa, S., & Schielzeth, H. (2012). A general and simple method for obtaining R2 from generalized linear mixed‐effects models. Methods in Ecology and Evolution, 4(2), 133–142.

Nelder, J. A. (2007). What is the Mixed-Models Controversy? International Statistical Review, 0(0), 071121035909002–???

Pinheiro, J., & Bates, D. (2006). Mixed-effects models in S and S-PLUS. Springer.

Pinheiro, J., Bates, D., & R Core Team. (2022). nlme: Linear and Nonlinear Mixed Effects Models.

Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15(3), 209–233.

R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models : applications and data analysis methods. SAGE.

Robinson, G. K. (1991). That BLUP is a Good Thing: The Estimation of Random Effects. Statistical Science, 6(1).

Sainani, K. (2010). The Importance of Accounting for Correlated Observations. PMR, 2(9), 858–861.

Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6(5), 309–316.

Senn, S. (2003). A Conversation with John Nelder. Statistical Science, 18(1).

Sera, F., Armstrong, B., Blangiardo, M., & Gasparrini, A. (2019). An extended mixed‐effects framework for meta‐analysis. Statistics in Medicine, 38(29), 5429–5444.

Silk, M. J., Harrison, X. A., & Hodgson, D. J. (2020). Perils and pitfalls of mixed-effects regression models in biology. PeerJ, 8, e9522.

Singer, J. D., & Willett, J. B. (2003). Exploring Longitudinal Data on Change. In Applied Longitudinal Data Analysis (pp. 16–44). Oxford University PressNew York.

Singmann, H., & Kellen, D. (2019). An Introduction to Mixed Models for Experimental Psychology. In New Methods in Cognitive Psychology (pp. 4–31). Routledge.

Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: an introduction to basic and advanced multilevel modeling. SAGE.

Üstun, T. B., Chatterji, S., Bickenbach, J., Kostanjsek, N., & Schneider, M. (2003). The International Classification of Functioning, Disability and Health: a new tool for understanding disability and health. Disability and Rehabilitation, 25(11-12), 565–571.

Van der Elst, W., Molenberghs, G., Hilgers, R., Verbeke, G., & Heussen, N. (2016). Estimating the reliability of repeatedly measured endpoints based on linear mixed‐effects models. A tutorial. Pharmaceutical Statistics, 15(6), 486–493.

Venables, W. N., & Ripley, B. D. (1997). Modern Applied Statistics with S-PLUS. In Statistics and Computing. Springer New York.

Voss, D. T. (1999). Resolving the Mixed Models Controversy. The American Statistician, 53(4), 352.

Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a World Beyond P < 0.05. The American Statistician, 73(sup1), 1–19.

Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143(5), 2020–2045.