Model Specification in Mixed-Effects Models A Focus on Random Effects
Main Article Content
Abstract
Mixed-effect models are flexible tools for researchers in a myriad of fields, but that flexibility comes at the cost of complexity and if users are not careful in how their model is specified, they could be making faulty inferences from their data. We argue that there is significant confusion around appropriate random effects to be included in a model given the study design, with researchers generally being better at specifying the fixed effects of a model, which map onto to their research hypotheses. To that end, we present an instructive framework for evaluating the random effects of a model in three different situations: (1) longitudinal designs; (2) factorial repeated measures; and (3) when dealing with multiple sources of variance. We provide worked examples with open-access code and data in an online repository. We think this framework will be helpful for students and researchers who are new to mixed effect models, and to reviewers who may have to evaluate a novel model as part of their review.
Metrics
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright of their manuscripts, and all Open Access articles are distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided that the original work is properly cited.
References
Fisher RA. The Correlation between Relatives on the Supposition of Mendelian Inheritance. Earth Environ Sci Trans R Soc Edinb. 1919;52:399–433.
Robinson GK. That BLUP is a Good Thing: The Estimation of Random Effects. Stat Sci. 1991 Feb;6(1):15–32.
McLean RA, Sanders WL, Stroup WW. A Unified Approach to Mixed Linear Models. Am Stat. 1991;45(1):54–64.
Singer J, Willett J. Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford university press; 2003.
Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. Vol. 1. SAGE; 2002.
Garcia TP, Marder K. Statistical Approaches to Longitudinal Data Analysis in Neurodegenerative Diseases: Huntington’s Disease as a Model. Curr Neurol Neurosci Rep. 2017 Feb;17(2):14.
Lohse KR, Shen J, Kozlowski AJ. Modeling Longitudinal Outcomes: A Contrast of Two Methods. J Mot Learn Dev. 2020 Jan 27;8(1):145–65.
Long JD. Longitudinal data analysis for the behavioral sciences using R. Sage; 2012.
Snijders TA, Bosker RJ. Multilevel analysis: An introduction to basic and advanced multilevel modeling. SAGE; 2011.
Sera F, Armstrong B, Blangiardo M, Gasparrini A. An extended mixed-effects framework for meta-analysis. Stat Med. 2019;38(29):5429–44.
Van der Elst W, Molenberghs G, Hilgers RD, Verbeke G, Heussen N. Estimating the reliability of repeatedly measured endpoints based on linear mixed-effects models. A tutorial. Pharm Stat. 2016;15(6):486–93.
Kenny DA, Korchmaros JD, Bolger N. Lower level mediation in multilevel models. Psychol Methods. 2003;8(2):115.
Preacher KJ, Zyphur MJ, Zhang Z. A general multilevel SEM framework for assessing multilevel mediation. Psychol Methods. 2010;15(3):209.
Agresti A. Foundations of Linear and Generalized Linear Models. John Wiley & Sons; 2015. 469 p.
Barr DJ. Random effects structure for testing interactions in linear mixed-effects models. Front Psychol. 2013;4:328.
Bates D, Kliegl R, Vasishth S, Baayen H. Parsimonious Mixed Models. ArXiv150604967 Stat [Internet]. 2018 May 26 [cited 2022 Jan 28]; Available from: http://arxiv.org/abs/1506.04967
Silk MJ, Harrison XA, Hodgson DJ. Perils and pitfalls of mixed-effects regression models in biology. PeerJ. 2020 Aug 12;8:e9522.
Bolker B, Brooks ME, Clark CJ, Geange SW, Poulsen JR, Stevens MHH, et al. Generalized linear mixed models: a practical guide for ecology and evolution. Trends Ecol Evol. 2009;24(3):127–35.
Brown VA. An Introduction to Linear Mixed-Effects Modeling in R. Adv Methods Pract Psychol Sci. 2021;4(1):2515245920960351.
Frossard J, Renaud O. The correlation structure of mixed effects models with crossed random effects in controlled experiments. ArXiv Prepr ArXiv190310766. 2019;
Singmann H, Kellen D. An introduction to mixed models for experimental psychology. In: New methods in cognitive psychology. Routledge; 2019. p. 4–31.
Nelder JA. What is the mixed-models controversy? Int Stat Rev. 2008;76(1):134–5.
Voss DT. Resolving the mixed models controversy. Am Stat. 1999;53(4):352–6.
Senn S, Nelder J. A Conversation with John Nelder. Stat Sci. 2003;18(1):118–31.
Faraway JJ. Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. CRC press; 2016.
Pinheiro J, Bates D. Mixed-effects models in S and S-PLUS. Springer science & business media; 2006.
Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. ArXiv Prepr ArXiv14065823. 2014;
Cohen J, Cohen P, West SG, Aiken LS. Applied multiple regression/correlation analysis for the behavioral sciences. Routledge; 2013.
Fox J. Applied regression analysis and generalized linear models. Sage Publications; 2015.
Fox J, Weisberg S. An R companion to applied regression. Sage publications; 2018.
Brysbaert M, Stevens M. Power Analysis and Effect Size in Mixed Effects Models: A Tutorial. J Cogn. 1(1):9.
Westfall J, Kenny DA, Judd CM. Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. J Exp Psychol Gen. 2014;143(5):2020–45.
R Core Team. R: A language and environment for statistical computing. [Internet]. R Foundation for Statistical Computing; 2021. Available from: https://www.R-project.org/
Pinheiro J, Bates D, DebRoy S, Sarkar D, NLME L. Nonlinear mixed-effects models. Mixed-Effects; 2012.
Kuznetsova A, Brockhoff PB, Christensen RH. lmerTest package: tests in linear mixed effects models. J Stat Softw. 2017;82(1):1–26.
Satterthwaite FE. Synthesis of variance. Psychometrika. 1941;6(5):309–16.
Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond “p< 0.05.” Vol. 73, The American Statistician. Taylor & Francis; 2019. p. 1–19.
Benjamin DJ, Berger JO. Three recommendations for improving the use of p-values. Am Stat. 2019;73(sup1):186–91.
Goodman SN. Why is getting rid of p-values so hard? Musings on science and statistics. Am Stat. 2019;73(sup1):26–30.
Lawrence MA. ez: Easy Analysis and Visualization of Factorial Experiments [Internet]. 2016. Available from: https://CRAN.R-project.org/package=ez
Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics. 1997;983–97.
Judd CM, McClelland GH, Ryan CS. Data analysis: A model comparison approach to regression, ANOVA, and beyond. Routledge; 2017.
Gurka MJ, Edwards LJ, Muller KE. Avoiding bias in mixed model inference for fixed effects. Stat Med. 2011 Sep 30;30(22):2696–707.
Kauermann G, Carroll RJ. A note on the efficiency of sandwich covariance matrix estimation. J Am Stat Assoc. 2001;96(456):1387–96.
Efron B, Tibshirani RJ. An introduction to the bootstrap. CRC press; 1994.
Mason F, Cantoni E, Ghisletta P. Parametric and semi-parametric bootstrap-based confidence intervals for robust linear mixed models. Methodology. 2021;17(4):271–95.
Johnson PC. Extension of Nakagawa & Schielzeth’s R2GLMM to random slopes models. Methods Ecol Evol. 2014;5(9):944–6.
Nakagawa S, Schielzeth H. A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol Evol. 2013;4(2):133–42.
Kreft IGG, Leeuw J de. Introducing Multilevel Modeling. SAGE; 1998. 164 p.
Gelman A. Analysis of variance—why it is more important than ever. Ann Stat. 2005;33(1):1–53.
Üstün TB, Chatterji S, Bickenbach J, Kostanjsek N, Schneider M. The International Classification of Functioning, Disability and Health: a new tool for understanding disability and health. Disabil Rehabil. 2003;25(11–12):565–71.
Imrie R. Demystifying disability: a review of the International Classification of Functioning, Disability and Health. Sociol Health Illn. 2004;26(3):287–305.
Dunlop DD. Regression for Longitudinal Data: A Bridge from Least Squares Regression. Am Stat. 1994 Nov 1;48(4):299–303.
Sainani K. The Importance of Accounting for Correlated Observations. PM&R. 2010 Sep;2(9):858–61.
McNeish D. Small sample methods for multilevel modeling: A colloquial elucidation of REML and the Kenward-Roger correction. Multivar Behav Res. 2017;52(5):661–70.
Venables WN, Ripley BD. Modern applied statistics with S-PLUS. Springer Science & Business Media; 2002.
Brown VA, Strand JF. About Face: Seeing the Talker Improves Spoken Word Recognition but Increases Listening Effort. J Cogn. 2019 Nov 22;2(1):44.
Hodges J. Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects. CRC Press; 2013.
Bolker B. Glmm FAQ. URL: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html
Altman DG, Bland JM. Statistics notes: Absence of evidence is not evidence of absence. Bmj. 1995;311(7003):485.