### A Monte Carlo Simulation Study to Assess Estimation Methods in CFA on Ordinal Data

#### Abstract

Likert-type scale data are ordinal data and are commonly used to measure latent constructs in the educational, social, and behavioral sciences. The ordinal observed variables are often treated as continuous variables in factor analysis, which may cause misleading statistical inferences. Two robust estimators, i.e., unweighted least square (ULS) and diagonally weighted least square (DWLS) have been developed to deal with ordinal data in confirmatory factor analysis (CFA). Using synthetic data generated in a Monte Carlo experiment, we study the behavior of these methods (DWLS and ULS) and compare their performance with normal theory-based ML and GLS (generalized least square) under different levels of experimental conditions. The simulation results indicate that both DWLS and ULS yield consistently accurate parameter estimates across all conditions considered in this study. The Likert data can be treated as a continuous variable under ML or GLS when using at least five Likert scale points to produce trivial bias. However, these methods generally fail to provide a satisfactory fit. Empirical studies in the field of psychological measurement data are reported to present how theoretical and statistical instances have to be taken into consideration when ordinal data are used in the CFA model.

**Keywords: **confirmatory factor analysis, diagonally weighted least square, generalized least square, Likert data, maximum likelihood.

#### Full Text:

PDF#### References

B. D. Zumbo, Validity: Foundational Issues and Statistical Methodology. In C. R. Rao & S. Sinharay (eds.), Handbook of Statistics, Vol. 26: Psychometrics(pp. 45-79), Amsterdam: Elsevier Science, 2007.

F. P. Holgado-Tello , M. A. Morata-Ramírez and Barbero-Garcí, "Confirmatory Factor Analysis of Ordinal Variables: A Simulation Study Comparing the Main Estimation Methods," Avances en Psicología Latinoamericana, vol. 36, no. 3, pp. 601-617, 2018.

B. Thompson, Explanatory and Confirmatory Factor Analysis., Washington DC: American Psychological Association, 2004.

K. G. Joreskog and D. Sorbom , Lisrel 8, User’s Reference Guide., Chicago: SSI Inc. (Scientific Software International), 1996.

Y. Li , Confirmatory Factor Analysis with Continuous and Ordinal Data: An Empirical Study of Stress Level, Uppsala University, 2014.

T. A. Brown, Confirmatory factor analysis for applied research, Guilford Press, 2012.

K. A. Bollen, Structural equations with latent variables, New York: Wiley, 1989.

J. F. Finch , S. G. West and D. P. MacKinnon , "Effects of sample size and nonnormality on the estimation of mediated effects in latent variable models," Structural Equation Modeling, vol. 4, pp. 87-107, 1997.

B. O. Muthén and D. Kaplan , "A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on the size of the model," British Journal of Mathematical and Statistical Psychology, vol. 45, pp. 19-30, 1992.

S. G. West, J. F. Finch and P. J. Curran, Structural equation models with nonnormal variables: problems and remedies. In R. H. Hoyle (Ed.), Structural Equation Modeling: concepts, issues and applications (pp. 56-75)., Thousand Oaks (CA): Sage, 1995.

A. Maydeu-Olivares , "Limited information estimation and testing of Thurstonian models for paired comparison data under multiple judgment sampling," Psychometrika, vol. 66, pp. 209-227, 2001.

R. Tate, "A comparison of selected empirical methods for assessing the structure of responses to test items," Applied Psychological Measurement, vol. 27, no. 3, p. 159–203, 2003.

C. G. Forero, A. Maydeu-Olivares and D. Gallardo-Pujol , "Factor analysis with ordinal indicators: a monte Carlo study comparing DWLS and ULS simulation," Structural Equation Modeling, vol. 16, pp. 625-641, 2009.

D. Shi and A. Maydeu-Olivares , "The effect of estimation methods on SEM fit indices," Educational and Psychological Measurement, vol. 80, no. 3, pp. 421-445, 2020.

T. P. Morris , "White IR, Crowther MJ. Using simulation studies to evaluate statistical methods," Statistics in Medicine, vol. 38, pp. 2074-2102, 2019.

A. Satorra, "Robustness issues in structural equation modeling: A review of recent developments," Quality and Quantity, vol. 24, no. 4, p. 367–386, 1990.

M. W. Browne, "Generalized least-squares estimators in the analysis of covariance structures.," South African Statistical Journal, vol. 8, pp. 1-24, 1974.

L. Ding, W. Velicer and L. Harlow , "Effect of estimation methods, number of indicators per factor and improper solutions on structural equation modeling fit indices," Structural Equation Modeling, vol. 2, pp. 119-143, 1995.

U. H. Olsson , T. Foss , S. V. Troye and R. D. Howell , "The performance of ML, GLS, and WLS estimation in structural equation modeling under conditions of misspecification and nonnormality," Structural equation modeling, vol. 7, no. 4, pp. 557-595, 2000.

W. F. Yang , K. G. Jöreskog and H. Luo, "Confirmatory factor analysis of ordinal variables with misspecified models," Structural Equation Modeling, vol. 17, no. 3, p. 392–423, 2010.

B. Muthén, S. H. C. du Toit and D. Spisic , Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes, Unpublished manuscript, 1997.

H. W. Marsh, K. Hau , J. R. Balla and D. Grayson, "Is more ever too much? The number of indicators per factor in confirmatory factor analysis," Multivariate Behavioral Research, vol. 33, p. 181–220, 1998.

P. Paxton , P. J. Curran, K. A. Bollen , J. Kirby and F. Chen, "Monte Carlo experiments: Design and implementation," Structural Equation Modeling A Multidisciplinary Journal, vol. 8, no. 2, pp. 287-312, 2001.

C. DiStefano, "The impact of categorization with confirmatory factor analysis," Structural Equation Modeling, vol. 9, p. 327–346, 2020.

C. -H. Li , "Confirmatory factor analysis with ordinal data: comparing robust maximum likelihood and diagonally weighted least squares," Behav Res, vol. 48, pp. 936-949, 2016.

Y. Rosseel, "lavaan: An R package for structural equation modeling," Journal of Statistical Software, vol. 48, no. 2, pp. 1-36, 2011.

The R Development Core Team, R: A language and environment for statistical computing., Vienna, Austria: : R Foundation for Statistical Computing, 2019.

M. Wallentin, A. H. Nielsen and M. Friis-Olivarius, "The Musical Ear Test, a new reliable test for measuring musical competence," Learning and Individual Differences, vol. 20, pp. 188-196, 2010.

D. Kaplan, "A study of the sampling variability and z-values of parameter estimates from misspecified structural equation models," Multivariate Behavioral Research, vol. 24, p. 41–57, 1989.

P. J. Curran , S. G. West and G. F. Finch , "The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis," Psychological Methods, vol. 1, p. 16–29, 1996.

Open-Source Psychometrics Project, "Raw data from online personality tests," https://openpsychometrics.org/_rawdata/, 2019.

J. A. Taylor, "A personality scale of manifest anxiety," The Journal of Abnormal and Social Psychology, vol. 48, no. 2, pp. 285-290, 1953.

E. Nisbet and J. M. Zelenskix, "The NR-6: A new brief measure of nature relatedness," Frontiers in Psychology, vol. 4, 2013. DOI: 10.3389/fpsyg.2013.00813.

B. Altemeyer, Right-wing authoritarianism, University of Manitoba Press, 1981.

B. Altemeyer, The Authoritarians, University of Manitoba, 2007.

DOI: https://doi.org/10.18860/ca.v7i3.14434

### Refbacks

- There are currently no refbacks.

Copyright (c) 2022 Nina Fitriyati, Madona Yunita Wijaya

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

**Editorial Office**

Mathematics Department,

Universitas Islam Negeri Maulana Malik Ibrahim Malang

Gajayana Street 50 Malang, East Java, Indonesia 65144

Faximile (+62) 341 558933

e-mail: cauchy@uin-malang.ac.id

CAUCHY: Jurnal Matematika Murni dan Aplikasi is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.