Measurement invariance is a complex topic because it has to do both with issues in multi-group factor analysis and because of the conceptual issues involved.
As an example of the latter, your reading this week discusses the measurement of depression in males and females. It is entirely possible that men and women might interpret items such as “I cry a lot” or “Gaining weight” differently. It’s also possible that symptoms or aspects of the disorder differ in their relevance to different groups.
To make this more concrete, let’s consider the structure of the multidimensional health locus of control scale, and the 18 items contained in the tool.
We will want to ask the following 4 questions across two or more known groups:
Importantly, the questions above pertain to variation across groups, not among the items of the scale So for example, you already found that different MHLC items had different loadings on different factors. The question above is whether that pattern of loadings is the same in men and women (or in different cultural groups, risk strata, etc.).
Lavaan makes this process very easy. All one has to do is to specify the model and then indicate the grouping variable that provides information on who is who.
Let’s start by specifying the model. We will use the conceptual model originally used by Wallston, Wallston, and Devellis. This is reflected in the names of the items, although what we call “External” they call “Powerful Others”
First, we’ll just specify a 3-factor model and not distinguish between participants at all. We’ll abbreviate the latent variables as:
# Model Specification
MHLC_mod <- "
# latent variables
IHLC =~ Internal1 + Internal2 + Internal3 + Internal4 + Internal5 + Internal6
PHLC =~ External1 + External2 + External3 + External4 + External5 + External6
CHLC =~ Chance1 + Chance2 + Chance3 + Chance4 + Chance5 + Chance6
"
# Fit the model
mod1 <- cfa(MHLC_mod, data = MHLC)Now check the fit and the parameter estimates
summary(mod1, fit.measures = T, standardized = T)## lavaan 0.6-19 ended normally after 58 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 39
##
## Used Total
## Number of observations 822 844
##
## Model Test User Model:
##
## Test statistic 654.460
## Degrees of freedom 132
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 2404.635
## Degrees of freedom 153
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.768
## Tucker-Lewis Index (TLI) 0.731
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -27289.795
## Loglikelihood unrestricted model (H1) -26962.565
##
## Akaike (AIC) 54657.590
## Bayesian (BIC) 54841.348
## Sample-size adjusted Bayesian (SABIC) 54717.499
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.069
## 90 Percent confidence interval - lower 0.064
## 90 Percent confidence interval - upper 0.075
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.000
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.071
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## IHLC =~
## Internal1 1.000 0.567 0.321
## Internal2 1.220 0.175 6.954 0.000 0.692 0.506
## Internal3 1.117 0.185 6.052 0.000 0.634 0.357
## Internal4 1.088 0.152 7.155 0.000 0.617 0.560
## Internal5 1.427 0.210 6.803 0.000 0.809 0.473
## Internal6 1.629 0.219 7.442 0.000 0.924 0.710
## PHLC =~
## External1 1.000 0.614 0.343
## External2 1.468 0.201 7.293 0.000 0.901 0.479
## External3 1.312 0.187 7.022 0.000 0.805 0.436
## External4 1.559 0.197 7.929 0.000 0.957 0.632
## External5 1.450 0.188 7.721 0.000 0.891 0.569
## External6 1.365 0.173 7.890 0.000 0.838 0.619
## CHLC =~
## Chance1 1.000 0.817 0.445
## Chance2 0.661 0.105 6.307 0.000 0.540 0.314
## Chance3 0.817 0.112 7.281 0.000 0.668 0.388
## Chance4 1.453 0.159 9.111 0.000 1.186 0.638
## Chance5 0.969 0.121 8.031 0.000 0.791 0.460
## Chance6 1.129 0.130 8.713 0.000 0.922 0.548
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## IHLC ~~
## PHLC 0.144 0.030 4.859 0.000 0.414 0.414
## CHLC 0.002 0.024 0.088 0.930 0.005 0.005
## PHLC ~~
## CHLC 0.217 0.040 5.376 0.000 0.432 0.432
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .Internal1 2.799 0.145 19.339 0.000 2.799 0.897
## .Internal2 1.390 0.080 17.399 0.000 1.390 0.744
## .Internal3 2.752 0.144 19.083 0.000 2.752 0.873
## .Internal4 0.834 0.051 16.410 0.000 0.834 0.687
## .Internal5 2.270 0.127 17.882 0.000 2.270 0.776
## .Internal6 0.837 0.071 11.814 0.000 0.837 0.495
## .External1 2.837 0.147 19.288 0.000 2.837 0.883
## .External2 2.723 0.151 18.045 0.000 2.723 0.770
## .External3 2.760 0.149 18.524 0.000 2.760 0.810
## .External4 1.376 0.090 15.245 0.000 1.376 0.600
## .External5 1.653 0.099 16.663 0.000 1.653 0.676
## .External6 1.134 0.073 15.596 0.000 1.134 0.617
## .Chance1 2.708 0.152 17.830 0.000 2.708 0.802
## .Chance2 2.672 0.139 19.202 0.000 2.672 0.902
## .Chance3 2.514 0.136 18.524 0.000 2.514 0.849
## .Chance4 2.047 0.153 13.396 0.000 2.047 0.593
## .Chance5 2.333 0.132 17.607 0.000 2.333 0.788
## .Chance6 1.982 0.124 15.955 0.000 1.982 0.700
## IHLC 0.321 0.081 3.982 0.000 1.000 1.000
## PHLC 0.377 0.087 4.325 0.000 1.000 1.000
## CHLC 0.667 0.123 5.429 0.000 1.000 1.000Obviously, the fit is poor as you all may have figured out using your EFA results (i.e., no one identified this as the EFA model of choice)
Let’s instead try something that fits with a few of your results
# Model Specification
MHLC_mod2 <- "
# latent variables
IHLC =~ Internal1 + Internal3 + Internal5 + Internal6
PHLC =~ External1 + External4 + External6
CHLC =~ Chance1 + Chance4 + Chance6
"
# Fit the model
mod2 <- cfa(MHLC_mod2, data = MHLC)
summary(mod2, fit.measures = T)## lavaan 0.6-19 ended normally after 66 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 23
##
## Used Total
## Number of observations 831 844
##
## Model Test User Model:
##
## Test statistic 186.078
## Degrees of freedom 32
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 1001.909
## Degrees of freedom 45
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.839
## Tucker-Lewis Index (TLI) 0.774
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -15529.176
## Loglikelihood unrestricted model (H1) -15436.137
##
## Akaike (AIC) 31104.352
## Bayesian (BIC) 31212.973
## Sample-size adjusted Bayesian (SABIC) 31139.933
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.076
## 90 Percent confidence interval - lower 0.066
## 90 Percent confidence interval - upper 0.087
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.285
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.060
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Internal1 1.000
## Internal3 1.554 0.260 5.990 0.000
## Internal5 2.051 0.327 6.281 0.000
## Internal6 1.415 0.226 6.263 0.000
## PHLC =~
## External1 1.000
## External4 1.468 0.188 7.792 0.000
## External6 1.562 0.210 7.425 0.000
## CHLC =~
## Chance1 1.000
## Chance4 1.917 0.327 5.867 0.000
## Chance6 1.157 0.164 7.064 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.135 0.031 4.381 0.000
## CHLC 0.034 0.022 1.526 0.127
## PHLC ~~
## CHLC 0.108 0.031 3.466 0.001
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.834 0.148 19.157 0.000
## .Internal3 2.459 0.146 16.847 0.000
## .Internal5 1.719 0.151 11.386 0.000
## .Internal6 1.124 0.082 13.644 0.000
## .External1 2.786 0.147 18.888 0.000
## .External4 1.397 0.117 11.915 0.000
## .External6 0.811 0.114 7.138 0.000
## .Chance1 2.870 0.164 17.511 0.000
## .Chance4 1.613 0.303 5.328 0.000
## .Chance6 2.174 0.152 14.315 0.000
## IHLC 0.286 0.082 3.506 0.000
## PHLC 0.416 0.095 4.371 0.000
## CHLC 0.498 0.120 4.135 0.000It’s not an awful fit, but not what we’d usually like to see. Regardless, we will see if these parameters differ across men and women
The first step in invariance testing is configural invariance. The main idea is that we now allow there to be two covariance matrices, one for each group, and the groups are constrained to have the same structure. That is, the model we just fit will be used for both groups. This will usually be the first model you fit, though it is advisable to make sure you have your best fitting model first.
# how many men and women
table(MHLC$SEX)##
## Female Male
## 475 369# fit the model specifying "SEX" as the grouping variable
config_mod <- cfa(MHLC_mod2, data = MHLC, group = "SEX")
summary(config_mod, fit.measures = T)## lavaan 0.6-19 ended normally after 122 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 66
##
## Number of observations per group: Used Total
## Female 464 475
## Male 367 369
##
## Model Test User Model:
##
## Test statistic 232.005
## Degrees of freedom 64
## P-value (Chi-square) 0.000
## Test statistic for each group:
## Female 137.036
## Male 94.969
##
## Model Test Baseline Model:
##
## Test statistic 1060.403
## Degrees of freedom 90
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.827
## Tucker-Lewis Index (TLI) 0.757
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -15493.684
## Loglikelihood unrestricted model (H1) -15377.681
##
## Akaike (AIC) 31119.368
## Bayesian (BIC) 31431.061
## Sample-size adjusted Bayesian (SABIC) 31221.468
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.079
## 90 Percent confidence interval - lower 0.069
## 90 Percent confidence interval - upper 0.091
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.483
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.061
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
##
## Group 1 [Female]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Internal1 1.000
## Internal3 1.516 0.322 4.713 0.000
## Internal5 2.376 0.497 4.779 0.000
## Internal6 1.246 0.261 4.770 0.000
## PHLC =~
## External1 1.000
## External4 1.478 0.271 5.460 0.000
## External6 1.552 0.296 5.238 0.000
## CHLC =~
## Chance1 1.000
## Chance4 2.766 0.751 3.684 0.000
## Chance6 1.304 0.252 5.164 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.108 0.036 3.008 0.003
## CHLC 0.009 0.021 0.424 0.671
## PHLC ~~
## CHLC 0.089 0.036 2.470 0.014
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 3.877 0.080 48.218 0.000
## .Internal3 2.920 0.079 36.933 0.000
## .Internal5 3.744 0.079 47.162 0.000
## .Internal6 4.808 0.062 77.164 0.000
## .External1 4.002 0.084 47.647 0.000
## .External4 4.179 0.070 59.721 0.000
## .External6 4.784 0.064 74.817 0.000
## .Chance1 3.190 0.085 37.504 0.000
## .Chance4 3.185 0.087 36.429 0.000
## .Chance6 2.545 0.077 33.100 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.717 0.189 14.380 0.000
## .Internal3 2.250 0.179 12.557 0.000
## .Internal5 1.325 0.240 5.513 0.000
## .Internal6 1.362 0.112 12.177 0.000
## .External1 2.872 0.204 14.068 0.000
## .External4 1.394 0.164 8.476 0.000
## .External6 0.929 0.161 5.765 0.000
## .Chance1 3.019 0.216 13.996 0.000
## .Chance4 0.966 0.623 1.551 0.121
## .Chance6 2.170 0.199 10.907 0.000
## IHLC 0.283 0.104 2.718 0.007
## PHLC 0.402 0.130 3.104 0.002
## CHLC 0.337 0.128 2.625 0.009
##
##
## Group 2 [Male]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Internal1 1.000
## Internal3 1.260 0.302 4.177 0.000
## Internal5 1.468 0.326 4.502 0.000
## Internal6 1.472 0.324 4.542 0.000
## PHLC =~
## External1 1.000
## External4 1.533 0.283 5.427 0.000
## External6 1.591 0.304 5.241 0.000
## CHLC =~
## Chance1 1.000
## Chance4 1.506 0.351 4.292 0.000
## Chance6 1.017 0.213 4.765 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.169 0.053 3.217 0.001
## CHLC 0.039 0.043 0.919 0.358
## PHLC ~~
## CHLC 0.146 0.055 2.668 0.008
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 3.706 0.094 39.355 0.000
## .Internal3 3.439 0.095 36.185 0.000
## .Internal5 3.896 0.089 43.797 0.000
## .Internal6 4.970 0.065 76.488 0.000
## .External1 4.311 0.091 47.217 0.000
## .External4 4.431 0.079 56.103 0.000
## .External6 4.910 0.069 71.565 0.000
## .Chance1 3.112 0.096 32.425 0.000
## .Chance4 2.766 0.094 29.550 0.000
## .Chance6 2.703 0.090 30.149 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.893 0.230 12.600 0.000
## .Internal3 2.742 0.230 11.901 0.000
## .Internal5 2.128 0.204 10.447 0.000
## .Internal6 0.768 0.132 5.819 0.000
## .External1 2.656 0.210 12.668 0.000
## .External4 1.342 0.164 8.176 0.000
## .External6 0.709 0.148 4.772 0.000
## .Chance1 2.734 0.253 10.824 0.000
## .Chance4 1.750 0.351 4.986 0.000
## .Chance6 2.282 0.228 10.001 0.000
## IHLC 0.361 0.139 2.603 0.009
## PHLC 0.402 0.134 2.996 0.003
## CHLC 0.646 0.211 3.060 0.002First of all, we see that the output now gives us two sets of results, one for men, and the other for women. We also see that the \(\chi^{2}\) value is lower for the male group. Again, the overall fit is not great, but we’ll proceed.
The next step is to now constrain the loadings to be equal. We call this metric invariance since this means that the two latent variables will now have the same scale, though possibly different means and variances.
We’ll test whether the fit got significantly poorer with a likelihood ratio test. A small p-value means we reject
\[ H_{0}: \text{the constrained model fits well} \]
# fit the model specifying "SEX" as the grouping variable and the group.equal option
metric_mod <- cfa(MHLC_mod2, data = MHLC, group = "SEX", group.equal = "loadings")
summary(metric_mod, fit.measures = T)## lavaan 0.6-19 ended normally after 84 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 66
## Number of equality constraints 7
##
## Number of observations per group: Used Total
## Female 464 475
## Male 367 369
##
## Model Test User Model:
##
## Test statistic 238.138
## Degrees of freedom 71
## P-value (Chi-square) 0.000
## Test statistic for each group:
## Female 139.400
## Male 98.738
##
## Model Test Baseline Model:
##
## Test statistic 1060.403
## Degrees of freedom 90
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.828
## Tucker-Lewis Index (TLI) 0.782
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -15496.751
## Loglikelihood unrestricted model (H1) -15377.681
##
## Akaike (AIC) 31111.501
## Bayesian (BIC) 31390.137
## Sample-size adjusted Bayesian (SABIC) 31202.773
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.075
## 90 Percent confidence interval - lower 0.065
## 90 Percent confidence interval - upper 0.086
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.239
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.062
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
##
## Group 1 [Female]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Intrnl1 1.000
## Intrnl3 (.p2.) 1.524 0.248 6.142 0.000
## Intrnl5 (.p3.) 2.090 0.326 6.404 0.000
## Intrnl6 (.p4.) 1.324 0.208 6.368 0.000
## PHLC =~
## Extrnl1 1.000
## Extrnl4 (.p6.) 1.499 0.195 7.682 0.000
## Extrnl6 (.p7.) 1.572 0.214 7.330 0.000
## CHLC =~
## Chance1 1.000
## Chance4 (.p9.) 2.072 0.358 5.781 0.000
## Chance6 (.10.) 1.181 0.166 7.101 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.120 0.033 3.604 0.000
## CHLC 0.013 0.028 0.463 0.644
## PHLC ~~
## CHLC 0.100 0.036 2.768 0.006
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 3.877 0.080 48.182 0.000
## .Internal3 2.920 0.080 36.657 0.000
## .Internal5 3.744 0.079 47.641 0.000
## .Internal6 4.808 0.063 76.706 0.000
## .External1 4.002 0.084 47.701 0.000
## .External4 4.179 0.070 59.590 0.000
## .External6 4.784 0.064 74.900 0.000
## .Chance1 3.190 0.086 37.190 0.000
## .Chance4 3.185 0.087 36.535 0.000
## .Chance6 2.545 0.077 33.166 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.702 0.188 14.391 0.000
## .Internal3 2.243 0.176 12.743 0.000
## .Internal5 1.545 0.186 8.312 0.000
## .Internal6 1.293 0.109 11.893 0.000
## .External1 2.873 0.201 14.301 0.000
## .External4 1.399 0.144 9.746 0.000
## .External6 0.923 0.134 6.881 0.000
## .Chance1 2.928 0.213 13.745 0.000
## .Chance4 1.441 0.365 3.955 0.000
## .Chance6 2.056 0.179 11.472 0.000
## IHLC 0.302 0.087 3.490 0.000
## PHLC 0.393 0.096 4.111 0.000
## CHLC 0.486 0.126 3.861 0.000
##
##
## Group 2 [Male]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Intrnl1 1.000
## Intrnl3 (.p2.) 1.524 0.248 6.142 0.000
## Intrnl5 (.p3.) 2.090 0.326 6.404 0.000
## Intrnl6 (.p4.) 1.324 0.208 6.368 0.000
## PHLC =~
## Extrnl1 1.000
## Extrnl4 (.p6.) 1.499 0.195 7.682 0.000
## Extrnl6 (.p7.) 1.572 0.214 7.330 0.000
## CHLC =~
## Chance1 1.000
## Chance4 (.p9.) 2.072 0.358 5.781 0.000
## Chance6 (.10.) 1.181 0.166 7.101 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.132 0.037 3.583 0.000
## CHLC 0.071 0.033 2.154 0.031
## PHLC ~~
## CHLC 0.119 0.040 2.954 0.003
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 3.706 0.094 39.392 0.000
## .Internal3 3.439 0.094 36.537 0.000
## .Internal5 3.896 0.090 43.188 0.000
## .Internal6 4.970 0.065 77.025 0.000
## .External1 4.311 0.091 47.150 0.000
## .External4 4.431 0.079 56.253 0.000
## .External6 4.910 0.069 71.480 0.000
## .Chance1 3.112 0.095 32.771 0.000
## .Chance4 2.766 0.094 29.434 0.000
## .Chance6 2.703 0.090 30.068 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.959 0.229 12.930 0.000
## .Internal3 2.579 0.218 11.841 0.000
## .Internal5 1.724 0.207 8.339 0.000
## .Internal6 1.021 0.102 10.017 0.000
## .External1 2.653 0.208 12.762 0.000
## .External4 1.346 0.151 8.928 0.000
## .External6 0.708 0.133 5.332 0.000
## .Chance1 2.883 0.232 12.425 0.000
## .Chance4 1.411 0.348 4.058 0.000
## .Chance6 2.372 0.210 11.315 0.000
## IHLC 0.289 0.085 3.391 0.001
## PHLC 0.414 0.102 4.054 0.000
## CHLC 0.426 0.115 3.718 0.000anova(metric_mod, config_mod)##
## Chi-Squared Difference Test
##
## Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
## config_mod 64 31119 31431 232.00
## metric_mod 71 31112 31390 238.14 6.1339 0 7 0.5242Note that now, in the two tables of output, the loadings are constrained to be the same. Though our original fit is not great, the \(\chi^{2}\) has a \(p = .52\) meaning we can retain the \(H_0\) and conclude that we have metric invariance (loadings equal doesn’t worsen the fit)
# fit the model specifying "SEX" as the grouping variable and the group.equal option
scalar_mod <- cfa(MHLC_mod2, data = MHLC, group = "SEX", group.equal = c("loadings","intercepts"))
summary(scalar_mod, fit.measures = T)## lavaan 0.6-19 ended normally after 88 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 69
## Number of equality constraints 17
##
## Number of observations per group: Used Total
## Female 464 475
## Male 367 369
##
## Model Test User Model:
##
## Test statistic 269.815
## Degrees of freedom 78
## P-value (Chi-square) 0.000
## Test statistic for each group:
## Female 152.269
## Male 117.545
##
## Model Test Baseline Model:
##
## Test statistic 1060.403
## Degrees of freedom 90
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.802
## Tucker-Lewis Index (TLI) 0.772
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -15512.589
## Loglikelihood unrestricted model (H1) -15377.681
##
## Akaike (AIC) 31129.178
## Bayesian (BIC) 31374.755
## Sample-size adjusted Bayesian (SABIC) 31209.621
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.077
## 90 Percent confidence interval - lower 0.067
## 90 Percent confidence interval - upper 0.087
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.318
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.066
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
##
## Group 1 [Female]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Intrnl1 1.000
## Intrnl3 (.p2.) 1.642 0.274 6.001 0.000
## Intrnl5 (.p3.) 2.159 0.347 6.221 0.000
## Intrnl6 (.p4.) 1.391 0.225 6.178 0.000
## PHLC =~
## Extrnl1 1.000
## Extrnl4 (.p6.) 1.487 0.189 7.869 0.000
## Extrnl6 (.p7.) 1.489 0.195 7.638 0.000
## CHLC =~
## Chance1 1.000
## Chance4 (.p9.) 2.525 0.490 5.155 0.000
## Chance6 (.10.) 1.136 0.164 6.913 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.118 0.033 3.583 0.000
## CHLC 0.007 0.023 0.304 0.761
## PHLC ~~
## CHLC 0.100 0.035 2.887 0.004
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Intrnl1 (.27.) 3.757 0.065 57.974 0.000
## .Intrnl3 (.28.) 3.054 0.070 43.697 0.000
## .Intrnl5 (.29.) 3.704 0.074 50.246 0.000
## .Intrnl6 (.30.) 4.809 0.054 88.326 0.000
## .Extrnl1 (.31.) 4.083 0.067 60.917 0.000
## .Extrnl4 (.32.) 4.204 0.064 65.607 0.000
## .Extrnl6 (.33.) 4.749 0.060 78.806 0.000
## .Chance1 (.34.) 3.217 0.069 46.848 0.000
## .Chance4 (.35.) 3.156 0.087 36.335 0.000
## .Chance6 (.36.) 2.674 0.065 41.294 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.727 0.188 14.469 0.000
## .Internal3 2.249 0.179 12.587 0.000
## .Internal5 1.572 0.183 8.573 0.000
## .Internal6 1.290 0.109 11.872 0.000
## .External1 2.868 0.201 14.241 0.000
## .External4 1.367 0.143 9.562 0.000
## .External6 0.966 0.127 7.592 0.000
## .Chance1 3.006 0.213 14.081 0.000
## .Chance4 0.968 0.477 2.031 0.042
## .Chance6 2.203 0.175 12.573 0.000
## IHLC 0.275 0.082 3.357 0.001
## PHLC 0.415 0.098 4.232 0.000
## CHLC 0.404 0.113 3.571 0.000
##
##
## Group 2 [Male]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Intrnl1 1.000
## Intrnl3 (.p2.) 1.642 0.274 6.001 0.000
## Intrnl5 (.p3.) 2.159 0.347 6.221 0.000
## Intrnl6 (.p4.) 1.391 0.225 6.178 0.000
## PHLC =~
## Extrnl1 1.000
## Extrnl4 (.p6.) 1.487 0.189 7.869 0.000
## Extrnl6 (.p7.) 1.489 0.195 7.638 0.000
## CHLC =~
## Chance1 1.000
## Chance4 (.p9.) 2.525 0.490 5.155 0.000
## Chance6 (.10.) 1.136 0.164 6.913 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.132 0.037 3.576 0.000
## CHLC 0.059 0.028 2.134 0.033
## PHLC ~~
## CHLC 0.105 0.037 2.858 0.004
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Intrnl1 (.27.) 3.757 0.065 57.974 0.000
## .Intrnl3 (.28.) 3.054 0.070 43.697 0.000
## .Intrnl5 (.29.) 3.704 0.074 50.246 0.000
## .Intrnl6 (.30.) 4.809 0.054 88.326 0.000
## .Extrnl1 (.31.) 4.083 0.067 60.917 0.000
## .Extrnl4 (.32.) 4.204 0.064 65.607 0.000
## .Extrnl6 (.33.) 4.749 0.060 78.806 0.000
## .Chance1 (.34.) 3.217 0.069 46.848 0.000
## .Chance4 (.35.) 3.156 0.087 36.335 0.000
## .Chance6 (.36.) 2.674 0.065 41.294 0.000
## IHLC 0.115 0.049 2.350 0.019
## PHLC 0.132 0.058 2.283 0.022
## CHLC -0.139 0.056 -2.506 0.012
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Internal1 2.996 0.231 12.990 0.000
## .Internal3 2.608 0.222 11.747 0.000
## .Internal5 1.745 0.205 8.527 0.000
## .Internal6 1.017 0.102 9.983 0.000
## .External1 2.650 0.209 12.701 0.000
## .External4 1.312 0.151 8.717 0.000
## .External6 0.751 0.126 5.950 0.000
## .Chance1 2.958 0.233 12.718 0.000
## .Chance4 1.030 0.444 2.322 0.020
## .Chance6 2.529 0.209 12.071 0.000
## IHLC 0.264 0.081 3.269 0.001
## PHLC 0.439 0.105 4.170 0.000
## CHLC 0.347 0.101 3.453 0.001anova(scalar_mod, metric_mod)##
## Chi-Squared Difference Test
##
## Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
## metric_mod 71 31112 31390 238.14
## scalar_mod 78 31129 31375 269.81 31.676 0.09211 7 4.663e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1This time, we see that the intercepts are now the same in the two groups. This is essentially the same as the item means being equal. However, the \(\chi^{2} = 31.68\) with a \(p < .001\) so we would reject the null hypothesis of no worsening in fit and declare that we do not have scalar invariance.
Usually, we would stop here and declare only metric invariance and discuss the implications. Residual invariance is actually quite a strict assumption, but here’s how we would fit it.
# fit the model specifying "SEX" as the grouping variable and the group.equal option
resid_mod <- cfa(MHLC_mod2, data = MHLC, group = "SEX", group.equal = c("loadings","intercepts","residuals"))
summary(resid_mod, fit.measures = T)## lavaan 0.6-19 ended normally after 87 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 69
## Number of equality constraints 27
##
## Number of observations per group: Used Total
## Female 464 475
## Male 367 369
##
## Model Test User Model:
##
## Test statistic 281.099
## Degrees of freedom 88
## P-value (Chi-square) 0.000
## Test statistic for each group:
## Female 158.765
## Male 122.334
##
## Model Test Baseline Model:
##
## Test statistic 1060.403
## Degrees of freedom 90
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.801
## Tucker-Lewis Index (TLI) 0.796
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -15518.231
## Loglikelihood unrestricted model (H1) -15377.681
##
## Akaike (AIC) 31120.462
## Bayesian (BIC) 31318.812
## Sample-size adjusted Bayesian (SABIC) 31185.435
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.073
## 90 Percent confidence interval - lower 0.063
## 90 Percent confidence interval - upper 0.082
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.107
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.067
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
##
## Group 1 [Female]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Intrnl1 1.000
## Intrnl3 (.p2.) 1.654 0.279 5.937 0.000
## Intrnl5 (.p3.) 2.162 0.351 6.160 0.000
## Intrnl6 (.p4.) 1.408 0.230 6.121 0.000
## PHLC =~
## Extrnl1 1.000
## Extrnl4 (.p6.) 1.479 0.188 7.849 0.000
## Extrnl6 (.p7.) 1.499 0.197 7.604 0.000
## CHLC =~
## Chance1 1.000
## Chance4 (.p9.) 2.503 0.479 5.220 0.000
## Chance6 (.10.) 1.132 0.164 6.903 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.125 0.034 3.696 0.000
## CHLC 0.005 0.023 0.229 0.819
## PHLC ~~
## CHLC 0.099 0.035 2.858 0.004
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Intrnl1 (.27.) 3.751 0.065 57.740 0.000
## .Intrnl3 (.28.) 3.065 0.070 43.486 0.000
## .Intrnl5 (.29.) 3.701 0.074 50.015 0.000
## .Intrnl6 (.30.) 4.808 0.054 89.506 0.000
## .Extrnl1 (.31.) 4.080 0.067 60.736 0.000
## .Extrnl4 (.32.) 4.204 0.064 65.271 0.000
## .Extrnl6 (.33.) 4.753 0.060 79.182 0.000
## .Chance1 (.34.) 3.217 0.069 46.890 0.000
## .Chance4 (.35.) 3.154 0.087 36.331 0.000
## .Chance6 (.36.) 2.685 0.065 41.235 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Intrnl1 (.11.) 2.850 0.148 19.256 0.000
## .Intrnl3 (.12.) 2.413 0.146 16.565 0.000
## .Intrnl5 (.13.) 1.663 0.152 10.912 0.000
## .Intrnl6 (.14.) 1.163 0.081 14.403 0.000
## .Extrnl1 (.15.) 2.772 0.147 18.821 0.000
## .Extrnl4 (.16.) 1.354 0.118 11.515 0.000
## .Extrnl6 (.17.) 0.861 0.107 8.019 0.000
## .Chance1 (.18.) 2.981 0.162 18.384 0.000
## .Chance4 (.19.) 1.016 0.415 2.452 0.014
## .Chance6 (.20.) 2.344 0.144 16.300 0.000
## IHLC 0.269 0.081 3.318 0.001
## PHLC 0.431 0.101 4.261 0.000
## CHLC 0.404 0.112 3.606 0.000
##
##
## Group 2 [Male]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## IHLC =~
## Intrnl1 1.000
## Intrnl3 (.p2.) 1.654 0.279 5.937 0.000
## Intrnl5 (.p3.) 2.162 0.351 6.160 0.000
## Intrnl6 (.p4.) 1.408 0.230 6.121 0.000
## PHLC =~
## Extrnl1 1.000
## Extrnl4 (.p6.) 1.479 0.188 7.849 0.000
## Extrnl6 (.p7.) 1.499 0.197 7.604 0.000
## CHLC =~
## Chance1 1.000
## Chance4 (.p9.) 2.503 0.479 5.220 0.000
## Chance6 (.10.) 1.132 0.164 6.903 0.000
##
## Covariances:
## Estimate Std.Err z-value P(>|z|)
## IHLC ~~
## PHLC 0.124 0.036 3.460 0.001
## CHLC 0.064 0.028 2.253 0.024
## PHLC ~~
## CHLC 0.107 0.037 2.881 0.004
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|)
## .Intrnl1 (.27.) 3.751 0.065 57.740 0.000
## .Intrnl3 (.28.) 3.065 0.070 43.486 0.000
## .Intrnl5 (.29.) 3.701 0.074 50.015 0.000
## .Intrnl6 (.30.) 4.808 0.054 89.506 0.000
## .Extrnl1 (.31.) 4.080 0.067 60.736 0.000
## .Extrnl4 (.32.) 4.204 0.064 65.271 0.000
## .Extrnl6 (.33.) 4.753 0.060 79.182 0.000
## .Chance1 (.34.) 3.217 0.069 46.890 0.000
## .Chance4 (.35.) 3.154 0.087 36.331 0.000
## .Chance6 (.36.) 2.685 0.065 41.235 0.000
## IHLC 0.115 0.049 2.359 0.018
## PHLC 0.132 0.058 2.281 0.023
## CHLC -0.140 0.056 -2.501 0.012
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .Intrnl1 (.11.) 2.850 0.148 19.256 0.000
## .Intrnl3 (.12.) 2.413 0.146 16.565 0.000
## .Intrnl5 (.13.) 1.663 0.152 10.912 0.000
## .Intrnl6 (.14.) 1.163 0.081 14.403 0.000
## .Extrnl1 (.15.) 2.772 0.147 18.821 0.000
## .Extrnl4 (.16.) 1.354 0.118 11.515 0.000
## .Extrnl6 (.17.) 0.861 0.107 8.019 0.000
## .Chance1 (.18.) 2.981 0.162 18.384 0.000
## .Chance4 (.19.) 1.016 0.415 2.452 0.014
## .Chance6 (.20.) 2.344 0.144 16.300 0.000
## IHLC 0.263 0.081 3.261 0.001
## PHLC 0.419 0.101 4.158 0.000
## CHLC 0.356 0.098 3.646 0.000anova(resid_mod, scalar_mod)##
## Chi-Squared Difference Test
##
## Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
## scalar_mod 78 31129 31375 269.81
## resid_mod 88 31120 31319 281.10 11.284 0.01758 10 0.3358Now, interestingly enough, constraining the residuals does not significantly worsen the fit compared to the scalar model, but since the scalar model was rejected, we cannot conclude that we have strict invariance because the scalar model wasn’t better than the metric model.
We asked the following 4 questions across men and women:
We answered yes to 1 and 2, but rejected 3 and 4. We would then suggest some kind of work be done with the tool to ensure that men and women responded the same to the items to enable at least scalar invariance, and ideally strict residual invariance. This is often a tall task, but it can be done.