Criteria for reliability and validity in SEM analysis

By Riya Jain & Priya Chetty on September 30, 2021

Structural equation modeling (SEM) techniques provide great tools for undertaking initial evaluations of differential validity and reliability of measurement equipment over a wide range of demographic groups. It is vital to consider reliability and validity for a research design, organizing procedures, and writing findings for research, especially in quantitative research. The principles of reliability and validity are used to assess the quality of research. They reflect the accuracy with which a method, approach, or test measures a problem.

The previous article has determined the scenarios wherein the SEM analysis should be applied. However, before analyzing the data it is important to test whether the acquired data is valid or not. Thus, this article focuses on providing a detailed idea about the required criteria for establishing the reliability and validity of SEM analysis.

The figure below represents the acceptable reliability and validity values in SEM analysis.

Figure 1: Reliability and validity in SEM analysis
Figure 1: Reliability and validity in SEM analysis

Criteria for validity in SEM analysis

The validity of instruments is critical for researchers to address. Validity is defined as the accuracy of the outcome of a test (Karakaya-Ozyer, 2018). The validity of a research instrument or dataset measures the coverage of the actual information from the collected or analyzed dataset. It is therefore essential to establish validity (Taherdoost, 2018). In the case of SEM analysis, it provides researchers with evidence that the results can be accurately interpreted.

While examining the validity, the assessment can be categorized into two major types:

  • Convergent validity
  • Discriminant validity

Criteria for convergent validity

Convergent validity refers to the correlation between responses of different variables in assessing the same construct. Convergent validity assures that variables are associated with the latent construct being measured. As a result, factors should have a strong correlation with the latent construct. To establish convergent validity, the AVE value is assessed (Hamid, 2017; Engellant et al., 2016). The average variance extracted (AVE) as a convergent validity test is appropriate since AVE can explain the degree to which items are shared between constructs. (Sujati, 2020). To attain this validity, the value of AVE must be greater than or equal to 0.5. (Ahmad, 2016)

Criteria for discriminant validity

A discriminant validity test is a requirement in the construction of a latent variable instrument. Discriminant validity, also known as divergent validity, is the validity that contributes toward demonstrating the distinction of one construct from another (Taherdoost, 2016). Correlating one construct to another could be used to show discriminant validity (Sujati, 2020). If the correlation value between the two constructs is less than the square root of the AVE value, discriminant validity exists (Engellant et al., 2016).

Criteria for reliability in SEM analysis

Reliability is defined as the consistency of measuring outcomes (Karakaya-Ozyer, 2018). Testing for reliability is significant since it pertains to the consistency of measuring the parts of the instrument. If the items on a scale “hang together” and measure the same A construct, the scale is said to have good internal consistency reliability.

The reliability of any dataset or construct could be measured in two forms:

  • Internal consistency
  • Composite reliability

Internal consistency for reliability in SEM analysis

Internal consistency reliability depicts the data consistency in results across tests. The reliability method determines the linkage of factors on the test with other factors (Hajjar, 2018). Cronbach alpha coefficient is the most often used internal consistency measure. When using the Likert scale, it is regarded as the most accepted measure of consistency. It is recommended that the reliability for an exploratory or pilot study be equal to or greater than 0.70. Herein, the value of 0.90 and above shows excellent reliability, 0.70-0.90 represents high reliability, 0.50-0.70 states moderate reliability, and 0.50 and below depict low reliability (Sideridis, 2018)

Composite reliability in SEM analysis

Composite reliability measures how well variables underlying constructs served in structural equation modeling. In SEM construct reliability is depicted using confirmatory factor analysis (CFA). Composite reliability is estimated based on the factor loading analysis (Lerdpornkulrat et al., 2017). It is allowed to have a build reliability coefficient greater than 0.70. A value of CR ≥ 0.7 is required to achieve construct reliability (Tentama & Anindita, 2020).

References

  • Ahmad, S. (2016). Assessing the Validity and Reliability of a Measurement Model in Structural Equation Modeling (SEM). British Journal of Mathematics & Computer Science.
  • Engellant, K. A. ., Holland, D. D. ., & Piper, R. T. (2016). Assessing Convergent and Discriminant Validity of the Motivation Construct for the Technology Integration Education (TIE) Model. Journal of Higher Education Theory & Practice, 16(1), 37–50. https://ezproxy.indstate.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eue&AN=114544179&site=eds-live&scope=site
  • Hajjar, S. T. EL. (2018). Statistical Analysis: Internal-Consistency reliability and construct validity. International Journal of Quantitative and Qualitative Research Methods, 6(1), 27–38. www.eajournals.org
  • Hamid, M. R. A. (2017). Discriminant Validity Assessment: Use of Fornell & Larcker criterion versus HTMT Criterion. Journal of Physics: Conference Series.
  • Karakaya-Ozyer, K. & A.-D. (2018). A Review of Structural Equation Modeling Applications in Turkish Educational Science Literature, 2010-2015. International Journal of Research in Education and Science (IJRES).
  • Lerdpornkulrat, T., Poondej, C., & Koul, R. (2017). Construct reliability and validity of the shortened version of the information-seeking behavior scale. International Journal of Information and Communication Technology Education, 13(2), 27–37. https://doi.org/10.4018/IJICTE.2017040103
  • Sideridis, G. (2018). Internal Consistency Reliability in Measurement: Aggregate and Multilevel Approaches. The Journal of Modern Applied Statistical Methods.
  • Sujati, H. (2020). Testing the Construct Validity and Reliability of Curiosity Scale Using Confirmatory Factor Analysis. Journal of Educational and Social Research.
  • Taherdoost, H. (2016). Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. International Journal of Academic Research in Management (IJARM).
  • Taherdoost, H. (2018). Validity and Reliability of the Research Instrument; How to Test the Validation of a Questionnaire/Survey in a Research. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3205040
  • Tentama, F., & Anindita, W. D. (2020). Employability scale: Construct validity and reliability. International Journal of Scientific and Technology Research, 9(4), 3166–3170.
NOTES

Discuss