How to determine validity for quantitative research?

By Priya Chetty on August 12, 2016

Measurement is a replicable and systematic process through which an object or instrument is quantified or classified as in the field of social science that deals with the quantification of behaviour. In this case, determining the validity of the measuring instrument (questionnaire) holds the utmost importance (Drost 2011). Consequently, measuring behaviours lead to the dilemma of whether to measure what is intended to be measured.

For example, when a study is intended to measure the engagement of employees in an organisation, the survey developed to answer the motivating factors is not considered to be valid.

Although the issue of validity cannot be established with complete certainty, it is still favoured to maintain the validity of the measuring instrument. The reason behind determining validity lies in the plethora of threats a research faces. This includes history, maturation, testing, instrumentation, selection, mortality, diffusion of treatment and compensatory equalization, rivalry, and demoralization.

Importance of determining validity in a research

Traditionally, the establishment of instrument validity was limited to the sphere of quantitative research. However, the concept of determination of the credibility of the research is applicable to qualitative data. Rooted in the positivist approach of philosophy, quantitative research deals primarily with the culmination of empirical conceptions (Winter 2000). Under such an approach, validity determines whether the research truly measures what it was intended to measure. Furthermore, it also measures the truthfulness of the research results (Kothari 2012).

Construct validity and construction of an initial concept

For example, the construct validity would determine whether the subject has high anxiety score in a survey. Does it actually have a high degree of anxiety?

Construct validity lays the ground for the construction of an initial concept notion, question, or hypothesis that determines the data to be collected. Furthermore, it plans the collection of data (Wainer & Braun 1988). Therefore, construct validity deals with determining the research instrument and what is intended to be measured.

Further, it uses three different parameters to check validity:

  • Homogeneity; research instrument measures one construct such as anxiety levels.
  • Convergence; the research instrument measures concepts which are similar to other instruments, in order to determine the convergence is results.
  • Theoretical evidence; when the findings are in sync with the theoretical evidence.

Determining the required variable with content validity

For example, to determine the anxiety level on different parameters the content validity helps to determine the role of every factor that contributes towards anxiety.

The content validity category determines whether the research instrument is able to cover the content with respect to the variables and tests.

Face validity is a sub-set of content validity. In face validity, experts or academicians are subjected to the measuring instrument to determine the intended purpose of the questionnaire.

Criterion validity in comparing different measuring instruments

Criterion validity helps to review the existing measuring instruments against other measurements. This is to determine the extent to which different instruments measure the same variable. There are three sub-sets of criterion validity; convergent, divergent, and predictive.

In the case of convergent, the results predict high correlation with the existing instrument i.e. they are measuring similar variables.

The second subset is divergent,  where the correlation with the measuring instrument is low. In such cases, the measuring instrument should be changed.

For example, if one of the instruments measures anxiety and the other instrument measures IQ level then there will be divergence.

Finally, in the case of predictive, the instrument should be able to “predict” the likelihood that IQ levels impact or predict the anxiety levels.

The following table show different validity applied in research.

Different methods to determining validity in quantitative research method
Determining validity in quantitative research (Source: Drost, 2011; p117)

The entire research process should establish validity. This is important in order to ensure the capability of the instrument (survey, interview, etc.) in deriving the results (Drost 2011).

References

  • Creswell, J.W. & Miller, D.L., 2000. Determining Validity in Qualitative Inquiry. Theory Into Practice, 39(3), pp.124–130.
  • Drost, E.A., 2011. Validity and Reliability in Social Science Research. Education Research and Perspectives, 38(1), pp.115–123.
  • Fraser, S. & Greenhalgh, T., 2001. Coping with complexity: educating for capability. BMJ, 323, pp.799–803.
  • Glyn Winter, 2000. A Comparative Discussion of the Notion of “Validity” in Qualitative and Quantitative Research. The Qualitative Report, 4(4). Available at: http://www.nova.edu/ssss/QR/QR4-3/winter.html.
  • Kothari, C.R., 2012. Research Methodology: An introduction. In Research Methodology: Methods and Techniques. p. 418.
  • Leung, L., 2015. Validity, reliability, and generalizability in qualitative research. Journal of family medicine and primary care, 4(3), pp.324–7. Available at: http://www.ncbi.nlm.nih.gov/pubmed/26288766 [Accessed July 11, 2016].
  • Lincoln, Y.S. & Guba, E.G., 1982. ESTABLISHING. DEPENDABILITY AND tl:ONFIRMABILITY IN NATURALISTIC INQUIRY THROUGH AN AUDIT. In American Educational Research Association Annual Meeting, New York. New York: U.S. Department of Education, p. 31. Available at: http://files.eric.ed.gov/fulltext/ED216019.pdf [Accessed October 21, 2015].
  • Lincoln, Y.S. & Guba, E.G., 1985. Naturalistic Inquiry, Beverly Hills, CA: SAGE Publication.
  • Long, T. & Johnson, M., 2000. Rigour, reliability and validity in qualitative research. Clin Eff Nurs, 4, pp.30–37.
  • Maxwell, J.A., 2005. Qualitative Research Design: An Interactive Approach, SAGE Publications. Available at: https://books.google.co.in/books/about/Qualitative_Research_Design.html?id=XqaJP-iehskC&pgis=1 [Accessed May 20, 2015].
  • Morse, J., Barrett, M. & Mayam, M., 2002. Verification strategies for establishing reliability validity in qualitative research. Int J Qual Res, 1, pp.1–19.
  • Nahid Golafshani, 2003. Understanding Reliability and Determining Validity in Qualitative Research. The Qualitative Report, 8(4), pp.597–607.
  • Noble, H. & Smith, J., 2015. Issues of validity and reliability in qualitative research. Evidence Based Nursing, 18(2), pp.34–35. Available at: http://ebn.bmj.com/lookup/doi/10.1136/eb-2015-102054 [Accessed July 11, 2016].
  • Sandelowski, M., 1993. Rigor or rigor mortis: the problem of rigor in qualitative research revisited. Adv Nurs Sci, 16, pp.1–8.
  • Slevin, E., 2002. Enhancing the truthfulness, consistency, and transferability of a qualitative study: using a manifold of two approaches. Nurse Res, 7, pp.79–197.
  • Wainer, H. & Braun, H.I., 1988. Test validity, Hilldale, NJ: Lawrence Earlbaum Associates.

Discuss

3 thoughts on “How to determine validity for quantitative research?”