In my previous article I have discussed how the validity can be ensured with respect to Quantitative and Qualitative analysis. This article discusses the threats to validity (internal and external) irrespective of the approach.
Threats to internal validity
- Timeline: Time is of paramount importance in a research. The opinions of respondents depend on the recall time to gather opinions.
For example, if the researcher ask the respondents about satisfaction with products at a coffee store and where they will consume it. Then the validity of their answers will increase. However, in case the research is conducted after a long duration then the opinions can be biased and misleading.
- Testing: Instances where the respondents are asked questions which is questionable for their performance.
For example, if the employees are asked to rate satisfaction level of their customers on different service quality parameters. They might be concerned about the findings of the research which can put them in a disadvantageous position in the organisation.
- Instrumentation: Effective changes in instrumentation or in the criteria of recording behavior can be cause threats to validity.
For example, the change in cutoff points for a TOEFL exam can impact the application process. Similarly, change in standard levels in medical laboratory tests can impact the overall efficacy of the results.
- Maturation: It is the changes that impact the subsequent analysis.
For example, performance of 2nd graders starts decreasing after 1 hour due to variable factors, like fatigue, stress, tiredness etc. Thus, it is difficult to calculate the overall performance average without bias.
- Mortality: Most of the studies undertaken follow ethical considerations where the respondents participate voluntarily. However, some respondents may drop out. This will change the defined sample size. Especially studies which have long timelines face this threat to their validity.
For example, a researcher conducting a study to determine the efficacy of protein diet for a duration of 6 months might face problem when the test subjects drop out of the program mid-way.
- Statistical regression: This threat to validity could be when sample is selected to study extreme behaviour in respondents.
For example if a researcher needs to study consumption of mangoes. Then the threat to validity would be when the collection of data is in a peak consumption season.
External threats to validity
- Impact of pre-testing: Most often researchers conduct pre-tests or pilot tests to determine efficacy of the measuring instrument. However, pre-tests might impact the sensitivity and responsiveness to the experimental variable.
For example, researcher conduct a pre-test on a sample of 25 respondents. However nearly 70% of responses changes when actually conducting the study ,reflecting the impact of pre-test.
- Effect of inclusion and exclusion criteria: Effect of selecting a sample based on specific selection criteria. This can impact the outcomes of study which would not have been the case, if there was random sampling.
- Multiple experiment interference: This happens in case of test subjects who have been exposed to same experiment multiple times. In such cases the effect of previous findings have an impact on overall results.
- Reactions to experimental arrangement: This is an effect of experiment because the respondents are aware about the experiment. This is also known as Hawthorne effect.
- Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally.
- Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin Company.
- Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students fifth edition (3rd ed.). Harlow: Prentice Hall. Retrieved from https://is.vsfs.cz/el/6410/leto2014/BA_BSeBM/um/Research_Methods_for_Business_Students__5th_Edition.pdf
She is a true Piscean. She loves doing things to perfection with passion. She is very creative and likes to make personalized gifts for her dear ones, this is actually something that keeps her going. Shruti loves adventure sports and likes river rafting and cliff jumping.
Latest posts by Shruti Datt (see all)
- Matrices and arrays in MATLAB - November 15, 2018
- Understanding the process of selecting data entry formats in meta-analysis - October 22, 2017
- Variable returns to scale in DEA and summary of efficiency and slacks - October 10, 2017