Examples of threats to the internal and external validity of a research

In my previous article, I have discussed how the validity can be ensured with respect to Quantitative and Qualitative analysis. This article discusses the threats to validity (internal and external) irrespective of the approach.

Threats to internal validity

1. Timeline: Time is of paramount importance in research. The opinions of respondents depend on the recall time to gather opinions.
   For example, if the researcher asks the respondents about satisfaction with products at a coffee store and where they will consume it. Then the validity of their answers will increase. However, in case the research is conducted after a long duration then the opinions can be biased and misleading.
2. Testing: Instances where the respondents are asked questions which is questionable for their performance.
   For example, if the employees are asked to rate the satisfaction level of their customers on different service quality parameters. They might be concerned about the findings of the research which can put them in a disadvantageous position in the organization.
3. Instrumentation: Effective changes in instrumentation or in the criteria of recording behavior can because of threats to validity.
   For example, the change in cutoff points for a TOEFL exam can impact the application process. Similarly, changes in standard levels in medical laboratory tests can impact the overall efficacy of the results.
4. Maturation: It is the changes that impact the subsequent analysis.
   For example, the performance of 2nd graders starts decreasing after 1 hour due to variable factors, like fatigue, stress, tiredness, etc. Thus, it is difficult to calculate the overall performance average without bias.
5. Mortality: Most of the studies undertaken follow ethical considerations where the respondents participate voluntarily. However, some respondents may drop out. This will change the defined sample size. Especially studies that have long timelines face this threat to their validity.
   For example, a researcher conducting a study to determine the efficacy of a protein diet for a duration of 6 months might face a problem when the test subjects drop out of the program mid-way.
6. Statistical regression: This threat to validity could be when a sample is selected to study extreme behavior in respondents.
   For example, if a researcher needs to study the consumption of mangoes. Then the threat to validity would be when the collection of data is in a peak consumption season.

External threats to validity

1. Impact of pre-testing: Most often researchers conduct pre-tests or pilot tests to determine the efficacy of the measuring instrument. However, pre-tests might impact the sensitivity and responsiveness of the experimental variable.
   For example, the researcher conducts a pre-test on a sample of 25 respondents. However, nearly 70% of responses change when actually conducting the study, reflecting the impact of the pre-test.
2. Effect of inclusion and exclusion criteria: Effect of selecting a sample based on specific selection criteria. This can impact the outcomes of the study which would not have been the case if there was random sampling.
3. Multiple experiment interference: This happens in the case of test subjects who have been exposed to the same experiment multiple times. In such cases, the effect of previous findings has an impact on the overall results.
4. Reactions to an experimental arrangement: This is an effect of the experiment because the respondents are aware of the experiment. This is also known as the Hawthorne effect.

References

• Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally.
• Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin Company.
• Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students fifth edition (3rd ed.). Harlow: Prentice Hall. Retrieved from https://is.vsfs.cz/el/6410/leto2014/BA_BSeBM/um/Research_Methods_for_Business_Students__5th_Edition.pdf