Examples of threats to internal and external validity in a research

In my previous article I have discussed how the validity can be ensured with respect to Quantitative and Qualitative analysis. This article discusses the threats to validity (internal and external) irrespective of the approach.

Threats to internal validity

  1. Timeline: Time is of paramount importance in a research. The opinions of respondents depend on the recall time to gather opinions.
    For example, if the researcher ask the respondents about satisfaction with products at a coffee store and where they will consume it. Then the validity of their answers will increase. However, in case the research is conducted after a long duration then the opinions can be biased and misleading.
  2. Testing: Instances where the respondents are asked questions which is questionable for their performance.
    For example, if the employees are asked to rate satisfaction level of their customers on different service quality parameters. They might be concerned about the findings of the research which can put them in a disadvantageous position in the organisation.
  3. Instrumentation: Effective changes in instrumentation or in the criteria of recording behavior can be cause threats to validity.
    For example, the change in cutoff points for a TOEFL exam can impact the application process. Similarly, change in standard levels in medical laboratory tests can impact the overall efficacy of the results.
  4. Maturation: It is the changes that impact the subsequent analysis.
    For example, performance of 2nd graders starts decreasing after 1 hour due to variable factors, like fatigue, stress, tiredness etc. Thus, it is difficult to calculate the overall performance average without bias.
  5. Mortality: Most of the studies undertaken follow ethical considerations where the respondents  participate voluntarily. However, some respondents may drop out. This will change the defined sample size. Especially studies which have long timelines face this threat to their validity.
    For example, a researcher conducting a study to determine the efficacy of protein diet for a duration of 6 months might face problem when the test subjects drop out of the program mid-way.
  6. Statistical regression: This threat to validity could be when sample is selected to study extreme behaviour in respondents.
    For example if a researcher needs to study consumption of mangoes. Then the threat to validity would be when the collection of data is in a peak consumption season.

External threats to validity

  1. Impact of pre-testing: Most often researchers conduct pre-tests or pilot tests to determine efficacy of the measuring instrument. However, pre-tests might impact the sensitivity and responsiveness to the experimental variable.
    For example, researcher conduct a pre-test on a sample of 25 respondents. However nearly 70% of responses changes when actually conducting the study ,reflecting the impact of pre-test.
  2. Effect of inclusion and exclusion criteria: Effect of selecting a sample based on specific selection criteria. This can impact the outcomes of study which would not have been the case, if there was random sampling.
  3. Multiple experiment interference: This happens in case of test subjects who have been exposed to same experiment multiple times. In such cases the effect of previous findings have an impact on overall results.
  4. Reactions to experimental arrangement: This is an effect of experiment because the respondents are aware about the experiment. This is also known as Hawthorne effect.

References

  • Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally.
  • Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin Company.
  • Saunders, M., Lewis, P., & Thornhill, A. (2007). Research methods for business students fifth edition (3rd ed.). Harlow: Prentice Hall. Retrieved from https://is.vsfs.cz/el/6410/leto2014/BA_BSeBM/um/Research_Methods_for_Business_Students__5th_Edition.pdf
8-step procedure to conduct qualitative content analysis in a research
Limitations and weakness of qualitative research methods
Shruti Datt

Shruti Datt

Project Handler at Project Guru
Shruti is B-Tech & M-Tech in Biotechnology. Some of her strengths include, Good interpersonal skills, eye for detail, well devised analytical and decision making skills and a positive attitude towards life. Her aim in life is to obtain a responsible and challenging position where her education and work experience will have valuable application.
She is a true Piscean. She loves doing things to perfection with passion. She is very creative and likes to make personalized gifts for her dear ones, this is actually something that keeps her going. Shruti loves adventure sports and likes river rafting and cliff jumping.
Shruti Datt

Related articles

  • Questionnaire administration In my previous article Questionnaire development in theses and dissertation, I have discussed about the questionnaire development process, the next step is to administer the questionnaire. Questionnaire administration in a proper way and to chose the sample from population is important. […]
  • Determining validity while conducting a quantitative research Measurement is a replicable and systematic process through which an object or instrument is quantified or classified as in the field of social science that deals with quantification of behaviour. In this case determining validity of the measuring instrument (questionnaire) holds utmost […]
  • Limitations and weakness of quantitative research methods Quantitative research main purpose is the quantification of the data. It allows generalisations of the results by measuring the views and responses of the sample population. Every research methodology consists two broad phases namely planning and execution (Younus 2014).
  • Defining research strategy in a research paper on business studies Sarantakos defined research method as “the theory of methods” (Sarantakos 2012; p. 465), or the way through which a researcher makes sense of the object of inquiry. Within research methodology, research strategy assumes as the “general plan of how the researcher will go about answering […]
  • Understanding research philosophy Research philosophy is an important part of research methodology. Research philosophy is classified as ontology, epistemology and axiology. These philosophical approaches enable to decide which approach should be adopted by the researcher and why, which is derived from research questions […]
Discussions

1 Comments.

  1. System Analysis and Design

Discuss

We are looking for candidates who have completed their master's degree or Ph.D. Click here to know more about our vacancies.