March, 2010. In most quantitative based dissertations and theses, it is important to address threats to validity. Although there are many different types of validity, one of the most frequently discussed is internal validity. Internal validity refers to the "assertion that an observed relation between two variables reflects a causal process or that the lack of an observed relation reflects the lack of a causal process" (Curren & Werth, 2004, p. 220). Any discussions of internal validity should help clarify how the researcher will ensure that conclusions made about cause and effect are accurate. In the case where threats to internal validity are adequately addressed, the researcher will be able to make correct inferences about the relationship between independent and dependent variables based on the design of the study and the data obtained.
Internal validity is typically discussed within the framework of experimental research. This is because experimental research is, by nature, concerned with cause and effect. A treatment variable (i.e., independent variable) is applied to a group of participants to see if the treatment has an effect on the participants as expressed in the dependent variable. By addressing threats to internal validity, a dissertation researcher, to the best of that person’s ability, anticipates and responds to factors that might interfere with the potential causal influence of the treatment variable upon the dependent variable. This does not mean the researcher attempts to make the treatment effective. That would itself be a bias compromising internal validity. It simply means the researcher attempts to remove obstacles that might obscure any possible treatment effect or conversely create an effect where there really is none.
With this in mind, some of the threats to internal validity described by Girden (1996) include the following:
1. Loss of participants in a study
2. Selection bias (non-random selection)
3. Factors that affect participants between pretests and posttests
4. Participants in the control group exceeding or lowering level of
performance because of unequal treatment.
5. Bias on the part of experimenters when interacting with treatment
groups and control groups
To this list, should be added problems arising from not addressing other possible variables that might be influencing or causing variation in the dependent variable.
By mitigating against the factors listed above, the researcher is able to increase the internal validity of the experiment. Further, mitigating steps should be stated in the validity section of the dissertation. This may seem obvious, but if the PhD student, for example, does not explain the preventative measures, neither the advisor nor the committee members will know that threats have been addressed.
Internal validity is more often discussed in terms of experimental or quasi-experimental research, because experimental research is more concerned with cause and effect compared to non-experimental correlational studies. Some experts caution against viewing correlational studies in terms of cause and effect. As Boslaugh and Watters (2008) remark, "The findings of a correlation between two variables does not imply that a change in one variable causes a corresponding change in another" (p. 169). Elsewhere, Cone and Foster (1996) suggest that researchers who are conducting correlational studies should avoid the use of causal terms in their dissertation in favor of words that emphasize association and relation.
But if internal validity is an issue related to causality and correlational studies are predictive rather than explanatory, why does the discussion of internal validity appear in so many correlation-based dissertations? Perhaps this is because, regardless the admonitions against claims of causality in correlational studies, correlational studies still have potential for providing a glimpse into causal relations between variables.
Indeed, despite Cone’s and Foster’s warning against the use of causal language, they themselves discuss with specific references to causality how threats to internal validity should be approached in correlational studies. If the researcher discusses causality in his or her correlational study, the issue of internal validity should be handled differently compared to experimental research. In particular, Cone and Foster emphasize three types of threats to the internal validity in correlation-based dissertations:
6. Reverse causation (dependent variable is actually the independent variable)
7. Confound variables (an entirely different variable that accounts for variation in both the
independent and dependent variable)
8. Reciprocal causation (changes in the independent variable influence the dependent variable
which in turn influences the independent variable again)
Among these three threats, perhaps item 7 is the most difficult to eliminate or even understand because with correlational studies of the observational kind, the researcher may have little control over the sample population. In an experiment, confound variables can be manipulated and sample populations randomized, restricted, and matched to control for confounds.
Some of the strategies employed in experimental research, namely sampling strategies, can, however, be applied to correlational dissertations to address problems with confound variables. For example, with correlational survey research, the dissertation student might control for gender by restricting the sample population to only men or women. Another strategy, particular to correlational research, would be to conduct a multiple regression study instead of a standard correlational study. Multiple regression allows the researcher to measure the possible influence of a specific independent variable of while accounting for other factors that influence the outcome (Neuman, 2006).Employing multiple regression in favor of standard two-variable regression is one way to statistically control for other variables in the sense that it allows you to compare the influence of independent variables against each other.
Unfortunately, many dissertation handbooks that discuss threats to internal validity tend to gloss over the differences between correlational and experimental research by examining the subject strictly in terms of experimental research. However, there are significant differences in how experimental and correlational research is carried out and how causality or lack thereof is determined. Additionally, a discussion of causality, in the case of correlational research, may not even be relevant. The PhD student should be particularly careful when writing about internal validity in the introduction or methods chapter of his or her dissertation or thesis, for the student may end up addressing threats to internal validity as if he or she were conducting an experiment. That is, they may end up discussing items 1-5 above, when a discussion of confounds or possible control variables would be more appropriate (items 6-8).
Boslaugh, S., & Watters, P.E. (2008). Statistics in a nutshell. Sebastopal, CA: O’Reilly Media.
Cone, J.D., & Foster, S. L. (1996). Dissertations and theses from start to finish: Psychology and related fields. Washington, D.C.: American Psychological Association.
Curren, P. J., & Wirth, R. J. (2004). Interindividual differences in intraindividual variation: Balancing internal and external validity. Measurement, 2(4), 219-247.
Girden, E. R. (1996). Evaluating research articles from start to finish. Thousand Oaks, CA: Sage Publications.
Neuman, W.L. (2006). Social research methods: Qualitative and quantitative approaches (6th ed.). Boston: Pearson Education.