NATURAL EXPERIMENTS
THERE ARE DIFFERENT TYPES OF QUASI-EXPERIMENTS
NATURAL EXPERIMENTS (US: Natural Experiments) The researcher studies the effect of a real-world event that has already happened. The groups already exist, so random allocation is impossible.
INDEPENDENT GROUPS QUASI-EXPERIMENTS (BETWEEN-GROUPS DESIGNS) (US: Nonequivalent Control Group Design) The researcher deliberately uses pre-existing group differences as the independent variable. The focus is on investigating the effect of those group differences themselves. For example, comparing the outcomes of patients from two different clinics where one clinic has introduced a new therapy and the other has not. The group membership (clinic, school, region, etc.) is treated as the IV because the researcher wants to study the impact of belonging to that particular group.
NON-EQUIVALENT GROUPS DESIGN (US: Nonequivalent Groups Design) The researcher compares two or more pre-existing groups that were not randomly assigned. Random allocation is not used because it is either infeasible, unethical, or not possible in the circumstances. The groups are simply the most practical ones available for comparison, rather than the group difference being the main variable of interest
PLEASE NOTE
Importantly, AQA and other examination boards refer only to “natural and quasi-experiments” in the specification and in examination questions, and never to either an independent groups quasi-experiment or a non-equivalent quasi-experiment.
Moreover, natural and quasi-experiments are often treated as interchangeable, but they are not, as will become clear. This raises the question of why they are being distinguished here and why this level of detail is necessary. The reason is that although AQA does not require this level of precision in the examination, quasi-experiments and natural experiments confuse students when taught. The underlying logic becomes unclear, making it more difficult to identify the type of experiment and evaluate its associated strengths and limitations. For example, when students are taught about true experiments, the emphasis is often placed on randomisation (in contrast to quasi-experiments) and manipulation of the independent variable (in contrast to natural experiments). This creates problems when comparing studies such as Hodges and Tizard (natural) with the Brady monkey study or research into sexual selection
NATURAL EXPERIMENTS
Be cautious not to be misled by the term "natural experiment." It might conjure images of conducting research amidst badgers and beavers in a forest clearing, but the "natural" aspect refers to the nature of the independent variable (IV) rather than the physical setting of the experiment. When we speak of a natural experiment, the IV is observed in its naturally occurring state rather than being directly manipulated by the researcher. For instance, in studying the impact of stress levels between individuals who have experienced an earthquake and those who haven't, the occurrence of the earthquake is the IV that is naturally varied, beyond the researcher's control. Natural experiments can unfold in any environment or context, distinct from field experiments, characterised by their naturalistic settings.
In a natural experiment, the researcher does not actively manipulate the independent variable (IV). Instead, the researcher observes naturally occurring variations in the IV, recognising that some IVs cannot be ethically manipulated. For instance, in studying the impacts of privation, Hodges and Tizard examined children who had experienced different upbringing scenarios—adoption, staying in an orphanage, or returning to their biological parents. These conditions represent naturally varying IVs. Manipulating such variables, such as by orchestrating the adoption of some children, keeping others in an orphanage, or sending others back to their biological families, would have been ethically impermissible.
All natural experiments have quasi-features; in other words, participants cannot be randomly allocated to conditions.
ADVANTAGES OF NATURAL EXPERIMENTS
Like field experiments, natural experiments have their own advantages and disadvantages. The biggest advantage of a natural experiment is that it allows researchers to study the effects of independent variables (IVs) that are impossible or unethical to manipulate in a controlled setting. Another advantage is high external validity: because the IV arises naturally from real-life circumstances and has not been artificially created by researchers, the results are more likely to apply to other real-life groups and situations.
DISADVANTAGES OF NATURAL EXPERIMENTS
Natural experiments deviate from the criteria of a true experiment because the independent variable (IV) is not manipulated. Without manipulation, researchers lose the ability to control extraneous variables or randomly assign participants to conditions. A prime example of this is the study conducted by Hodges and Tizard on institutionalised children who were placed in children's homes from as young as four months of age. When the children were four, some had returned to their original homes, while others were adopted. This study had three naturally occurring independent variables: returning home and staying at the institution or being adopted. The results indicated that adopted children fared better than other groups. However, since the IVs could not be manipulated due to ethical concerns, such as randomly allocating children to conditions like adoption, returning home, or staying at the institution, the researchers could not ascertain whether adoption can overcome privation. Factors such as the children's demeanour or behaviour may have influenced people's choice to adopt them. Researchers cannot control extraneous variables or randomly allocate participants to conditions without manipulation.
Moreover, because the IV is not manipulated, researchers must study the IV conditions as they occur naturally. In other words, they must make do with the participants that present themselves, regardless of participant variables. This immensely lowers internal validity and makes it hard to draw confident conclusions about cause and effect. The use of retrospective data collected for other purposes may introduce inaccuracies, incompleteness, or access difficulties. For these reasons, some people don't regard natural experiments as proper experiments at all.
ETHICS: Natural experiments pose unique ethical challenges because they rely on uncontrolled external events. These challenges include ensuring equitable representation, interpreting results within the context of pre-existing inequalities, and managing inadvertent ethical complexities. Ethical considerations also extend to post-hoc analysis, longitudinal responsibilities, community impact, and the use of public or private data. Researchers must navigate these issues carefully, balancing scientific inquiry with respect for individual privacy, community sensitivities, and the broader societal implications of their work. Ethical rigour in natural experiments demands a nuanced approach to study design, data interpretation, and dissemination of findings, ensuring that research contributes positively to knowledge without compromising ethical standards.
EXAMPLES OF NATURAL EXPERIMENTS
Oregon Health Insurance Experiment:
This study used the Oregon Medicaid lottery, in which some low-income individuals received Medicaid coverage while others did not because of limited slots. Researchers analysed the impact of Medicaid coverage on various health outcomes, healthcare utilisation, financial strain, and overall well-being. The study used a randomised controlled design, with individuals selected via a lottery.
Reference: Finkelstein, A., Taubman, S., Wright, B., Bernstein, M., Gruber, J., Newhouse, J. P., Allen, H., Baicker, K., Oregon Health Study Group. (2012). The Oregon Health Insurance Experiment: Evidence from the First Year. Quarterly Journal of Economics, 127(3), 1057–1106.
Impact of Hurricane Katrina on Birth Outcomes:
This study examined the effects of Hurricane Katrina on birth outcomes by comparing data from areas affected by the hurricane to unaffected areas. Researchers investigated various birth outcomes such as low birth weight, preterm birth, and infant mortality rates. The study found significant adverse effects on birth outcomes in areas affected by the hurricane.
Reference: Currie, J., Rossin-Slater, M. (2013). Weathering the Storm: Hurricanes and Birth Outcomes. Journal of Health Economics, 32(3), 487–503.
Effect of Smoking Bans on Heart Attacks:
This study investigated the impact of smoking bans on the incidence of heart attacks by comparing data from areas with and without smoking bans. Researchers analysed hospital admissions for heart attacks before and after the implementation of smoking bans and found a significant decrease in heart attack rates following the bans.
Reference: Bartecchi, C. E., MacKenzie, T. D., Schrier, R. W. (2006). The Effects of Cigarette Smoking on Dose-Response and Magnitude of Glomerular Filtration Reductions with Advanced Age. Journal of the American Society of Nephrology, 17(6), 158S-163S
WHY NATURAL EXPERIMENTS ARE NOT A TRUE EXPERIMENTS
The observed differences may reflect pre-existing characteristics of the groups rather than the effect of the naturally occurring independent variable. The IV was not created or manipulated by the researcher, and participants were already in their conditions due to real-world circumstances. As a result, uncontrolled group differences undermine internal validity.
Therefore, the study is classified as a natural experiment rather than a true experiment. Causal conclusions remain tentative because alternative explanations cannot be ruled out.
MORE EXAMPLES
Comparing mental health outcomes in individuals before and after a natural disaster. The disaster is the IV, mental health outcomes the DV. Random allocation is impossible (people cannot be assigned to experience a disaster), so groups may differ in prior vulnerability, support networks, or exposure levels.
Assessing the impact of a new law on crime rates by comparing data before and after its introduction. The law is the IV, crime rates the DV. Random allocation is impossible (citizens cannot be assigned to different legal systems), so differences may reflect broader social or economic changes.
Investigating health outcomes in populations exposed to air pollution compared with those who are not. Exposure is the IV, health outcomes the DV. Random allocation is impossible (people cannot be assigned to polluted environments), so groups may differ in income, lifestyle, or access to healthcare
NATURAL EXPERIMENTS VERSUS NON-EQUIVALENT QUASI-EXPERIMENTS
Many teachers and students confuse natural experiments with non-equivalent quasi-experiments because neither type of experiment can randomly allocate participants to conditions and, as a result, have non-equivalent groups (group differences). Superficially, they look similar. However, the independent variable has a fundamentally different cause, and this distinction is often insufficiently understood. The crucial difference concerns control over the independent variable. In a natural experiment, the independent variable occurs naturally. The researcher does not create, choose, or manipulate it. The researcher simply studies an event or condition that already exists in the real world. For example, looking at PTSD in New Yorkers before and after 9/11. Clearly, the researcher did not manipulate or cause 9/11 to happen (this would be insane), but is studying this devastating event nevertheless and cannot randomise participants to before 9/11 and after 9/11. The most common form of quasi-experiment is used when random assignment is not possible or ethical, for example, using different therapies in different hospitals, resulting in groups that may differ on key characteristics before any treatment is applied. In a non-equivalent quasi-experiment, however, the researcher manipulates or selects the independent variable but cannot randomly allocate participants to conditions. The researcher is deliberately testing something, even though randomisation is not possible.
THE DIFFERENCE BETWEEN NATURAL EXPERIMENTS AND QUASI-EXPERIMENTS
Natural experiments lack random allocation and are therefore often classified as quasi-experiments. However, there is an important distinction between them. In a natural experiment, the lack of random allocation is retrospective. The groups have already been formed by real-world events (such as a natural disaster, a new law, or a factory closure). The researcher cannot randomly assign people to groups because the conditions already exist — they simply study the groups as they are. In contrast, a quasi-experiment is prospective. The researcher deliberately manipulates or selects the independent variable from the outset when designing the study. The lack of random allocation is a conscious choice made by the researcher at the planning stage, not a result of real-world circumstances. This difference in timing and control is the key nuance: natural experiments are observational studies of events that have already happened, whereas quasi-experiments are planned studies where the researcher actively chooses the conditions.
HODGES AND TIZARD AS A NATURAL EXPERIMENT
Hodges and Tizard’s study of institutionalised children provides a clear example of a natural experiment. The independent variable was whether children who grew up in an institution were either adopted, sent back to their original homes, or stayed at the institution. These circumstances already arose in real-life social and caregiving situations. Hodges and Tizard simply studied the long-term effects of naturally occurring differences in early attachment experience. In addition, the researcher did not randomly assign children to these conditions, as this would have been extremely unethical. If an IV occurs naturally, participants are also assigned to conditions. Randomisation is not possible because the groups pre-existed. Natural experiments are retrospective.
WHY NON-EQUIVALENT QUASI-EXPERIMENTS ARE DIFFERENT
The key distinction is that natural experiments are typically retrospective, whereas non-equivalent quasi-experiments are typically prospective.
In a natural experiment, the independent variable has already occurred before the researcher begins the study. The researcher looks back at naturally formed groups and examines their outcomes. The groups pre-exist, and the researcher has no control over how they were formed. Hodges and Tizard is a retrospective study because the children had already been adopted, returned home, or remained institutionalised due to social and legal processes outside the researchers’ control. The researchers could not have randomly allocated participants to conditions because the groups already existed. The same is true of the 9/11 example: the groups that existed before and after the event were formed naturally, even though the independent variable itself was a real-world event. The researcher did not determine the IV, and the groups were naturally assigned. In contrast, a non-equivalent quasi-experiment is prospective. The researcher deliberately introduces or selects the independent variable and then measures its effects. However, participants still cannot be randomly allocated.
To make this explicit, using Hodges and Tizard, if the study had been a non-equivalent quasi-experiment, the researchers would have determined the independent variable themselves. For example, they might have decided that the IV would be adoption, return home, or continued institutional care in order to test the effects of these caregiving environments. However, they would have been unable to randomly assign children fully because of ethical constraints. You cannot decide who is adopted and who remains institutionalised. That would have been prospective and researcher-determined. But that is not what happened. In the actual study, the caregiving outcomes had already occurred. The researchers simply followed up with the children and assessed their attachment and social functioning later. The IV arose naturally, and the study examined its consequences retrospectively.
Therefore:
Natural experiments are retrospective because the IV is not determined by the researcher, but the groups are naturally assigned.
Non-equivalent quasi-experiments are prospective, so the IV is determined or selected by the researcher, but the groups are naturally assigned due to ethical or practical constraints.
Thus, natural experiments are often classified as quasi-experiments, but not all quasi-experiments are natural experiments.
SUMMARY OF THE DISTINCTION
Natural experiments are retrospective: the IV is not determined by the researcher, and groups are assigned by naturally occurring events.
Non-equivalent quasi-experiments are prospective: the IV is determined or selected by the researcher, but groups are naturally assigned due to ethical or practical limitations
