3.7 Experimental designs | Quantitative methods | Research Designs | UvA

Summary

This script discusses the importance of true experiments for maximizing internal validity in research. It outlines key elements such as manipulation of the independent variable, comparison between conditions, and random assignment. The video introduces four common experimental designs: the two-group design, the two-group pre-test post-test design, the Solomon four-group design, and the repeated measures design. Each design is tailored to address potential threats to internal validity, such as maturation and pre-test sensitization, ensuring robust conclusions can be drawn from the study.

Takeaways

• 🔬 A true experiment is essential for maximizing internal validity in research.
• 🧬 Key elements of a true experiment include manipulation of an independent variable, comparison between different conditions, and random assignment to these conditions.
• 📝 Four common experimental designs are discussed: two-group, two-group pre-test post-test, Solomon four group, and repeated measures.
• 🎲 The two-group design involves random assignment to an experimental and control condition, with the independent variable differing between them.
• 📊 The two-group pre-test post-test design adds a pre-assessment of the dependent variable to account for initial differences and potential maturation effects.
• 🚫 The pre-test can sometimes alert participants to the study's purpose or cause a practice effect, impacting internal validity.
• 🤔 The Solomon four group design combines pre-test and no pre-test conditions to account for potential pre-test effects.
• 🔄 The repeated measures design involves participants experiencing all levels of the independent variable, with random assignment to the order of conditions.
• 📐 This design controls for individual differences by using the same subjects in all conditions, but can introduce order effects.
• 📈 The effectiveness of a pre-test can be assessed by comparing results from groups with and without it, informing the design of follow-up studies.
• 📚 The transcript emphasizes the importance of considering various threats to internal validity and the use of different designs to mitigate them.

Q & A

What is a true experiment and why is it important for maximizing internal validity?

A true experiment is a research design that includes manipulation of the independent variable, comparison between conditions with different levels of the independent variable, and random assignment to these conditions. It is important for maximizing internal validity because it helps ensure that any observed effects are due to the independent variable rather than extraneous factors.

What are the key elements of a true experiment?

The key elements of a true experiment are manipulation of the independent variable, comparison between conditions exposed to different levels of the independent variable, and random assignment of participants to these conditions.

What is the simplest experimental design mentioned in the script?

The simplest experimental design mentioned is the two-group design, where participants are randomly assigned to an experimental condition and a control condition.

What is the purpose of the experimental condition in a two-group design?

The purpose of the experimental condition is to introduce the hypothesized cause or independent variable to observe its effect on the dependent variable.

What is the purpose of the control condition in a two-group design?

The control condition serves as a baseline to compare against the experimental condition, where the hypothesized cause is absent, allowing for the assessment of the independent variable's effect.

What is a two-group pre-test post-test design and why is it used?

A two-group pre-test post-test design adds a pretest of the dependent variable before exposure to the independent variable. It is used to check for initial equality between groups and to control for maturation or other threats to internal validity.

What is a potential issue with using a pretest in an experiment?

A pretest can sometimes sensitize participants, leading to a practice effect or alerting them to the study's purpose, which may negatively affect internal validity.

What is the Solomon four group design and how does it address the issues with pretests?

The Solomon four group design is a combination of the two-group design and the two-group pre-test post-test design, run twice—once with a pretest and once without. It helps to account for the unwanted effects of pretests by comparing the results of groups with and without pretests.

What is the repeated measures design and how does it differ from between-subjects designs?

The repeated measures design involves one within-subjects factor where all participants are exposed to all levels of the independent variable. Unlike between-subjects designs, random assignment is not to the conditions themselves but to the order in which the conditions are experienced.

Why might a researcher choose to use a repeated measures design?

A researcher might choose a repeated measures design to control for individual differences between participants, as each participant serves as their own control by experiencing all conditions.

How does the order of conditions in a repeated measures design affect the study?

The order of conditions in a repeated measures design can affect the results due to factors like learning effects, fatigue, or other order-related biases. Random assignment to the order helps to mitigate these effects.

Outlines

00:00

🔬 Maximizing Internal Validity in True Experiments

This paragraph discusses the importance of maximizing internal validity in true experiments through the manipulation of the independent variable, comparison between conditions, and random assignment. It introduces the two-group design as the simplest experimental design, where participants are randomly assigned to either an experimental or control condition. The paragraph also touches on the potential issues with randomization in small groups and introduces the two-group pre-test post-test design to account for maturation threats and to assess natural improvement versus the effect of the independent variable. It further explains the Solomon four-group design, which combines pre-test and post-test conditions to mitigate the effects of pre-test sensitization, and concludes with the repeated measures design, which involves participants experiencing all levels of the independent variable in a within-subjects factor setup.

Keywords

💡True Experiment

A true experiment is a research design that aims to establish a cause-and-effect relationship between variables. In the context of the video, it is described as the best way to maximize internal validity, which is the extent to which the observed effects can be attributed to the independent variable rather than extraneous factors. The script mentions the key elements of a true experiment, such as manipulation of the independent variable and random assignment, which are crucial for ensuring the reliability of the findings.

💡Internal Validity

Internal validity refers to the degree to which a study's design and execution allow for confident conclusions about the cause-and-effect relationship between variables. The video emphasizes that the true experiment is designed to maximize this aspect, by using elements like random assignment and manipulation of variables to control for confounding factors and isolate the effect of the independent variable.

💡Independent Variable

The independent variable is the element that researchers manipulate in an experiment to observe its effect on another variable. In the script, it is highlighted as a key component of a true experiment. For instance, the script discusses the manipulation of the independent variable such as the gender of math teachers to investigate its impact on boys' math performance.

💡Dependent Variable

The dependent variable is the outcome that researchers measure in an experiment to see if it changes in response to the manipulation of the independent variable. The video script uses the example of boys' math performance as the dependent variable, which is measured after exposure to different levels of the independent variable (e.g., male vs. female math teachers).

💡Random Assignment

Random assignment is the process of assigning participants to experimental conditions in a way that each participant has an equal chance of being placed in any condition. This method is crucial for a true experiment, as it helps to control for pre-existing differences between groups, ensuring that any observed effects are due to the independent variable. The script explains that random assignment is used to ensure that conditions are similar in all respects, including their pre-existing position on the dependent variable.

💡Two Group Design

The two group design is a simple experimental setup where participants are randomly assigned to either an experimental group or a control group. The script describes this design as having one condition where the hypothesized cause (independent variable) is present and another where it is absent, allowing for a comparison between the two conditions.

💡Pre-test Post-test Design

A pre-test post-test design involves measuring the dependent variable before and after exposure to the independent variable. The script mentions this design as a way to check for initial equality between groups and to assess the change in the dependent variable over time. It also discusses the potential issues with pre-tests, such as practice effects or sensitization of participants.

💡Solomon Four Group Design

The Solomon four group design is a combination of the two group design and the two group pre-test post-test design. It involves running both experimental and control conditions with and without a pre-test. The script explains that this design can help account for the unwanted effects of pre-tests and provide a more robust assessment of the independent variable's impact.

💡Repeated Measures Design

A repeated measures design is an experimental setup where all participants are exposed to all levels of the independent variable, serving as their own control. The script describes this design as having a within-subjects factor, where the order of conditions is randomly assigned, allowing for the comparison of the same participants' responses to different levels of the independent variable.

💡Maturation

Maturation refers to the natural development or aging process that participants may undergo during the course of a study. In the script, maturation is mentioned as a plausible threat to internal validity, as it could lead to changes in the dependent variable that are unrelated to the independent variable. The use of a pre-test can help control for maturation effects by providing a baseline measure before the experimental manipulation.

💡Practice Effect

The practice effect is a phenomenon where participants' performance improves simply because they have been exposed to the task before, as in taking a pre-test. The script warns that a pre-test may result in a practice effect, leading to higher scores on the post-test, which could confound the results of the experiment if not properly accounted for.

Highlights

A true experiment is essential for maximizing internal validity in research.

Key elements of a true experiment include manipulation of the independent variable, comparison between different conditions, and random assignment.

Four common experimental designs are discussed for implementing these elements.

The simplest design is the two-group design with random assignment to experimental and control conditions.

The independent variable can differ in amount or kind between conditions.

The two-group pre-test post-test design is used to check initial proficiency and control for maturation effects.

A pretest can introduce practice effects or alert participants to the study's purpose, affecting internal validity.

The Solomon four-group design combines pre-test and post-test conditions to account for pretest effects.

This design helps to discern the effect of the independent variable despite practice effects.

A repeated measures design involves all participants experiencing all levels of the independent variable.

In repeated measures, random assignment is to the order of condition experience, not the conditions themselves.

The repeated measures design controls for individual differences by having the same subjects in all conditions.

A follow-up study can refine tests to better show differences between pre-test and non-pretest groups.

Experimental designs must consider threats to internal validity such as maturation and practice effects.

Random assignment is crucial for controlling pre-existing differences in the dependent variable.

The choice of experimental design depends on the research question and potential threats to validity.

Each design has its strengths and limitations in maximizing internal validity and controlling confounding variables.

Transcripts

a true experiment is the best way to

maximize internal validity the key

elements of a true experiment are

manipulation of the independent variable

comparison between conditions exposed to

different levels of the independent

variable and of course random assignment

to these

conditions of course these elements can

be implemented in very different ways

I'll discuss four experimental designs

that are very common the simplest design

is the two group design participants are

randomly assigned to one of two

conditions usually an experimental

condition where the hypothesized cause

is present and a control condition where

it's

absent the independent variable could

also differ between the conditions in

amount or kind for example if we're

investigating the effect of male versus

female Math teachers on math performance

of boys for example

in the two group design the dependent

variable is measured after exposure to

the independent variable to assess the

difference between the conditions which

are likely to be similar in all respects

due to the random

assignment including their pre-existing

position on the dependent

variable of course in small groups

randomization doesn't always work in

such cases it might be wise to use a two

group pre-test post test design which

adds a preest of the dependent variable

before exposure to the independent

variable with a pretest you can check

whether for example both groups of boys

were equally proficient in math before

being exposed to a female versus a male

math teacher for a

month this is an especially good idea

when maturation forms a plausible threat

to internal validity a pretest also

allows the researcher to compare the

size of the increase or decrease in

scores in the experimental and control

condition for example we can assess how

much the boy's math performance

increased due to Natural Improvement and

what the additional effect of teacher

sex

was unfortunately a preest can sometimes

sensitize participants the pretest may

result in a practice effect leading to

higher scores on the post test or it may

alert participants to the purpose of the

study especially if this effect is

stronger for one of the conditions

internal validity will be negatively

affected

but there's a way to take such unwanted

effects of a pretest into account by

using a Solomon four group Design This

is a combination of the two group design

and the two group pre-test postest

design the experimental and control

condition are run twice once with a

preest and once without for example it's

possible that the math test isn't very

hard to begin with and provides good

practice in those math skills that the

boys still lack on the post test the

boys in both conditions get perfect

scores obscuring any effect that teacher

sex might

have if we had two other groups of boys

that didn't take the pre-test we might

see the effect of teacher sex because

these groups have had less

practice of course if we find a

difference between these groups could

still be attributable to an already

existing difference in math proficiency

but together with the results of the

pre-test groups we could come up with a

better more difficult test showing

differences between the two preest

groups and two non- pretest groups in a

follow-up

study another very common design is the

repeated measures design with one within

subjects factor in this design all

participants are exposed to all levels

of the independent variable they

experience all conditions for example we

could randomly select half of the boys

to have a female math teacher for a

month and then a male teacher the

following month and the other half of

the boys would be taught by the male

teacher during the first month and the

female teacher during the second month

the only thing that's really different

to the previous between subjects designs

is that the random assignment of

participants is not to the conditions

themselves because they experience all

of them but to the order in which the

conditions are

experienced