KEY CONCEPTS AND IDEAS IN CHAPTER 10
[NOTE: Numbers in parentheses refer to pages in the fifth edition of Research Methods in Psychology by Shaughnessy, Zechmeister, and Zechmeister (2000), where these concepts and ideas are more fully defined.]
applied research (see basic vs. applied research)
basic vs. applied research Whereas basic research mainly seeks knowledge about nature simply for the sake of understanding it better, applied research seeks knowledge that will modify or improve the present situation; however, basic and applied research are considered to have a reciprocal relationship, for example, when basic research is used to identify abstract principles that can be applied in "real-world" settings, and when applied research is used to help reveal possible limitations or extensions of these principles. (372)
contamination When there is communication of information about the experiment between groups of participants. (381)
Hawthorne effect Refers to changes in a person's behavior brought about by the interest shown in that person by significant others. (382)
nonequivalent control group design A quasi-experimental procedure in which a comparison is made between control and treatment groups that have been established on some basis other than through random assignment of subjects to groups. (385)
program evaluation Research that seeks to determine whether a change proposed by an institution, government agency, or other unit of society is needed and likely to have an effect as planned, or, when implemented, to actually have an effect. (403)
quasi-experiments Procedures that resemble the characteristics of true experiments, for example, that some type of intervention or treatment is used and a comparison is provided, but are lacking in the degree of control that is found in true experiments. (383)
regression to the mean Because some component of a test score is due to error (as opposed to true score), extreme scores on one test are likely to be closer to the mean on a second test, thus posing a threat to the validity of an experiment in which extreme scores are selected; the amount of regression will be greater for less reliable tests. (378)
simple interrupted time-series design A quasi-experimental procedure in which changes in a dependent variable are observed for some period of time both before and after a treatment is introduced. (394)
threats to internal validity Possible causes of a phenomenon which must be controlled so that a clear cause-effect inference can be made. (376)
time series with nonequivalent control group (see also simple interrupted time-series design) A quasi-experimental procedure that improves on the validity of a simple time-series design by including a nonequivalent control group; both treatment and comparison groups are observed for a period of time both before and after the treatment. (398)
Differences Between Laboratory Experiments and Experiments in Natural Settings (370)
Characteristics of True Experiments (374)
- Experiments conducted in a laboratory setting differ from experiments conducted in a natural setting in terms of control, external validity, goals, and consequences.
- Researchers usually have less control over experimental conditions in natural settings than in laboratories.
- As control decreases, researchers' ability to make causal inferences also decreases.
- Research is conducted in natural settings to establish the external validity of findings from laboratory studies.
- Basic research is carried out with the goal of understanding a phenomenon, whereas applied research is usually done with the goal of improving the present situation.
- Research conducted in natural settings tends to have greater impact on the lives of more individuals than research conducted in laboratories.
Obstacles to Conducting True Experiments in Natural Settings (374)
- In true experiments, researchers manipulate an independent variable with treatment and comparison condition(s) and exercise a high degree of control (especially through random assignment to conditions).
Threats to Internal Validity Controlled by True Experiments (376)
- Researchers may experience difficulty gaining access to participants and permission to conduct true experiments in natural settings.
- Although random assignment is viewed by some as unfair because it may deprive individuals from a new treatment, it is still the best way and fairest way to determine if a new treatment is effective.
Problems That Even True Experiments May not Eliminate (380)
- Threats to internal validity are confoundings that serve as plausible alternative explanations for a research finding.
- The major classes of threats to internal validity include history, maturation, testing, instrumentation, regression, selection, subject mortality, and interactions with selection.
- Threats to internal validity that can occur in any study include contamination, experimenter expectancy effects, and the Hawthorne effect.
- Contamination occurs when information about the experiment is communicated between groups of participants, which may lead to resentment, rivalry, or diffusion of treatment.
- Threats to external validity occur when treatment effects may not be generalized beyond the particular people, setting, and time of the experiment.
- The Hawthorne effect occurs when people's behavior changes simply because they believe someone (e.g., researchers, persons in authority) is interested in them.
The Nonequivalent Control Group Design (385)
- Quasi-experiments provide an important alternative when true experiments are not possible.
- Quasi-experiments lack the degree of control found in true experiments; most notably, quasi-experiments typically lack random assignment.
- Researchers must seek additional evidence to eliminate threats to internal validity when they do quasi-experiments rather than true experiments.
- The one-group pretest-posttest design is called a preexperimental design or a bad experiment because it has so little internal validity.
Nonequivalent Control Group design: The Langer and Rodin Study (387)
- In the nonequivalent control group design, a treatment group and a comparison group are compared using pretest and posttest measures.
- If the two groups are similar in their pretest scores prior to treatment but differ in their posttest scores following treatment, researchers can more confidently make a claim about the effect of treatment.
- Threats to internal validity due to history, maturation, testing, instrumentation, and regression can be eliminated in a nonequivalent control group design.
Sources of Invalidity in the Nonequivalent Control Group Design (389)
- Quasi-experiments often assess the overall effectiveness of a treatment that has many components; follow up research may then determine which components are critical for achieving the treatment effect.
The Issue of External Validity (394)
- To interpret the findings in quasi-experimental designs, researchers examine the study to determine if any threats to internal validity are present.
- The threats to internal validity must be considered when using the nonequivalent control group design include interactions with selection, differential regression, observer bias, contamination, and Hawthorne effects.
- Although groups may be comparable on a pretest measure, this does not ensure that the groups are comparable in all possible ways that are relevant to the study outcome.
Interrupted Time-Series Designs (394)
- Similar to internal validity, the external validity of research findings must be critically examined.
- The best evidence for the external validity of research findings is replication with different populations, settings, and times.
Time Series with Nonequivalent Control Group (398)
- In a simple interrupted time series design, researchers examine a series of observations both before and after a treatment.
- Evidence for treatment effects occurs when there are abrupt changes (discontinuities) in the time series data at the time treatment was implemented.
- The major threats to internal validity in the simple interrupted time series design are history effects and changes in measurement (instrumentation) that occur at the same time as the treatment.
An Extension of Single-case Designs to Research in Natural Settings: Rationale (400)
- In a time series with nonequivalent control group design, researchers make a series of observations before and after treatment for both a treatment group and a comparable comparison group.
- Archival data frequently provide the basis for time-series designs.
The ABAB Design (401)
- Both time series designs and single-case experimental designs uses observations before and after a treatment in order to make inferences about the effectiveness of the treatment.
- With both types of design, researchers must consider history threats as a plausible alternative explanation for apparent treatment effects.
- Single-case experimental designs can be conducted as quasi-experiments when groups of individuals are tested using the ABAB design or multiple-baseline design.
Program Evaluation (403)
- When the ABAB design is used as a quasi-experiment, treatment is alternately introduced and withdrawn (baseline) for a group of individuals.
- The effect of treatment can be observed by comparing records for observations made during treatment and baseline periods.
- Program evaluation is used to assess the effectiveness of human service organizations and provide feedback to administrators about their services.
- Program evaluators assess needs, process, outcome, and efficiency of social services.
- Despite society's reluctance to use experiments, true experiments and quasi-experiments are the best approaches for evaluating social reforms.