control—any procedure used by researcher to counteract potential threats to validity of research

 statistical validity—when reliable dependent measures are used or when appropriate statistical tests are applied

construct validity—when appropriate constructs are used and accurately operationalized

internal validity—when the research design is appropriate to measure what is intended

external validity—when results can be generalized beyond the study’s subjects

 4 types of controls

1. general control procedures

--preparation of research setting

--response measurement—use known reliable and valid measures

--replication

2. control over subject and experimenter effects

--single blind and double-blind procedures

--single-blind occurs when researcher (assistant) is unaware of research hypotheses and assignment of subjects

--researcher (assistant) is unaware of assignment of subjects and the subjects themselves are blind to their assignments (neither researcher nor subjects know which group they are assigned to)

--test subjects and score data as blindly as possible to avoid experimenter bias

--automation

--use electronic equipment such as tape recorders

--use of objective measures

--base on empirically observable events that two or more people could agree upon

--multiple observers

--interrater reliability coefficients can show percent agreement

--use of deception

--obscure true hypothesis of experiment

3. control through selection and assignment of subjects

--general populationà target populationà accessible populationà sample

--need representative sample

--random sampling—every member of population has equal chance of being selected and that selections do not affect each other (independent)

--stratefied random sampling—separate samples from each of several subpopulations is drawn

--best subject assignment is random

--free random assignment—carried out using random number tables or generators

--matched random assignment—match subjects on important variables (pairs) and then randomly assign one of the pair to a group

1. control through specific experimental design

 True Experiment:

1. states one or more hypotheses about predicted causal effects of independent variable on dependent variable
2. Includes at least two levels of the independent variable
3. assigns subjects to conditions in an unbiased manner (preferably randomly)
4. includes procedures for testing hypotheses
5. includes controls for major threats to internal validity

experimental design controls many sources of unwanted, extraneous, and chance variation

--seeks to control variance to increase internal validity

 variance = sum of squares / degrees of freedom

 Forms of variance

1. systematic between-groups variance

--looking for significantly high variance between groups

--significant difference may be result of

--systematic effects of independent variable (experimental variance)

--systematic effects of confounding variables (extraneous variance)

--combination of the two

--should seek to maximize experimental variance and control extraneous variance

2. nonsystematic within-groups variance (error variance)

F Statistic = measure based on between-groups variation / measure based on within-groups variation

 F = systematic effects + error variance / error variance

 F = 1.0 (no systemtic effects)

 **rule in experimentation—maximize experimental variance, control extraneous variance, and minimize error variance

--do manipulation check of the independent variable—make sure that independent variable has variation between control and experimental groups

--make sure independent variable is only difference in treatment of control and experimental groups

--randomly assign subjects to conditions

--eliminate confounding variable (e.g., social class) by selecting subjects who are as homogenous as possible (cost is generalizability)

--build it into experiment as additional variable (factorial design)

--matching subjects

To minimize error variance

--maintain carefully controlled conditions of measurement

 Non-experimental Designs

1. Ex post facto Design

Group A (Naturally Occuring Events) à Measurement

2. Single Group, Posttest Only Design

--single group

3. Single Group, Pretest-Posttest Design

4. Pretest-Posttest, Natural Control Group Design

Experimental Designs (Testing One Independent Variable)

--subjects must be randomly assigned to conditions

1. Randomized, Posttest Only, Control Group Design

2. Randomized, Pretest—Posttest, Control Group Design

3. Multi-level, Completely Randomized, Between Subjects Design

4. Solomon Four Group Design

Statistical Analysis of Randomized Designs

t-tests—two or more groups

ANOVA—more than two groups

F = Mean square between groups / Mean square within groups

--F will be large when between group variation is much larger than within group variation