1. Test the external validity of causal conclusions arrived at in the laboratory
2. To determine the effects of events that occur in the field

--meet growing demands to test the effectiveness of social programs, public health campaigns, and various govt. programs [effectiveness of ecosystem management, new forestry, drainage systems, trail hardening materials, edge effect (biodiversity)

--many social programs are seldom tested for effectiveness

3. To improve generalization across settings

--cannot adequately apply laboratory controls completely or assign subjects to groups at all

--like an experiment but not equal to it

--can sometimes draw causal inference but without the same confidence

Still have 2 groups (control/experimental)

1. state causal hypotheses
2. include at least two levels of the independent variable but cannot always manipulate the independent variable
3. usually cannot assign subjects to groups but must accept already existing groups
4. include specific procedures for testing hypotheses
5. include some controls for threats to validity

--should be similar on most relevant variable to the study

2 major problems

1. the groups may differ on the dependent measures at start of the study

--can be overcome by measuring the experimental and control group on the dependent measure before and after the manipulation (use difference score)

--if the differences are different, then independent may be the source of variation

2. there may be differences between the groups that have not been controlled by random assignment

**looking for change in experimental group, but not control group (crossover effect is best)

--cross-over effect occurs when control/experimental groups start at different levels of dependent variable—change causes experimental measures to "cross" control group measures

--single group of subjects is measured several times both before and after some event or manipulation

--the multiple measures are what adds to the validity of the study

--history and instrumentation are two major confounding variables

--history—any of a number of events could have caused the change in measurement

--instrumentation—record keeping procedures may be the cause of change

--visual interpretation not enough—still require a statistical test of pre-post differences

--not all that relevant to environmental sciences research

--effects of independent variable on a dependent variable are demonstrated by measuring dependent variable over three or four time periods with each time period associated with a treatment/non-treatment

Multiple Baseline Design

--multiple dependent variables in the baseline

Single Subject, Randomized, Time Series Design

--manipulation is selected randomly in the midst of multiple measures of respondent over time

--how successfully does a program meet its goals

--creates accountability

--not a distinct set of research designs but designs are modified to meet context

--usually restricted to observing the public behavior of subjects

--often interested in how effective program is in meeting needs of clients

--program participants generally are not volunteers

--involves ethical dilemmas—is it appropriate to deny some group participation in the program?

--program often administered in a natural setting not under control of researcher

--program staff may resent taking time to evaluate services rather than provide services

--staff may bias evaluation results to make program look better

--clients may inflate value of program to ensure its continuance

-select appropriate dependent measures

--need to use *several* dependent measures because programs often have mutliple goals

--need to minimize bias in dependent measures by using objective measures gathered by individuals not directly involved in program delivery

Randomized Control Group Design

--random assignment of subjects to conditions

Non-equivalent Control Group Design

--identify similar existing group to those not participating in the program

Single Group, Time Series

--if control group not available, use repeated measures on same group

Pretest-Protest

--weak design—no control group and only two measures (**not recommended for program evaluation)