it can not determine temporal precedence or that there is a third variable causing the correlation
why can’t a simple bivariate correlational study meet all 3 criteria for establishing causation?
it studies an association between more than two measured variables
why is a longitudinal design considered a multivariate design?
cross-sectional correlations
autocorrelations
cross-lag correlations
what are the 3 kinds of correlations obtained from a longitudinal design?
cross-sectional correlations - test if two variables measured at same point in time are correlated
autocorrelations - evaluate correlation of each variable with itself across time
cross-lag correlations - show if earlier measure of one variable is associated with the later measure of the other variable (address directionality problem + establish temporal precedence)
what does each of the 3 kinds of correlations in a longitudinal design represent?
first variable was correlated to the second
second variable could lead to the first
variables are mutually reinforcing
describe which patterns of temporal precedence are indicated by different cross-lag correlational results
keep a variable the same / stable / constant in each group for the experiment (like age, temperature, etc.)
what does it mean to say that some variable “was controlled for” in a mutlivariate study
1
how many criterion variables are there in a multiple-regression analysis?
at least 2
how many predictor variables are there in a multiple-regression analysis?
when 95% CI does not include 0 we can say the beta is statistically significant - p < .05
what is the relationship between the 95% CI for beta and the beta’s statistical significance?
“controlled for”
“adjusting for”
“considering”
what are 3 phrases indicating a study used a multiple-regression analysis?
cannot always establish temporal precedence
researchers cannot control for variables they do not measure
what are 2 reasons multiple-regression analyses cannot completely establish causation?
regression cannot control for every possible third variable
some experiments would not be ethical or practical
why do many researchers find pattern and parsimony an effective way to support a causal claim?
report on the entire body of evidence as well as the theoretical background
what is a responsible way for journalists to cover single studies on a specific topic?
1
a headline about social media use makes the following bivariate association claim: “social media use is linked to lower grades in college.” the two variables in this headline are:
social media use and quality of grades
high school media use or low social media use
good grades or poor grades
1
suppose a researcher uses a longitudinal design to study the relationship between social media use (e.g., Instagram and Snapchat) and grades over time. she measures both of these variables in Year 1 and then measures both variables again in Year 2. which of the following is an example of an autocorrelation in the results?
The correlation between social media use in Year 1 and social media use in Year 2.
The correlation between social media use in Year 1 and grades in Year 2.
The correlation between grades in Year 1 and social media use in Year 2.
The correlation between grades in Year 1 and social media use in Year 1.
3
consider this statement: “People who use social media got worse grades in college, even when the researchers controlled for the level of college preparation (operationalized by SAT scores) of the students.” what does it mean?
Social media use and grades are correlated only because both of these are associated with SAT score.
SAT score is a third variable that seems to explain the association between social media use and grades.
SAT score can be ruled out as a third variable explanation for the correlation between social media use and college grades.
3
which of the following statements is an example of a mediator of the relationship between social media use and college grades?
Social media use and college grades are more strongly correlated among nonathletes, and less strongly correlated among athletes.
Social media use and college grades are only correlated with each other because they are both related to the difficulty of the major. Students in more difficult majors get worse grades, and those in difficult majors have less time to use social media.
Social media use and college grades are correlated because social media use leads to less time studying, which leads to lower grades
2
a news outlet reported on a study of people with dementia. the study found that among patients with dementia, bilingual people had been diagnosed 3–4 years later than those who were monolingual. what are the variables in this bivariate association?
Being bilingual or monolingual
Being bilingual or not, and age at dementia diagnosis
Age at dementia diagnosis
2
the journalist reported that the relationship between bilingualism and age at diagnosis did not change, even when the researchers controlled for level of education. what does this suggest?
That the relationship between bilingualism and dementia onset is probably attributable to the third variable: level of education.
That the relationship between bilingualism and dementia onset is not attributable to the potential third variable: level of education.
That being bilingual can prevent dementia.
1
researchers speculated that the reason bilingualism is associated with later onset of dementia is that bilingual people develop richer connections in the brain through their experiences in managing two languages, and these connections help stave off dementia symptoms. This statement describes:
A mediator
A moderator
A third variable
multivariate design
a study designed to test an association involving more than two measured variables
longitudinal design
study where the same variables are measured in the same people at different points in time
cross-sectional correlation
in a longitudinal design, a correlation between two variables that are measured at the same time
autocorrelation
in a longitudinal design, the correlation of one variable with itself, measured at two different times
cross-lag correlation
in a longitudinal design, a correlation between an earlier measure of one variable and a later measure of another variable
multiple regression
statistical technique that computes the relationship between a predictor variable and a criterion variable, controlling for other predictor variables
a.k.a. multivariate regression
control for
holding a potential third variable at a constant level while investigating the association between two other variables
criterion variable
variable in a multiple-regression analysis that the researchers are most interested in understanding or predicting
a.k.a. dependent variable
predictor variable
variable in multiple-regression analysis that is used to explain variance in the criterion variable
a.k.a. independent variable
parsimony
degree to which a theory provides the simplest explanation of some phenomenon
in context of investigating a claim, the simplest explanation of a pattern of data
the best explanation that requires making the fewest exceptions or qualifications
mediator
variable that helps explain the relationship between two other variables
a manipulated variable and a measured variable
what are the minimum requirements for a study to be an experiment?
experiments establish covariance - answer ‘compared to what’
establish temporal precedence - researchers manipulate the independent variable to ensure it comes first
establish internal validity - ensure causal variable responsible for change in outcome variable
why do experiments usually satisfy 3 causal criteria?
control variables are used to eliminate potential design confounds
how are design confounds and control variables related?
each participant has an equal chance of being selected for each condition / group
desystematizing types of participants who end up in each level
creates situation where experimental groups will become virtually equal
how does random assignment prevent selection effects?
ensures groups are equal on some important variable before the manipulation of the independent variable
how does using matched groups prevent selection effects?
satisfy all three criteria & randomly assigned
describe why posttest-only and pretest/posttest designs are both independent-groups designs
independent-groups - separate groups of participants are placed into different levels of independent variable
within-groups - each participant is presented with all levels of independent variable
what is the difference between independent-groups and within-groups designs?
convince researchers their experimental manipulation worked
help researchers determine whether the operationalization worked as intended
how to manipulation checks provide evidence for construct validity of an experiments independent variable?
evaluate how well measures and manipulations researchers used in their study capture the conceptual variables in their theory
why does theory matter in evaluating construct validity?
generalization to other situations
what other aspect of generalization does external validity address?
experiment
study where at least one variable is manipulated and another is measured
manipulated variable
variable in an experiment that a researcher controls, such as by assigning participants to its different levels
measured variable
variable in a study whose levels are observed and recorded
independent variable
in an experiment, a variable that is manipulated
in a multiple-regression analysis, a predictor variable used to explain variance in the criterion variable
condition
one of the levels of the independent variable in an experiment
dependent variable
in an experiment, the variable that is measured
in a multiple-regression analysis, the single outcome or criterion variable the researchers are most interested in understanding or predicting
a.k.a. outcome variable
control variable
in an experiment, a variable that a researcher holds constant on purpose
comparison group
group in an experiment whose levels on the independent variable differ from those of the treatment group in some intended and meaningful way
a.k.a. comparison condition
control group
a level of an independent variable that is intended to represent “no treatment” or a neutral condition
a.k.a. control condition
treatment group
participants in an experiment who are exposed to the level of the independent variable that involves a medication, therapy, or intervention
placebo group
control group in an experiment that is exposed to an inert treatment, such as a sugar pill
confound
general term for a potential alternative explanation for a research finding
threat to internal validity
design confound
a threat to internal validity in an experiment where a second variable happens to vary systematically along with the independent variable and therefore is an alternative explanation for the results
systematic variability
in an experiment, a description of when the levels of a variable coincide in some predictable way with experimental group membership, creating a potential confound
unsystematic variability
in an experiment, a description of when the levels of a variable fluctuate independently of experimental group membership, contributing to variability within groups
selection effect
a threat to internal validity that occurs in an independent-groups design when the kinds of participants at one level of the independent variable are systematically different from those at the other level
random assignment
use of a random method to assign participants into different experimental groups
matched groups
an experimental design technique where participants who are similar on some measured variable are grouped into sets & then randomly assigned into different experimental conditions
independent-groups design
an experiment design where different groups of participants are exposed to different levels of the independent variable, such that each participant experiences only one level of the independent variable
a.k.a. between-subjects design / between-groups design
within-groups design
an experimental design where each participant is presented with all levels of the independent variable
posttest-only design
an experiment using an independent-groups design where participants are tested on the dependent variable only once
pretest/posttest design
an experiment using an independent-groups design where participants are tested on the key dependent variable twice : once before and once after exposure to the independent variable
repeated-measures design
an experiment using a within-groups design where participants respond to a dependent variable more than once, after exposure to each level of the independent variable
concurrent-measures design
an experiment using a within-groups design where participants are exposed to all the levels of an independent variable at roughly the same time, and a single attitudinal or behavioral preference is the dependent variable
order effect
in a within-groups design, a threat the internal validity where exposure to one condition changes participant responses to a later condition
practice effect
type of order effect where participants’ performance improves over time because they become practiced at the dependent measure (not because of the manipulation of treatment)
a.k.a. fatigue effect
carryover effect
type of order effect where some form of contamination carries over from one condition to the next
counterbalancing
in a repeated-measures experiment, presenting levels of the independent variable to participants in different sequences to control for order effects
full counterbalancing
a method of counterbalancing where all possible condition orders are represented
partial counterbalancing
a method of counterbalancing where some, but not all, of the possible condition orders are represented
Latin square
formal system of partial counterbalancing to ensure that every condition in a within-groups design appears in each position at least once
demand characteristic
a cue that leads participants to guess a study’s hypotheses or goals
a threat to internal validity
a.k.a. experimental demand
manipulation check
in an experiment, an extra dependent variable researchers can include to determine how well a manipulation worked
pilot study
a study completed before (or sometimes after) the study of primary interest, usually to test the effectiveness or characteristics of the manipulations
no comparison group
how does a one-group, pretest/posttest design differ from a pretest/posttest design?
maturation threats, history threats, regression threats, attrition threats, testing threats, instrumentation threats
which threats to internal validity are especially applicable to the one-group, pretest/posttest design?
observer bias, demand characteristics, placebo effects
which of the internal validity threats would be relevant even to a (two-group) posttest-only design?
weak manipulations, insensitive measures, ceiling & floor effects, reverse design confounds
how can a study maximize variability between independent variable groups?
measurement error, irrelevant individual differences, situation noise
how can a study minimize variability within groups?
if all the participants are exposed to the same thing they are compared to themselves and not compared to other groups which would naturally minimize the unsystematic variability
describe how within-groups designs minimize unsystematic variability
useful to learn which interventions do not work
push to revise theories
when a study results in a null effect should it be published?
1
Dr. Weber conducted a long-term study in which people were tested on happiness, asked to make two new friends, and then tested on happiness 1 month later. he noticed that six of the most introverted people dropped out by the last session. therefore, his study might have which of the following internal validity threats?
Attrition
Maturation
Selection
Regression
4
how is a testing threat to internal validity different from an instrumentation threat?
A testing threat can be prevented with random assignment; an instrumentation threat cannot.
A testing threat applies only to within-groups designs; an instrumentation threat applies to any type of study design.
A testing threat can be prevented with a double-blind study; an instrumentation threat can be prevented with a placebo control.
A testing threat refers to a change in the participants over time; an instrumentation threat refers to a change in the measuring instrument over time.
2
A regression threat applies especially:
When there are two groups in the study: an experimental group and a control group.
When the researcher recruits a sample whose average is extremely low or high at pretest.
In a posttest-only design.
When there is a small sample in the study.
1
Dr. Banks tests to see how many training sessions it takes for dogs to learn to “Sit and stay.” She randomly assigns 60 dogs to two reward conditions: one is miniature hot dogs, the other is small pieces of steak. Surprisingly, she finds the dogs in each group learn “Sit and stay” in about the same number of sessions. Given the design of her study, what is the most likely explanation for this null effect?
The dogs loved both treats (her reward manipulation has a ceiling effect).
She used too many dogs.
She didn’t use a manipulation check.
There were too many individual differences among the dogs.
1
Dr. Banks modifies her design and conducts a second study. She uses the same number of dogs and the same design, except now she rewards one group of dogs with miniature hot dogs and another group with pieces of apple. She finds a big difference, with the hot-dogs group learning the command faster. Dr. Banks avoided a null result this time because her design:
Increased the between-groups variability.
Decreased the within-groups variability.
Improved the study’s internal validity.
4
When a study has a large number of participants and a small amount of unsystematic variability (low measurement error, low levels of situation noise), then it has a lot of:
Internal validity
Manipulation checks
Dependent variables
Power and precision
one-group, pretest/posttest design
an experiment where a researcher recruits one group of participants - measures them on a pretest - exposes them to a treatment, intervention, or change - then measures them on a posttest
maturation threat
threat to internal validity that occurs when an observed change in an experimental group could have emerged more or less spontaneously over time
history threat
threat to internal validity that occurs when it is unclear whether a change in the treatment group is caused by the treatment itself or by an external or historical factor that affects most members of the group
regression threat
threat to internal validity related to regression to the mean, a phenomenon where any extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured (w/ or w/o the experimental treatment or intervention)
regression to the mean
a phenomenon where an extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured, because the same combination of chance factors that made the finding extreme are not present the second time
attrition threat
in a pretest/posttest, repeated-measures, or quasi-experimental study, a threat to internal validity that occurs when a systematic type of participant drops out of the study before it ends
testing threat
in a repeated-measures experiment or quasi-experiment, a kind of order effect where scores change over time just because participants have taken the test more than once (includes practice effects)
instrumentation threat
a threat to internal validity that occurs when a measuring instrument changes over time
selection-history threat
a threat to internal validity where a historical or seasonal even systematically affects only the participants in the treatment group or only those in the comparison group, not both
selection-attrition threat
a threat to internal validity where participants are likely to drop out of either the treatment group or the comparison group, not both
observer bias
a bias that occurs when observer expectations influence the interpretation of participant behaviors or the outcome of the study
demand characteristic
a cue that leads participants to guess a study’s hypotheses or goals (a threat to internal validity)