Psych Methods and Statistics

Ethics

the principles of right and wrong that guide an individual in making decisions

utilitarian perspective

ethical decisions should be based on doing the greatest good for the greatest number of people

altruistic perspective

ethical decisions should be based on helping without personal benefit

Egoism perspective

Ethical decisions should be based on acting in accordance with one's own self-interest

benefit analysis

a way of thinking about a problem that compares the costs of an action to the benefits received

beneficence

Actively promoting the welfare of others; an ethical obligation to maximize benefits in research studies

Nonmaleficence

duty to do no harm

Confidentiality

the act of holding information in confidence, not to be released to unauthorized individuals

anonymity

a guarantee in research studies that individual responses cannot be linked back to individual participants

Physical harm

a researcher must consider the physical toll that study participation may have; a potential risk to participants

Psychological harm

a researcher must consider the psychological toll that study participation may have such as stress, negative emotions, or loss of self-esteem; a potential risk to participants

cost of not doing the research

considering the potential beneficial application of study findings when doing a cost-benefit analysis

Justice

fairness in selecting study participants and in determining which participants receive the benefits of participation and which bear the burden of risk.

Autonomy

freely making an informed decision about participation in research

Assent

an active affirmation of a desire to participate from a person who does not have the ability to consent themselves; consent must also be sought from the legal guardian.

Confederate

a person who is given a role to play in a study so that the social context can be manipulated

deception

misleading participants about the true purpose of a study or the events that will actually transpire

Institutional Review Board (IRB)

A committee at each institution where research is conducted to review every experiment for ethics and methodology.

Exempt Review (IRB)

a review of study proposals that pose no risk to subjects; the full IRB is not required to participate

Expedited Review (IRB)

a review of study proposals that pose minimal risk to subjects; one or two IRB members participate

Full Review (IRB)

A review of study proposals that pose more than minimal risk to subjects, that do not qualify for exempt status, and in which the full IRB committee participates.

The Institutional Animal Care and Use Committee

A committee at each institution where research is conducted to review every experiment involving animals for ethics and methodology.

scientific integrity

commitment to intellectual honesty and adherence to ethical principles in scientific research

file drawer problem

a bias in the scientific community to only publish findings that confirm a researcher's hypothesis

Plagiarism

the practice of taking someone else's work or ideas and passing them off as one's own.

Paraphrasing

rewriting or restating another person's ideas or thoughts into your own words

true experiment

an experiment in which the experimenter directly manipulates the IV, with participants in randomized groups

quasi-independent variable

in a non-experimental study, the "independent variable" that is used to create the different groups of scores

quasi-experimental design

Research method similar to an experimental design except that it makes use of naturally occurring groups rather than randomly assigning subjects to groups.

self-report

a series of answers to a questionnaire that asks people to indicate the extent to which sets of statements or adjectives accurately describe their own behavior or mental state

social desirability bias

A tendency to give socially approved answers to questions about oneself.

demand characteristics

cues in an experiment that tell the participant what behavior is expected

retrospective bias

Tendency for people to not remember certain aspects of their lives clearly or to misremember certain aspects

Behavioural measures

measures based on systematic observation of people's actions or animals' activities either in their normal environment or in a laboratory setting

Behavioural Observations

To produce a structured record of what the researcher hears or sees. The target behaviour is broken into behavioural categories that are observable and measurable. All target behaviour is included.

behavioural choice

a behavioural measure involving participants making a purposeful selection from several options

participant reactivity

participants act differently or unnaturally because they know someone is watching them

unobtrusive measures

ways of observing people so they do not know they are being studied

raw score

A test score that has not been transformed or converted in any way

true score

An individual's actual score on a variable being measured, as opposed to the score the individual obtained on the measure itself.

Error

extraneous influences that cause the raw score to deviate from the true score.

random error

an error that occurs when the selected sample is an imperfect representation of the overall population

systematic error

unidirectional error in measurement arising due to bias introduced by the investigator or the subjects under investigation

Standardization

defining uniform testing procedures and meaningful scores by comparison with the performance of a pretested group

Observer bias

systematic errors in observation that occur because of an observer's expectations

Sensitivity

the range of data a researcher can gather from a particular instrument.

Ceiling/floor effect

Measurement problem whereby the researcher cannot measure the effects of an independent variable or a possible interaction effect because performance has reached a maximum (minimum) in any condition of the experiment.

Reliability

the extent to which a test yields consistent results, as assessed by the consistency of scores on two halves of the test, on alternate forms of the test, or on retesting

Validity

the extent to which a test measures or predicts what it is supposed to

Population

the entire group of individuals or instances about whom we hope to learn

Sample

A relatively small proportion of people who are chosen in a survey so as to be representative of the whole.

Sampling Plan

The course of action for gathering marketing information from a section or portion of a target market

representative sample

a sample that accurately reflects the characteristics of the population as a whole

probability sampling

A type of sampling in which every element in the population being studied has a known chance of being selected for study

simple random sampling

every member of the population has an equal probability of being selected for the sample

stratified random sampling

A form of probability sampling; a random sampling technique in which the researcher identifies particular demographic categories of interest and then randomly selects individuals within each category.

cluster random sampling

dividing the total population into groups (or clusters), then using simple random sampling to select which clusters participate; all observations in a selected cluster are included in the sample

nonprobability sampling

a sampling technique in which there is no way to calculate the likelihood that a specific element of the population being studied will be chosen

convinience sampling

create a sample by using data from population members that are readily available

quota sampling

An interviewer or researcher selects a sample that reflects the characteristics of the whole population

purposive sampling

selecting sample members to study because they possess attributes important to understanding the research topic

Snowball sampling

recruitment of participants based on word of mouth or referrals from other participants

Nonresponse bias

bias introduced to a sample when a large fraction of those sampled fails to respond

volunteer subject problem

The subjects that volunteer for our studies might be unrepresentative of our population

college sophomore problem

an external validity problem that results from using mainly college sophomores as subjects in research studies

behavioral trace

a behavioral measure that relies on evidence left behind by a participant who is no longer present

Qualitative Research

research that relies on what is seen in field or naturalistic settings more than on statistical data

Quantitative Research

research that collects and reports data primarily in numerical form

mixed methods research

uses both quantitative and qualitative techniques, in an effort to build convincing claims about the relationships between attributes and outcomes

Triangulation

the use of multiple research methods as a way of producing more reliable empirical data than are available from any single method

bottom-up approach

the researcher develops a theory by exploring a topic using information provided from participants' direct experiences.

top-down approach

the researcher tests preconceptions and previously established theories with the collected data.

Situated Analysis

an approach where the researcher examines a topic while it is embedded within its naturally occurring context

holistic analysis

an approach where the researcher examines how numerous properties contribute to patterns within the larger and more complex system

phenomenological approach

assumes that one has to appreciate individuals' personal, subjective experiences to truly understand their behavior

action research

A research design that explicitly involves participants in the research and tries to change some aspect of the research's focus

Case study

an observation technique in which one person is studied in depth in the hope of revealing universal principles

content analysis

applying a systematic approach to record and value information gleaned from secondary data as it relates to the study at hand

conversation analysis

the study of how participants in social interaction recognize and produce coherent conversation

Ethnography

the method by which researchers attempt to understand a group or culture by observing it from the inside, without imposing any preconceived notions they might have

Focus group

A small group of individuals who are led in discussion by a professional consultant in order to gather opinions on and responses to candidates and issues.

Grounded theory technique

A research method where the researcher has no preconceived theory about how the data are connected. A theory is built from the data.

interview

A face-to-face or telephone questioning of a respondent to obtain desired information.

Narrative analysis

a form of qualitative analysis in which the analyst focuses on how respondents impose order on the flow of experience in their lives and so make sense of events and actions in which they have participated

Postmodern approach

a perspective that explicitly questions basic assumptions about the nature and capabilities of research ( is it possible to truly understand a person's experience)

visual ethnography

a type of ethnography that combines observations and interactions with visual media such as photographs or videos

Archival research

Researchers can examine data that has already been collected for other purposes.

collective case study

study of multiple cases for the purpose of comparison

Descriptive case study

a study that aims to chronicle or describe some aspect of reality; involves questions of who, what, where and when

explanatory case study

To describe and find possible explanations for the phenomenon under investigation

exploratory case study

A case study that seeks to outline the potential outcomes of an intervention

Instrumental case study

case study in which the researcher studies a case in order to understand something more general than the particular case

intrinsic case study

a case is examined in depth due to some inherent interest in learning about that particular case

structured interview

a research procedure in which all participants are asked to answer the same questions

unstructured interview

no fixed set of questions and no systematic scoring procedure involves asking probing questions to find out what the applicant is like

semistructured interview

An interview in which the interviewer determines the major questions beforehand, but allows sufficient flexibility to probe into other areas as needed to evaluate an applicant's personality.

Critical incident technique

an interview technique where the researcher purposefully has the interviewee focus on a key event or specific behavior.

Unit of analysis

major entity under investigation or type of data (e.g., individual or group, etc.) that is the focus of the study.

Interviewer bias

A response bias that occurs because the presence of the interviewer influences respondents' answers.

Interview Schedule/Agenda

a type of protocol that includes the questions to ask and anticipated order in which the interviewer should ask them

descriptive research

research conducted to clarify the characteristics of certain phenomena to solve a particular problem (The "what")

experimental research

research designed to discover causal relationships between various factors (The "why")

systematic observational research

the viewing and recording of a predetermined set of behaviors

external validity

extent to which we can generalize findings to real-world settings

laboratory observation

research method in which all participants are observed under the same controlled conditions

ecological validity

The extent to which a study is realistic or representative of real life.

participant observation

a research method in which investigators systematically observe people while joining them in their routine activities

naturalistic observation

watching behavior in real-world settings without trying to manipulate the situation

nonconcealed observation

a participant observation in which observers inform participants that they are under observation

participant reactivity

participants act differently or unnaturally because they know someone is watching them

concealed observation

a participant observation in which the observer never reveals to the participants that they are under observation

blind observation

when observers are trained to look for particular behaviors but are uninformed about study expectations or the overall purpose of the research investigation

intra-observer reliability

A measure of the extent to which an observer codes a variable the same way each time they observe it.

pilot testing

preliminary, exploratory testing that is done prior to the complete research project

Inter-observer reliability

The extent to which two or more observers are observing and recording behaviour in the same way

interval recording

A type of behavior recording procedure in which the observation period is divided into a number of consecutive time intervals and the behavior is recorded as occurring or not occurring in each of the intervals.

continuous recording

the observer observes the client continuously throughout the observation period and records each occurrence of the behavior

contrived observation

artificially introducing a variable of interest and unobtrusively observing what happens

descriptive statistics

statistical procedures used to describe characteristics and responses of groups of subjects

categorical variable

a variable that names categories (whether with words or numerals)

frequency distribution

an arrangement of data that indicates how often a particular score or observation occurs

continuous variable

variable for which, in theory, there are an infinite number of values between any two values

central tendency

a number that describes something about the "average" score of a distribution

Mode

the most frequently occurring score(s) in a distribution

Median

the middle score in a distribution; half the scores are above it and half are below it

Mean

the arithmetic average of a distribution, obtained by adding the scores and then dividing by the number of scores

Cohen's kappa coefficient

A statistic that reflects the extent to which the scores of two raters are consistent with one another

Bar chart

A form of graph in which numeric values are represented by horizontal or vertical rectangles.

coding system

a set of rules to help guide how the researcher classifies and records behaviors under observation

correlational study

a research project designed to discover the degree to which two variables are related to each other, but do not allow for cause-and-effect conclusions

survey

a technique for ascertaining the self-reported attitudes or behaviors of a particular group, usually by questioning a representative, random sample of the group

open-ended questions

questions a person is to answer in his or her own words

closed-ended questions

questions a person must answer by choosing from a limited, predetermined set of responses

scale

The numbers assigned to illustrate the degree to which a participant embodies a specific variable

Likert Scale

a numerical scale used to assess attitudes; includes a set of possible answers with labeled anchors on each extreme (Typically 5 to 9 response options)

Response set

a tendency to respond to questions in a particular way that is unrelated to the content of the questions

Acquiescent response set

a response bias where a participant tends to agree with most, if not all, of the items on a scale, regardless of what they are asking

Forced choice scale

a scale where a person must choose between only two response alternatives for each item

error of central tendency

a response bias where a participant tends to avoid using the extreme response alternatives on a scale

reverse coding

a scoring strategy where more negative response alternatives are assigned higher numerical values and more positive response alternatives are assigned lower numerical values; used to minimize the potential for an acquiescent response set

internal consistency reliability

assesses whether the items on a test are related to one another

test-retest reliability

a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions

Correlation

A measure of the extent to which two factors vary together, measured on a scale of -1 to +1

alternative-form method

a method of calculating reliability by repeating different but equivalent measures at two or more points in time

face validity

the extent to which a test item appears to fit the particular trait it is measuring

Content validity

Whether or not the items on a scale measure all elements of the variable it is seeking to measure

Construct validity

The extent to which there is evidence that a test measures a particular hypothetical construct.

convergent validity

the measure should correlate more strongly with other measures of the same constructs

discriminant validity

scores on the measure are not related to other measures that are theoretically different

criterion validity

an empirical form of measurement validity that establishes the extent to which a measure is correlated with a behavior or concrete outcome that it should be related to

concurrent validity

the degree to which the measures gathered from one tool agree with the measures gathered from other assessment techniques

predictive validity

The success with which a test predicts the behavior it is designed to predict; it is assessed by computing the correlation between test scores and the criterion behavior.

nondirectional hypothesis

research hypothesis that does not predict a particular direction of difference between the population like the sample studied and the population in general

directional hypothesis

a hypothesis that makes a specific prediction about the direction of the relationship between two variables

evaluation apprehension

Anxiety participants experience about how their answers or behaviour appear to the researcher

Distractor items

items included in a scale to mislead participants as to the real purpose of the scale

area probability sampling

a sampling strategy where the researcher first divides the population into subgroups based on geographic area and then randomly selects participants from each geographical subgroup

Web surveys

Surveys accessed online. Cheaper and more accessible than in-person surveys. However, the impersonal nature can reduce response rate, and create nonresponse bias

response rate

the percentage of people contacted who complete the questionnaire

Study incentives

A reward given for completing a study that encourages participation (eg. Money, gift cards, class credit)

Cronbach's alpha

An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question. Acceptable level is at least 0.70

Variability

in a set of numbers, how widely dispersed the values are from each other and from the mean

standard deviation

a measure of variability that describes an average distance of every score from the mean

statistical hypothesis testing

A method for determining how likely the results were to have occurred by chance given the prediction of the researcher.

statistical significance

a statistical statement of how likely it is that an obtained result occurred by chance

double-barreled questions

questions that attempt to get at multiple issues at once, and so tend to receive incomplete or confusing answers