Research Methods

Why? How?

Explanations and predictions - based on what happens in the past

• personal experience

• Intuition (easy way out)

• Authority (rely on what experts say)

• Appeals to traditions, customs and faith

• Magic, superstitions and mysticism

Insufficient/incomplete data

Either no or biased inquiry

• systematic process→ increase chance conclusion is correct

• Falsifiable theory

• Replication

• Reflective and self-critical approach

• Cumulative and self-correcting process

• Cyclical process

1. Question / puzzle

2. Conceptualization

3. Operationalization

4. Research design

5. Observation

6. Data analysis

7. Interpretation

• From intuitions and anecdotes to systematic evidence

• Question and problem driven

  • Good research question is important

• Research methods are useful tools

  • provide transparency and replicability

• Methodological pluralism and diversity

• Methods as constraints vs opportunities

  • Limit due to very clear guide lines

  • Advantage gives credibility through use

What is the nature of the social world

What can we know about social phenomena

How do we gain/ obtain knowledge

Developed by French philosopher August Comte

search for the truth though systemic collection of observable facts

Scientific study of the social world

• classical

• Logical

• Falsification

• naturalism

• Empiricism

• Laws

  • Establish social laws that allow us to predict people behavior

• Induction (observation → theory)

• Cause and effect relationship (observable ‘constant conjunction’ David Hume)

• Science is objective and value free → establish facts

Social science = natural science

• approach them in the same way

• Observation is KEY

Knowledge of the wold is limited to what can be observed and measured (sensory experience)

Social world is subject to regular and systematic processes; laws are explanatory and predictive

• empiricism (observation) + logical reasoning

• Deduction (theory → observation)

• Retroduction (observation ←→ theory)

• Verification (establishing truth claims) → find support for their reasoning

Karl Popper

• Rejection of induction: not a good way of creating/ testing a theory

  • Particular experience /→ general knowledge

  • One counter-observation and law is falsified → swan example

• Rejection of verifiability: develop our theories with logical resonating , key is more to look for evidence that contradicts the theory

  • Verifying theory is pointless

  • Goal must be falsification

Carl Gustav Hempel - logical positivist

• observed phenomenon is explained if it can be deduced from a universal law-like generalization

• Laws express necessary connections between properties, accidental generalization does not exist

• test ability of law to predict events

• Law → hypothesis → explicit prediction

  • Prediction correct→ hypothesis corroborated / supported

  • Prediction incorrect → hypothesis falsified/ not supported

Response: there is a lot going on that we can not observe objectively

Similarities to positivism:

• social and natural worlds (sciences) are similar

• Realism: ‘objective’ reality exists

Key differences:

• Realist can consist on unobservable elements as well

  • Ex. Structural relationships, legitimacy

• Assessment by observable consequences

  • Ex. Consider the gov legitimate - more likely to vote

• Causal mechanisms instead of law-like generalizations

  • Don’t like to talk about laws- there are always exceptions

  • If a cause is present it makes the theory more likely 

• ‘Best’ theory is the one that explains phenomena the ‘best’

  • Not correct but most likely (highest explanatory value)

Mechanism (Tilly)

• Environmental: externally generated influences on conditions affecting social life

  • Contextual factors impact how political actors act

• Cognitive: operate through alterations of individual and collective perception

  • Protestant mindset worked well with capitalism

  • Stereotypes

• Relational: alter connections among people, groups and interpersonal networks

• Individualism - break everything down to smallest unit of analysis

  • Individuals make up society, we have to study them to understand

• Holism: states can make decisions —> more than just the individual that make up the state but as its own entity

  • More than the sum of its parts —> becomes the actor itself

• Start with general explanations, then go very specific, put all the individual decisions/outcomes together you come back to general outcomes

  • General= macro

  • Specific = micro

• Ex. Democratic peace theory

• Fundamentally different from positivism

  • Social world and natural world are fundamentally different -> different methods needed

• Social world

  • Subjectively created -> the only thing that matters is from the perspective of the research - how they see it

  • Understanding human behavior by interpretation of meaning of social behavior

  • Example of approaches: hermeneutical (reader perspective), critical theory, constructivism, post-colonialism, feminism

  • But: some similarities in collecting evidence and establishing causal relations (?)

• Critical theory: yes, can’t be separated!

• Positivism: well, let’s distinguish:

  • Normative theory (what ought or should be)

  • Empirical theory (what is)

• Robert Cox: all theory is normative

• Max Weber: distinguish, yes, but values cannot be ignored (what is seen as relevant, point of view)

• Transparency

  • Self-aware/disclosure

  • Critical examination by larger scientific community 

    • If you are not aware of your bias, other people will be

• Expectations shape perceptions

  • Image of bunny/duck

• Observation can change (social) phenomena

  • Hawthorn effect

• Everyone should be aware of how humans operate and take this into account during research

if you know you are being observed you change your behavior

• Science is a social institutions

• Scientific community subscribes to a common view, paradigm or conceptual scheme (=‘normal science’)

• Defines objects, norms and methods of investigation

• ‘Truth’ is based on consensus in scientific community

• Paradigm shifts: ‘Revolutionary’ change in paradigms

People like to stick to what they believe- only when they have to admit that they are wrong is when new science can come about

Imre Lakatos - Falsification and the methodology of scientific Research Programs (1970)

• Scientific research programs= incremental, cumulative, progressive articulation of scientific research programs lead to the growth of scientific knowledge

• Hard core with a protective belt of auxiliary hypothesis

• Novel facts: progressive (problem shifts) or degenerating research programs 

Likes a puzzle

Problem, topic, puzzle → general RQ → specific RQ → scientific inquiry

• Process is like a funnel- start wide then you narrow it down

Process = narrow it down, find question

Presentation = journal, article

Two criteria: does the answer to the question have…

1. Scientific relevance /importance

2. Social (real world) relevance/importance

• Should guide and structure the whole research process (’wheel of science’)

• Be researchable (possible to answer)

• New (not answered before)

Tension with relevance of RQ

• Real world events and problems

  • Recent (& historic) events are good choices, but ongoing events are very risky (better avoided)

• Existing literature

  • ‘gaps’ and ‘controversies’

  • Caution: blind acceptance of ‘normal science’ might limit search for better understanding

• “Puzzle” : unexpected contradictions

• Replication (but not fully accepted yet)

1. General RQ/ working hypothesis

2. Literature review

   • What do we already know?

   • What do we not know?

3. Theory/theoretical framework

   • Relevant concepts and factors

   • Expectations and hypotheses

4. Research design (data and sources)

• descriptive

• Explanatory

• Predictive

• Prescriptive

• Normative

• open ended

• Collect information

• Ex. Two factors, are they related to each other

Make claim about causal relationship

If we do this, what are the implications in x years

Ex. Policy implemented to try and lower emissions, what will happen or emissions look like in x years

Goal oriented

Ex. We want to lower emissions→ what policy is needed to reach this

Philosophical questions

Descriptive, explanatory, normative

Predictive and prescriptive

• Core books in library (open stacks?)

• Databases

  • Via library

  • Google scholar

• Reviews/state-of-the-art articles

  • Handbooks/encyclopedias

  • Annual Review of Political Science

  • Then following references (”snowball sampling”)

    • (Wikipedia can be a good starting point)

    • ChatGPT

• Summarize: outline the relevant existing research/knowledge/theories/methods/evidence (topical/thematic review)

• Evaluate: identify the contributions (strengths) and limitations (weaknesses) of existing research

• Conceptualist: use it to define key concepts

• not annotated bibliography or personal diary

• Analytical review

• Develop general into specific RQ/hypothesis

• Simplified model of reality

• Identifies key concepts/factors and their relationship

• “A proposition which has been elaborated and/or has withstood repeated testing”

• A theory is “a set of interrelated constructions, definitions, and propositions that present a systematic view of phenomena by specifying relations among variables, with the purpose of explaining and predicting the phenomena ”

• Scope/level: grand theory vs middle-range theory (Merton 1968)

• Process: inductive vs deductive

• Nature of question: empirical vs normative

• Sounds like a theory but is actually a research method -> qualitative

  • Inductive and qualitative approach: method of inquiry and theory building

• Process

  • Coding: close (tentative) coding of collected data

  • Sorting: compare, sort and synthesize the codes

  • Memo writing: were memos outlining/describing codes

• Apply existing theory to new cases/data

• Revise theory and test with old and new cases/data

• Identify and define concepts/factors and their relationships, usually but not always in the form of hypothesis

•  a proposed explanation for a phenomena

  • Usually by stating some kind of (testable) cause and effect relationship

• specify relationship between factors→ link two or more variables

  • Cause: explanatory factor/ independent variable

  • Outcome: dependent variable

2 concepts not related

Concept A present, B present as well (occur at the same time)

A causes B

Can be reciprocal causation

An empirical regularity - democracies don’t go to war with each other (macro level)

• Hawks and Doves - tested the theory on a individual level

• Explanations

  • Liberal norms (based on reason or culture/norms) 

  • Institutional explanations

    • Institutional constraints (checks and balances)

    • Audience costs (casualties and re-election)

  • System-level explanations

    • Historical context (ex. Cold War)

    • Geographical proximity

    • Economic links/trade

  • Decision-makers (micro foundations)

    • Beliefs, interests, leadership style, etc

Necessary conditions (after Stuart Mill)

• Covariance (correlation)

• Temporal Ordering (time order)

• Spatial and Temporal contiguity/link (process)

• Nonspurious connection (no confounds)

Important: covariance is a necessary but not sufficient condition for causality

X & Y

cause and effect happen at the same time = + go up at same time - one goes up and one goes down

• Does not say anything about being linked only that they occur

Show that the cause changes before the effect takes place

have to prove that X moves first and Y just responds to changes in X

• Crucial to establish causality

Why and how the cause is linked to the effect

• Spatial link- they can effect each other

• Temporal link- makes sense that they can effect each other

Rule out alternative explanations and 3rd factors

• Deterministic vs Probabilistic relationships

  • Natural laws- deterministic

    • Rarely happens in social sciences

• Use caution with nomothetic causality (esp. in social sciences)

  • Complete causation -> unlikely

  • Exceptional cases -> possible

  • Majority of cases -> not required

    • Tends to speak very broadly (ex states/democracy) but might only apply to a small set of cases (ex. Western democracy)

• Key question: intervening and/or anteceding factors?

• Cause: independent variable

• Outcome: dependent variable

• Intervening factors: moderation mediator

• Antecedent factors: confound control 

independent variable → dependent variable

+ Example

Independent variable→ mediator→ dependent variable

IV and DV can have a partial relationship (red line)

+example

Independent variable→ moderator→ dependent variable

Moderator: can reinforce (+) or suppress(-) relationships, even extreme cases distort(x) them

+example

independent variable ← confound → dependent variable

Confound: completely explains relationship

+examples

A strategy (blueprint) for investigating the research question/hypothesis in a coherent and logical way, including what kind of data is needed, how the data is collected, and what methods of analysis will be used

• Type of research

• Evidence or data needed (for test, not confirmation)

• Method of data collection

• Modes of analysis (logic to draw inferences)

• How threats to internal and external validity and measurement reliability and validity are minimized

• experimental

• Cross-sectional and longitudinal

• Comparative

• Historical (paired with comparative)

• Randomized intervention/treatment

  • Causal factor is randomly assigned (med experiments (placebo vs real))

  • No confounding variable

• Lab, field & survey experiments (3 types)

  • Social science - traditionally field experiments

• (Natural experiments) -> NOT an experiment 

  • Does not include a randomized treatment, does not treat/manipulte anything

• Observation

  • Cross-sectional - different cases

  • Longitudinal - over time

• Large n research (counties, conflicts, individuals, etc)

• Panel & Cohort Designs

  • Panel- select people/states and follow them over time

  • Cohort- use same survey (ex) but new people/sample taking it each time

• Single n case study - in-depth process tracing (find out step by step of what happened)

• Small n (2-4) and large n case study

• Requires the definition of clear boundaries:

  • Time | place/space | units/actors | factors/variables

• Data sources & quality: availability, reliability, validity

• Questionnaires/surveys

• Interviews/focus groups

• Participation observation/ ethnographic research (exception, does not happen often)

• Textual/content/discourse analysis (existing documents)

• Statistical data

Combination of different designs & methods can be useful

• study occurred in 60s

• Hypothesis: individuals will obey requests by authority even if requests is considered to be unethical

• Cover story: ‘learning by punishment’

• Participants in experiment:

  • experimenter (knew)

  • Teacher (volunteer)

  • Learner (knew)

• Experiment: volunteers (teachers) were brought in to ask questions to learner, every time they got a question wrong the teacher would give them an electric shock (15-450 volts)

  • The experiment was fake and was actually to see how long people would go doing immoral activity when pressured from authority (experimenter)

• 2/3 went to the max voltage

Holocaust - Nazi officers after the war was over often used the excuse ‘i was just following orders’

• verbal protest

• Knocking on wall

• Refusal/silence

• ‘Please continue’

• ‘The experiment requires you to continue’

• ‘You have no other choice, you must go on’

• Would have sever psychological damage-> after study completed they would get to know what was actually happening (full debriefing)

  • Followed up with them a year later -> claimed no problems had occurred

Personality traits of prisoners & guards are key to understanding abusive prison situations (situation vs personality)

• 2 week ‘prison simulation’ (planned)

• 24 participants (pre-screened)

• Random assignment as “prisoner” or “guard” (in uniforms)

• Quickly spun out of control

• Early termination after 6 days

• 1/3 of “guards” exhibit sadistic tendencies

  • Can be seen with peacekeepers today

• Conduct study on effect of canvassing before referendum of legalization of same sex marriage -> large difference in results

• Another group wanted to replicate-> did not work

• LaCour had taken real data and used it with fake data to show referendum differences

• By 2015, 58 (co-) authored papers retracted 

• Went out to collect data alone-> came back with data and published together with others (co-authors did not know the data was fake)

• Plagiarized master thesis or PhD dissertation

• Germany (2011): Karl-Theodor zu Gutenberg (defense Secretary)

• Romania (2012): Victor Ponta (Prime Minister)

• Germany (2013): Annette Schawan (Education Minister)

• Spain (2018): Carmen Montón (Health Minister)

• Norway (2024): Sandra Borch (Higher Education Minister)

All stages of research

• Data collection & storage: risks, privacy & fraud prevention

• Data analysis: transparency & replication

• Academic writing: plagiarism

1. Do no harm: benefits much outweigh risks

   1. Ex. Vaccines

2. Informed consent: voluntary participation

3. Protection of privacy/confidentiality

4. Transparency and documentation (good research practice)

• Topic and nature of questions (sensitive/intrusive)

• Purpose/goal (incl. disclosure of financial interests/sponsors)

• Use of information

• Participation requirements/expectoration

  • Voluntary participation

  • Freedom to stop

  • Permission to record (audio/video) & to quote responses

  • Permission to use data

• Risks involved

• Competition & comprehension (marginalized/vulnerable population)

• Incentives for participation?

• (Unobtrusive) observation

  • Consent after observation?

• Experimental manipulation

  • Concealment vs deception

    • Both of these cases need debriefing (explaining the truth)

• Public information (’on record’) (rules also apply)

• Confidential information (not always possible/in your control)

• Anonymous information (ideal, but rarely possible)

• In practice:

  • don’t make promises you can’t keep

  • When in doubt: make advance & explicit agreements

• Avoiding bias

• No incorrect reporting

• No inappropriate use of information

• No restrictions imposed by the sponsor

• No misuse of information

• Formal review of research projects:

  • USA: Institutional Review Board (IRB)

  • Leiden: Ethics Committee

• More formal rules for documentation and archiving:

  • Pre-registration

    • Publicly committing to your research

  • EU General Data Protection Regulation (GDPR)

  • Data Access & Research Transparency (DART)

  • Data Management plans (Leiden)

presenting, intentionally or otherwise, someone else’s words, thoughts, analysis, argumentation, pictures, techniques, computer programs, etc. as your own work

• ability to draw causal inferences

  • Confidence in (observed) causal relationship

  • Ability to reuse out alternative explanations (confounds)

generalizability of research findings