Lesson 3, Topic 2

In Progress

← Previous Next→

Research Methods and Evaluation

Jukka 2.9.2025

Lesson Progress

0% Complete

(Lesson: Methodologies and Practical Tools — Intercultural Dialogue: Theory and Practice)

Overview and learning objectives

This topic provides practical guidance for designing, conducting and evaluating research on intercultural dialogue. It covers:

Qualitative and mixed‑methods research designs appropriate to intercultural settings.
Monitoring and evaluation (M&E) frameworks, including theory of change and logic models.
Indicators to measure social cohesion and attitudinal change.
Ethical and culturally responsive considerations specific to intercultural work.

By the end of this topic learners will be able to:

Select an appropriate research or evaluation design for an intercultural intervention.
Construct a theory of change and measurable indicators aligned with expected outcomes.
Apply qualitative methods and mixed‑methods strategies to capture attitudinal and behavioural change.
Implement ethically robust procedures that respect cultural protocols and data sovereignty.
Use UNESCO’s e‑Platform and IDIU network resources to locate instruments, case studies and best practice examples.

Context and use of UNESCO e‑Platform resources

The UNESCO e‑Platform on Intercultural Dialogue and the IDIU Network resources (including Interculturalism at the Crossroads / L’interculturalisme à la croisée des chemins) are primary repositories for:

Theoretical frameworks and scholarly literature.
Case studies and evaluation reports from diverse cultural contexts.
Published instruments and annotated bibliographies on intercultural measures.

Practical step: map your research questions to relevant entries on the e‑Platform (search by keywords, region, or methodology) and identify existing measurement instruments and case studies to adapt, rather than starting from scratch.

1. Framing the research or evaluation

Define the purpose:
- Exploratory research, formative evaluation, process monitoring, summative/outcome evaluation, or impact evaluation.
Articulate the problem and stakeholders:
- Who benefits? Who is affected? Which cultural groups, communities, institutions?
Develop a Theory of Change (ToC):
- Inputs → Activities → Outputs → Short/medium/long‑term outcomes → Impact.
- Explicitly state assumptions and contextual factors (e.g., past intergroup conflict, power imbalances).
Formulate clear research questions and indicators aligned to the ToC.

2. Qualitative methods: designs and applications

Qualitative methods are essential for understanding meanings, contexts and mechanisms of intercultural dialogue.

Common methods and uses:

Semi‑structured interviews: explore personal narratives, attitudes, experiences of dialogue processes.
Focus groups: examine group norms, collective narratives; useful for comparative perspectives across groups (facilitate with cultural sensitivity and balanced representation).
Participant observation / ethnography: capture interaction dynamics in situ (rituals, meetings, informal exchanges).
Discourse and narrative analysis: analyse language, framing, media and institutional texts shaping intercultural relations.
Participatory Action Research (PAR) and Photovoice: engage participants as co‑researchers to surface local priorities and empower communities.
Case study method: in‑depth analysis of an intervention or community to draw transferable lessons.

Quality and trustworthiness:

Credibility: prolonged engagement, member checking.
Transferability: thick description.
Dependability and confirmability: audit trails, reflexive notes.
Triangulation: data sources, researchers, methods.

Sampling:

Purposive and theoretical sampling to ensure maximum variation (age, gender, religious/ethnic background, spatial location).
Sample size is determined by saturation for qualitative inquiry (typically 15–30 interviews for focused studies; fewer or more depending on heterogeneity).

3. Mixed‑methods designs appropriate for intercultural research

Mixed‑methods combine the strengths of qualitative depth and quantitative generalisability. Common designs include:

Convergent (parallel) design:
- Collect qualitative and quantitative data concurrently; compare and merge results to corroborate findings (useful for capturing attitudes and observed behaviours simultaneously).
Explanatory sequential design:
- Collect quantitative data first (e.g., survey measuring attitudes); follow up qualitatively to explain results (e.g., why attitudes changed).
Exploratory sequential design:
- Start with qualitative inquiry to develop culturally valid instruments or hypotheses; then administer quantitative measures for generalisation.

Practical advice:

Use qualitative findings to adapt survey language and items for cultural relevance and translation accuracy.
Use mixed methods to distinguish between expressed attitudes and actual interactional behaviours (attitudinal vs behavioural indicators).

4. Monitoring and evaluation frameworks

Essential components:

Logic model or Theory of Change.
Clear definition of outputs, outcomes (short/medium/long), and impacts.
Performance indicators mapped to each level.
Data collection plan: methods, timing (baseline, midline, endline), responsibility, resources.
Attribution strategy: how will you assess the contribution of your intervention? (experimental designs, matched comparisons, contribution analysis, realist evaluation).
Reporting and utilisation plan: who will use results and how?

Types of evaluation:

Formative (design improvement).
Process (implementation fidelity, participation, barriers).
Summative/outcome (achievement of objectives).
Impact (causal effects, long‑term change).
Realist evaluation (what works, for whom, under what conditions).

Practical M&E tip: combine routine monitoring indicators (frequency, reach, participation) with outcome indicators (change in attitudes, social cohesion) and qualitative evidence of mechanism.

5. Indicators of social cohesion and attitudinal change

Principles for indicators:

Align to the ToC and be SMART (Specific, Measurable, Achievable, Relevant, Time‑bounded).
Use both quantitative (scales, behavioural counts) and qualitative (narratives, community perceptions) indicators.

Suggested indicator domains and examples:

A. Social cohesion (structural and relational)

Intergroup trust: proportion reporting trust in other community groups (survey item).
Cross‑group social ties: number/frequency of intergroup friendships, mixed‑group activities attended per month.
Civic participation: participation in civic or cultural events with cross‑cultural composition.
Institutional inclusion: representation of diverse groups in governance bodies (percentage).
Perceived safety and belonging: proportion feeling safe and belonging in shared public spaces.

B. Attitudinal change

Outgroup attitudes: validated scales (e.g., social distance measures, prejudice items).
Example survey item: “Would you be willing to have a close relative marry someone from [outgroup]?” (Likert scale)
Intercultural competence and empathy: self‑reported measures of perspective taking, cultural curiosity.
Perceived discrimination and respect: frequency of perceived discriminatory incidents.
Intent to engage: willingness to participate in future intercultural activities.

C. Behavioural indicators (to complement attitudes)

Actual participation counts: attendance logs disaggregated by group.
Observed cooperative behaviours in mixed‑group activities (e.g., proportion of time speaking in mixed groups).
Joint initiatives: number of cross‑community projects launched and sustained after 6–12 months.

Measurement guidance:

Use validated instruments where available; if adapting, conduct pilot testing and cognitive interviewing for cultural validity.
Disaggregate data by demographic variables (gender, age, ethnicity, religion, socioeconomic status) to detect differential impacts.

6. Data collection tools and analytic approaches

Quantitative tools:

Structured surveys (face‑to‑face, online, phone) with validated scales; ensure sampling strategy supports inference.
Administrative data and participation records.

Qualitative tools:

Interview guides, FGDs, observation protocols, visual methods.

Analysis:

Quantitative: descriptive statistics, difference‑in‑differences, regression models controlling for confounders, propensity score matching where randomisation is not feasible, multi‑level models for group/clustering effects.
Qualitative: thematic analysis, framework analysis aligned to ToC, narrative synthesis.
Mixed‑methods integration: joint displays, triangulation protocols, meta‑inferences that combine quantitative effect sizes with qualitative mechanisms.

Attribution and causality:

Use experimental or quasi‑experimental designs where ethical and feasible (randomised controlled trials, stepped‑wedge designs).
Where not feasible, employ contribution analysis, process tracing, or realist evaluation to infer causal pathways.

7. Ethical and culturally responsive considerations

Ethics are central in intercultural research. Key principles and actions:

Respect for cultural protocols:
- Seek guidance from community leaders and Elders; incorporate local protocols (greetings, acknowledgements).
- Acknowledge Traditional Custodians where relevant and follow Indigenous ethical frameworks (e.g., OCAP®, CARE principles).
Consent and power dynamics:
- Obtain informed consent in accessible language; consider group consent where culturally appropriate.
- Be sensitive to power asymmetries between researchers and participants; employ community co‑researchers where possible.
Data sovereignty and confidentiality:
- Clarify governance of data, storage, access and sharing; secure community agreement on secondary uses.
- Protect identities in small or marginalised communities; use aggregation or anonymisation appropriately.
Risks, benefits and reciprocity:
- Assess potential harms (social reprisals, re‑traumatisation); include safeguarding protocols.
- Ensure benefits to communities (capacity building, dissemination in local languages, tangible outcomes).
Translation and interpretation:
- Use accredited translators; back‑translate survey instruments; pilot test for semantic and conceptual equivalence.
Ethics approval and local review:
- Obtain institutional ethics clearance and, where required, community or Indigenous review processes.
Reporting and dissemination:
- Share findings with participating communities first; co‑author reports with community partners if appropriate.
- Use multiple dissemination formats (community briefs, presentations, policy briefs, academic publications).

8. Practical example: brief evaluation plan outline

Project: Community intercultural dialogue series aiming to increase cross‑group friendships and reduce prejudice.

ToC: Dialogue workshops → increased intergroup contact + trust → formation of cross‑group friendships → reduced prejudice and greater civic cooperation.
Outcomes and indicators:
- Short term: attendance and participant diversity (output).
  - Indicator: number of participants; % from each target group.
- Medium term: increased intergroup contact and trust.
  - Indicator: mean score change on Intergroup Contact Frequency Index; trust scale.
- Long term: reduced prejudice and increased joint initiatives.
  - Indicator: change in social distance scale; number of joint community projects after 12 months.
Design: Explanatory sequential mixed‑methods
- Quantitative baseline and endline survey (n=400; stratified sampling).
- Follow‑up qualitative interviews (n=30 purposively sampled) to explain survey changes.
M&E timeline:
- Baseline (month 0), process monitoring (monthly attendance), midline (month 6), endline (month 12), dissemination (month 14).
Ethics:
- Community approvals, participant consent forms in preferred languages, data access agreements.
Use UNESCO e‑Platform:
- Identify validated instruments and similar case studies; adapt measures using local pilot testing.

9. Practical checklist for researchers and evaluators

[ ] Align research questions to an explicit Theory of Change.
[ ] Select indicators that are SMART and culturally valid.
[ ] Choose a design (qualitative, quantitative, mixed) fit for purpose and context.
[ ] Pilot instruments with target communities and revise language.
[ ] Plan baseline and comparison data, and consider counterfactuals for attribution.
[ ] Establish M&E responsibilities, timelines and budgets.
[ ] Implement participatory methods and build local capacity.
[ ] Secure ethical approvals and community permissions; document data governance.
[ ] Use UNESCO e‑Platform and IDIU resources to ground design in existing evidence.
[ ] Plan dissemination to stakeholders and communities in accessible formats.

10. Further resources and readings (selected)

UNESCO e‑Platform on Intercultural Dialogue — curated case studies and bibliographies (use for instruments and examples).
Interculturalism at the Crossroads / L’interculturalisme à la croisée des chemins — thematic chapters on theory and practice.
UNITWIN/IDIU Network publications and case reports on participatory approaches and evaluation.
Key methodological references on mixed methods and culturally responsive evaluation (seek these via the e‑Platform bibliography).

If you would like, I can:

Provide a sample survey module (attitude and contact items) adapted to a specific context.
Draft a template Theory of Change and logic model for a particular project.
Curate a short reading list from the UNESCO e‑Platform tailored to a country or region.