Video summary

21.05.2024 р. | ЄФВВ | Методи дослідження у педагогіці та психології

Main summary

Key takeaways

Educational

Main ideas and concepts (study quality in pedagogy/psychology research)

  • The webinar continues the topic of planning research and ensuring its quality (part two).
  • It focuses on sampling and the quality of information obtained from samples:
    • Representativeness
    • Validity
    • Reliability
  • Central idea: since measuring everyone is usually impossible, researchers use a sample to infer conclusions about the general population.

Sampling methodology: key definitions and structure

Core terms

  • General population (population of the study / генеральна сукупність):
    • The entire set of objects/respondents that could be studied (all adolescents in the aggression example, all students of a university, etc.).
  • Sample (вибірка):
    • A subset of respondents randomly selected from the general population.
  • Respondent:
    • An individual in the sample whose qualities/characteristics are measured.
  • Sample size:
    • Denoted as n (typically needs at least two respondents mentioned).
  • Population homogeneity/heterogeneity:
    • Homogeneous population: characteristics are common to each element.
    • Heterogeneous population: characteristics are concentrated in different subgroups.

Why sampling is necessary (example logic)

  • Research hypotheses are tested on a limited subset because measuring all people is impossible.
  • Example:
    • Hypothesis: TV violence increases adolescent aggression.
    • Full measurement would require all adolescents (not feasible), so a subset is used.

Experimental design elements related to sampling

Groups in experiments

  • Experimental group:
    • Selected participants who receive/exposed to the independent variable.
  • Control group:
    • Participants under the same conditions, but without the independent variable; provides a baseline for comparison.

Dependent vs. independent samples (comparative structure)

  • Dependent (paired) samples:
    • One case in sample X corresponds to exactly one case in sample Y (and vice versa) (e.g., husband & wife).
    • Typically same volume.
  • Independent samples:
    • No pairing correspondence (e.g., men vs. women; psychologists vs. mathematicians).
    • Volumes may differ.

Requirements for sample quality (the webinar’s main “quality of information” framework)

1) Representativeness

  • Meaning:
    • The sample reflects the general population’s structure and characteristics.
  • How it is ensured:
    • By random selection so every population unit has an equal chance to be included.
  • Trade-offs / degrees:
    • Perfect reproduction is impossible; representativeness can vary.
    • If representativeness is low, researchers compensate by increasing sample size (larger n).
    • If sample size is small, representativeness must be higher.
  • Connection to heterogeneity:
    • Representativeness is related to whether the population is heterogeneous (mix of categories such as sex, age, professions, etc.).
  • Goal:
    • Patterns found in the sample can be generalized to the general population with sufficient confidence.

Methods to increase representativeness

  • Systematic selection (systematic sampling):
    • First study the general population structure.
    • Ensure inclusion of representatives from all selected categories.
    • Downside: may be uneconomical/large; can still miss categories if the population structure isn’t fully analyzed.
  • Randomized selection (random sampling):
    • Uses probability principles (random number generator / random tables).
    • Saves time/material and increases the likelihood of including most categories.
    • Example:
      • Randomly choose 3 classes from different schools, then randomly choose 10 students from each.

2) Reliability and validity (quality of information)

  • Reliability (надійність):
    • Ensures results correctly reflect the studied reality.
  • Validity (валідність):
    • Whether the measurement truly measures what it is intended to measure (confirmation/proof).
  • Note: conclusions can be harmed by study errors.

Types of errors described

  • Registration errors:
    • Random: mistakes from lack of knowledge/typos.
    • Systematic: consistent distortions (e.g., interviewer records incorrect data) and also unintentional systematic causes (device malfunctions, respondent fatigue).
  • Representativeness errors:
    • Occur when the sample’s composition does not reproduce the population.

Statistical control of errors

  • Mentions “arithmetic and logical” control as part of identifying errors.

Types of samples and sampling strategies (organized list)

A) By how “units” are selected (types named in statistics)

  • Random (actually random) sampling
    • Can be repeated or non-repeated.
    • Repeated (“with replacement”): elements can appear multiple times.
    • Non-repeated (“without replacement”): elements are not reselected.
    • Example:
      • Lottery winnings:
        • Put all numbers into an urn; draw with/without replacement as described.
  • Mechanical sampling
    • Order the population (alphabetically/by time/by space).
    • Select every 2nd / 3rd / 4th / 10th unit, etc.
    • Use cases:
      • product quality control,
      • selecting enterprises for research,
      • budget surveys.
  • Typical sampling
    • Divide the population into homogeneous groups (“typical groups/districts/zones”).
    • Randomly select units from each group, proportionally to the group’s share.
    • Goal: include representatives of all typical groups.
  • Serial sampling
    • Select clusters/nests (whole groups) randomly instead of individual units.
    • Then observe all units within each selected nest.

B) Methods to form representative samples (two main methods)

  • Simple random sample
    • Each element has an equal chance of being included.
  • Stratified random sample
    • Divide the population into strata (e.g., by age, social status).
    • Randomly sample within each stratum.
    • Combines stratification with randomness.

C) Probability sampling concepts (general)

  • In probability sampling:
    • The probability of inclusion of each element is known.
  • Key theoretical form emphasized:
    • Simple random sample (with known inclusion probability).
  • Practical note:
    • Theory often assumes sampling with replacement; for large populations and smaller n, differences vs. without replacement are minor.

D) Changed proportions (deliberate oversampling)

  • If a subgroup is important and must be represented more strongly than its population percentage:
    • Researchers deliberately increase its share in the sample.
  • Example:
    • If patients are rare in the population (disease vs. healthy), sample includes a higher proportion of patients than reality to study the disease thoroughly.

Determining sample size (guidelines mentioned)

  • Sample size depends primarily on research objectives, but recommendations were listed:
    • Developing a diagnostic method: 200–1000 participants (or up to 2500).
    • Comparing two samples: total at least 50 people.
    • Studying relationships between properties: at least 335 participants.
    • Greater variability of the studied property → larger sample size needed.
  • Trade-off:
    • Reducing variability by making the sample more homogeneous (e.g., by sex/gender/age) may reduce needed size, but may narrow practical applicability of conclusions.

Sample selection examples: when simple random sampling can fail

  • Requirement for simple random sampling:
    • Need a sampling frame (a list/table that includes the population elements at least).
  • Failure example:
    • Survey “young families” by randomly calling from a telephone directory.
    • Not representative because:
      • not all families have phones,
      • some aren’t home during calling hours,
      • some people can’t be reached by phone,
      • some numbers aren’t listed in the directory.

Multistage and cluster sampling (additional designs described)

Multilevel (multistage) sampling

  • Several steps with a hierarchical structure:
    • Randomly select top-level groups → then subgroups → then further units → until individuals are selected.
  • Example:
    • university students: faculties → departments → courses → student groups → students.

Cluster sampling (geographical multistage structure)

  • Uses geography as hierarchy:
    • randomly select cities/villages → districts → streets → houses → apartments → individuals.
  • Random selection is applied at each step.

Stratified vs. cluster vs. weighting (proportions)

  • Stratified sampling is presented as often better than ordinary random sampling.
  • Weighting procedure
    • Used when stratum proportions are known but direct stratified selection may not be done.
    • Example:
      • If men:women in population is 51:49 but the sample is 60 men / 40 women, apply a correction factor to each salary to compensate.
  • Caution:
    • Weighting can worsen results and increase errors (example described where error becomes larger when a woman has an unusually high salary).

Final synthesis of “quality of information” and how sampling affects it

  • Representativeness error:
    • A sample cannot literally match the population; the task is to estimate how much it deviates.
  • For many studies, instead of full representativeness, researchers may use:
    • a main group and a control group with analytical comparison,
    • when the goal is deeper analysis rather than public-opinion style estimation (e.g., entrepreneurs vs. non-entrepreneurs).
  • Analytical approach requirements:
    • Groups must differ in one main differentiated feature.
    • Other parameters should be as identical as possible (age, gender, education, type of activity).
  • Reliability in particular is connected to:
    • random error (sampling error) due to population heterogeneity,
    • the ability to calculate random error and consider it when generalizing.
  • Reliability criterion:
    • stability of results across repeated surveys under similar conditions.
    • Example:
      • repeat the survey with the same questionnaire, similar procedure and sample size, different people → similar results indicate higher reliability.

Speakers / sources featured

  • Oksana Albertivna — Doctor of Pedagogical Sciences, Associate Professor; Head of the Department of Primary Vocational Education (speaker delivering/leading the sampling and quality questions).
  • “Webinar hosts / colleagues” are referenced indirectly (“Dear colleagues, thank you for your attention”), but no additional named speakers are provided in the subtitles.

Original video