Video summary

None of the Above - Why Standardized Testing Fails: Bob Sternberg at TEDxOStateU

Main summary

Key takeaways

Educational

Main ideas / concepts

  • Standardized testing can create and reinforce failure through “self-fulfilling prophecy.”

    • Bob Sternberg describes being labeled as “stupid” after poor performance on a group IQ test.
    • Because of those expectations, he did “stupid work,” which confirmed the teachers’ beliefs.
    • A later teacher (Mrs. Alexa) changed his trajectory by seeing beyond test scores and expecting more—showing how test-based judgments can shape outcomes.

  • College entrance tests (SAT/ACT/GRE) function like broad IQ measures.

    • The talk argues these tests primarily measure memory and analytical ability.
    • Statistically, they correlate strongly with socioeconomic status, meaning they often reproduce inequality rather than create a true meritocracy.

  • The world has changed, so skill demands have expanded beyond what these tests measure.

    • In the early 1900s, test populations were more homogeneous (e.g., mostly white, male, middle/upper class).
    • Today’s students come from more diverse backgrounds, and modern life requires more than what old test designs capture.
    • Modern success requires a wider set of abilities:
      • Creativity and adaptability (handling rapidly changing environments)
      • Analytical skill (judging whether ideas are good)
      • Practical execution skills (making ideas work)
      • Persuasion/common-sense abilities (communicating value; avoiding “great at ideas but can’t implement”)
      • Ethical/wisdom-based skills (acting for the common good)

  • Why these outdated tests persist (six reasons):

    1. Inertia: “We’ve used them for 100 years,” so institutions resist changing.
    2. Pseudo-quantification: People trust exact-looking numbers as if they are meaningful predictions.
    3. Similarity: Decision-makers may favor applicants who resemble those who already succeeded on the tests.
    4. Who pays: Students pay (benefiting colleges financially), so incentives remain.
    5. Publication/ranking trap: Media ranking systems encourage schools to rely on these metrics.
    6. Superstition: A belief that the tests have “magical” validity, or that society “can’t do any better.”

Proposed alternative methodology / instructional approach (detailed)

  • Core question: Can admissions assessment be improved beyond standardized tests?

    • Answer presented: Yes.

  • Guiding principle: Assess the whole person, including a broader set of skills needed for real-world success.

  • Skill domains the new approach measures:

    • Creative skills
    • Analytical skills
    • Practical skills
    • Wisdom-based and ethical skills

  • “Rainbow Project” (research methodology)

    • Created tests for students covering creative, analytical, and practical skills.
    • Conducted a national-scale project across colleges (from community colleges to highly prestigious colleges).
    • Compared results against conventional standardized tests (e.g., SAT/ACT).
    • Reported outcomes:
      • Doubling prediction of first-year academic average versus standardized tests.
      • Cutting ethnic group differences by about half.
      • Demonstrating that creative/practical performance can be separated from what memory/analytical tests primarily measure.
    • Funding issue: The project was funded by a testing company that later cut funding after results were published.

  • “Kaleidoscope” (implementation methodology at Tufts)

    • Based on similar skill-measurement ideas from the “Rainbow” research.
    • Implemented for Tufts undergraduate admissions during Sternberg’s tenure (5 years).
    • Reported outcomes:
      • Improved prediction of academic performance.
      • Improved prediction of extracurricular and leadership performance.
      • Nearly eliminated ethnic group differences (as described in the talk).
      • Higher satisfaction for applicants and families because the system evaluates more than a single score.

  • “Panorama” (implementation at Oklahoma State)

    • Started July 1, 2012 at Oklahoma State University with collaborators including Kyle Ray.
    • Used for admissions based on:
      • SAT/ACT and grades, plus
      • creative, analytical, practical, and wisdom/ethical skills.

  • Important teaching/learning alignment (instructional lesson)

    • Sternberg warns that admitting students based on creative/practical/wisdom abilities fails if instruction doesn’t develop those abilities.
    • Analogy/lesson from his own experience:
      • He learned to change/choose courses when teaching style didn’t match how he could succeed.
      • He later credited his career success to learning in ways that matched the required skills.
    • Therefore, Oklahoma State includes support structures:
      • A Learning and Student Success Opportunity Center
      • An Institute for Teaching and Learning Excellence
    • Conclusion of this portion: education must be designed to teach and assess the broader set of skills that admissions will evaluate.

Conclusion / main lesson

  • Standardized tests used for admissions do something, but they are out of date.
  • A better admissions system is possible by measuring broader, job-relevant and ethical forms of capability, and by teaching students in ways that cultivate those same skills.

Speakers / sources featured

  • Bob Sternberg (speaker; mentions roles as professor at Yale, Dean at Tufts, and Provost at Oklahoma State)
  • Mrs. Alexa (Sternberg’s 4th-grade teacher)
  • Kyle Ray (collaborator mentioned in launching the Oklahoma State “Panorama” program)
  • American Psychological Association (referenced via Sternberg’s anecdote about becoming president)
  • TEDxOStateU (event context from the video title)

Original video