Why the Brain Doesn’t Start From Scratch

Core idea

The brain uses compositionality: it builds reusable neural “modules” (like Lego bricks) for features and actions, then composes them to solve new tasks instead of relearning from scratch.

Flexibility arises from a two-step mechanism — fixed, orthogonal reusable modules for features/motor commands, plus a central belief/gating signal that adjusts gains and sets routing.

Experimental design & key result

Subjects: monkeys trained on three visual discrimination tasks that used the same stimuli (images morphed between a bunny and a T, and colors morphed between red and green).

Tasks

• S1: report shape along motor axis 1 (bunny → upper left; T → lower right).
• C2: report color along motor axis 2 (red → top right; green → bottom left).
• C1: report color but move along motor axis 1 (compositional task: color rule from C2 + motor axis from S1).

Neural recordings

• Hundreds of neurons recorded in dorsolateral prefrontal cortex (PFC) with implanted electrodes while monkeys performed the tasks.

Key result

• The experiment showed that neural representations for sensory features and motor outputs occupy reusable, task-specific subspaces; these subspaces are composed and routed to produce behavior in new, compositional tasks.

Analytical framework

• Represent population activity as a point in high-dimensional neural space (each neuron = an axis).
• Identify subspaces / hyperplanes that separate neural-activity clouds corresponding to different task variables (e.g., red vs green).
• Train classifiers (decoders) on one task’s neural data and test across tasks (cross-task decoding) to determine whether the same neural subspace is reused.

Main findings

• Task-specific information is represented in stable, separable population subspaces:

  • Color is decodable from PFC activity during color tasks (C1, C2) but not during the shape task (S1).
  • Shape is decodable during S1 but not during the color tasks.

• Cross-task decoding generalizes:

  • A classifier trained to decode color in C2 also decodes color in C1, and vice versa — the color subspace is physically reused across color tasks.
  • Motor-direction decoders generalize between S1 and C1 — motor subspace reuse.

• Neural dynamics show routing between subspaces:

  • Within a trial, activity first moves into the relevant sensory subspace (~100 ms after stimulus), then flows/rotates into the appropriate motor subspace a few hundred milliseconds later.
  • Routing is context-dependent: the same sensory signal (e.g., “red”) is routed to different motor outputs depending on task context (a railroad-switch analogy).

• Task-belief signal controls input selection:

  • During block transitions when the rule is uncertain, monkeys gradually infer the rule from feedback; a belief signal is decodable from neural activity.
  • This belief signal acts like gain control: it amplifies (stretches apart) the relevant sensory subspace and suppresses (squashes) irrelevant subspaces, ensuring only relevant information is routed.

• Overall conclusion: flexibility is achieved by composing fixed, orthogonal sensory and motor modules, with a top-down belief/gating signal that selects and amplifies the appropriate modules.

Methods / procedure

• Design tasks with explicit compositional structure (S1, C2, C1) to enable direct tests of module reuse.
• Record population neural activity from dorsolateral PFC during task performance.
• Embed activity in high-dimensional space (neurons as axes).
• Identify decision subspaces (hyperplanes) by training decoders for task variables.
• Test cross-task generalization of decoders to assess reuse of subspaces.
• Track trial-by-trial neural trajectories to visualize temporal routing between subspaces.
• Use task blocks with initial uncertainty to decode an evolving task-belief signal and assess its effect on subspace geometry (gain modulation).

Scientific concepts & phenomena presented

• Compositionality / modularity in neural coding.
• Neural population geometry and subspaces (hyperplanes separating clouds of activity).
• Cross-task generalization as evidence for physical reuse of neural modules.
• Dynamic routing: context-dependent transformation of sensory subspace activity into motor subspace activity (neural trajectories).
• Top-down belief/gain control that suppresses irrelevant inputs and amplifies relevant ones.
• Efficient reuse of existing circuitry explains rapid generalization to novel but compositional tasks.

Researchers / sources featured

• Tim Bushman’s group (Princeton) — paper published in Nature.
• Sponsor mentioned in the video: Recall / getriall.ai (not a research source).