LECTURE 16

Main Ideas

• Contrastive representation learning trains a model to place similar examples near each other in an embedding space and dissimilar examples farther apart. What counts as “similar” or “dissimilar” depends on the task.
• Contrastive learning can be used in supervised or self-supervised settings. In supervised learning, labels help identify positives from the same class and negatives from different classes. In self-supervised learning, pairs must be generated without labels.

Creating Positive and Negative Pairs

• Image patches: Patches from the same image can be treated as likely positives, while patches from different images can be treated as likely negatives. This is a useful assumption, but it is not guaranteed.
• Geographic proximity: The first law of geography suggests that nearby locations tend to be more related than distant ones. Nearby areas can therefore provide positive pairs, with more distant areas serving as potential negatives.
• Temporal proximity: Images of the same location acquired close together in time can be treated as positives. Images separated by longer periods may be less related and used as negatives.
• Cross-sensor matching: A cited multi-sensor approach pairs corresponding optical and SAR patches as positives. Shuffling the SAR patches creates mismatched optical–SAR pairs that can serve as negatives.

SimCLR-Style Learning Process

1. Start with a batch of (N) images.
2. Independently sample two transformations from an augmentation family for each image to create two views.
3. Pass each view through a feature network (F) and a projection network (G), producing two feature vectors per image and (2N) vectors in total.
4. Compute similarities between the projected vectors, such as cosine similarity.
5. For each view, treat the other view of the same image as its positive. The other views in the batch act as negatives.
6. Use a contrastive loss to pull positive pairs together and push negative pairs apart. A temperature parameter scales the similarity values. Average the loss across positive pairs to obtain the batch loss.

Lessons About SimCLR

• Augmentation is central to the method. Each image’s two views must differ enough to make the task challenging while retaining the underlying content the model should recognize.
• Augmentations can operate in different domains:
  • Spatial: cropping and rotation.
  • Spectral or image-processing based: blur and Sobel filtering.
• Combining spatial and spectral transformations can create more challenging views than combining transformations from only one domain. The lecture notes that SimCLR requires stronger augmentation than is typical in supervised learning; augmentation is part of the learning method, not merely an optional addition.
• The original SimCLR work found that performance can improve as model size increases. The lecture identifies this scaling behavior as an important property to revisit later.

Speakers and Sources

• Speaker: The lecturer presenting the NPTEL IIT Delhi lecture.
• Methods and sources mentioned: SimCLR and its original paper; a cited contrastive multi-sensor self-supervised learning paper involving optical and SAR imagery; and the first law of geography. The subtitles do not provide the lecturer’s name or full citation details for the papers.