LECTURE 24

Summary

This lecture surveys deep-learning approaches to several SAR image analysis challenges: speckle reduction, combining SAR with optical imagery, and accounting for SAR-specific acquisition properties such as viewing angle and polarization.

1. Speckle Noise and Despeckling

• Speckle gives SAR images a grainy appearance. Unlike additive noise, it is multiplicative, so it often needs to be addressed before later image-analysis steps.
• Despeckling can use traditional filters or deep-learning methods. Deep-learning approaches usually lack clean, supervised training pairs, so researchers often create synthetic data by applying a speckle model—described in the lecture as multiplicative noise with a gamma distribution—to clean images.
• Despeckling networks are often shallower than networks used for tasks such as semantic segmentation. Their loss functions may also be tailored to the task.

Example: IDCNN

The Image Despeckling Convolutional Neural Network (IDCNN) described in the lecture:

• Trains on synthetic speckled examples, pairing a noisy image with its clean counterpart.
• Uses eight convolutional layers as its noise-estimation backbone.
• Uses convolution, batch-normalization, and ReLU blocks in intermediate layers. The first and final layers use convolution and ReLU without batch normalization.
• Uses zero padding to preserve the input’s spatial dimensions and avoids downsampling, since the output should match the input’s size.
• Produces a single-channel speckle estimate.
• Combines:
  • Euclidean loss to encourage pixel-level accuracy.
  • Total variation loss to encourage spatial smoothness by discouraging frequent, unnecessary changes between neighboring pixels.

The lecture also mentions a deeper, 17-layer CNN approach. Deeper models may use regularization such as batch normalization to stabilize training and skip connections to pass information between layers.

Practical Considerations

• Despeckling needs may vary across land-cover types: urban, agricultural, and forested areas may not benefit from identical filtering. Some areas may even be better left unfiltered.
• Multi-temporal SAR offers multiple noisy observations of the same scene. If treated as independent realizations under the same conditional distribution, these observations can provide a form of pseudo-supervision.
• Deep-learning despeckling is not always necessary: some workflows use traditional filters, while others omit a separate despeckling step and design the model around the downstream task.

2. Using Optical Imagery with SAR

The lecture discusses several ways optical data can assist SAR analysis.

SAR-to-Optical Translation with CycleGAN

• A CycleGAN can learn translation between SAR and optical image domains using two generators and two discriminators, along with cycle-consistency objectives.
• After training, a SAR image can be translated into an optical-like image and passed to a downstream model trained on optical imagery.
• CycleGAN’s advantage is that it does not require paired SAR and optical images; samples from corresponding distributions or geographic areas can be used.
• Translation artifacts and imperfect conversion can reduce performance. Pix2Pix is mentioned as another translation option, though it is distinct from CycleGAN in its data requirements.
• The lecturer notes diffusion models as another possible direction, to be covered later in the course.

Knowledge Distillation

• In general, a teacher produces outputs such as logits, and a student is trained to imitate them through a distillation loss while also learning the downstream task.
• Distillation can be useful when annotations are limited.
• For cross-sensor learning, a trained source model can guide a model for a target sensor. For example, a frozen optical-image encoder can guide a trainable SAR encoder:
  • Align or match their feature distributions.
  • Train the SAR model on its downstream task, such as classification.
• The same idea can apply across SAR types or sensors, and it can be extended to multiple teachers rather than just one.

Predicting Optical-Derived Quantities from SAR

• Another approach is to train a model using a target quantity normally derived from optical imagery.
• The example given is NDVI, normally calculated from multispectral bands such as near-infrared and red.
• A model can take SAR as input and predict NDVI, using a segmentation-style architecture such as U-Net and a regression loss.
• The task is intended to encourage the model to learn meaningful representations from SAR data.

3. Accounting for SAR-Specific Properties

Viewing Angle

• SAR features can vary with the sensor’s viewing angle.
• An angle-aware classifier can add a feature-transformation module after the encoder and before the classifier.
• The module uses angle or angle-difference information to adjust the encoded features before classification. The general idea may also apply to other downstream tasks.

Polarization and Missing Inputs

• A model may use separate encoders for different polarization channels, such as VV and VH, combine their features, and pass them to a decoder.
• At deployment, one polarization may be unavailable even if both were present during training. This is a missing-modality problem.
• One practical strategy is to vary the available inputs during training: sometimes provide VV, sometimes VH, and sometimes both. This helps the model learn to operate when only partial polarization information is available.

Speakers and Sources Featured

• Speaker: An unnamed course lecturer; no guest speakers are identified in the subtitles.
• Course/source: AI for Earth Observation, Lecture 24, from NPTEL IIT Delhi (channel information provided in the metadata).
• Methods and concepts referenced: IDCNN; a 17-layer CNN despeckling approach; CycleGAN; Pix2Pix; knowledge distillation; U-Net; NDVI; multi-temporal SAR; angle-aware classification; VV/VH polarization. Specific papers or authors are not named in the subtitles.