CS 465 Lecture 15 - Midterm Review

Midterm Review

Topics and Scope

• The exam covers material taught so far, but encoder–decoder networks and sequence-to-sequence models are excluded.
• For RNNs and LSTMs, the instructor expects a high-level understanding rather than derivations:
  • How RNNs extend ordinary feedforward neural networks.
  • How LSTMs improve on RNNs.
  • No derivation of backpropagation through time or detailed RNN weight updates is planned.

Exam Format and Logistics

• The exam is expected to take about one hour, with some extra time as a buffer.
• The instructor anticipates roughly 8–10 questions:
  • About 3–4 true/false or multiple-choice questions, which will also require explanations or justification.
  • About 3–4 short-answer questions.
  • Up to 2 computation questions.
• Students may bring two double-sided pages of notes, or four single-sided pages. The notes can contain any material.
• A calculator is not expected to be needed, but the instructor said they would confirm this in a Canvas message before the exam.
• Students should show their work. The instructor will award partial credit when the reasoning is visible.

Likely Computation Questions

Neural-Network Forward and Backward Calculations

A computation problem may ask students to:

1. Run a forward pass by calculating weighted sums and applying the network’s activation functions.
2. Calculate error/delta values at the output and hidden nodes.
3. Update weights using the learning rate, the input to a weight, and the relevant delta.

The instructor said the exam would provide the loss function and activation functions; students would perform the calculations. The network is likely to be relatively small—probably one hidden layer and one output node, though a slightly more complex network is possible.

In the review example:

• The instructor used ReLU activations and squared error.
• The output’s ReLU input was negative, making the output delta zero. That, in turn, made the hidden-layer deltas zero, so none of the weights changed. The example was mainly intended to demonstrate the procedure, not to produce interesting updates.
• For a hidden node connected to multiple outputs, its delta is found by summing the downstream deltas multiplied by their connecting weights, then applying the hidden node’s activation derivative.
• The instructor’s weight-update setup used the learning rate multiplied by the activation feeding into the weight and the delta at the receiving node.

N-Gram Frequency and Conditional Probability

A possible computation question may provide n-gram counts and ask for a conditional probability, such as the probability of a word given the preceding word or words.

• Divide the count of the relevant n-gram by the count of the context:
  • For a bigram, (P(\text{next word}\mid\text{current word})) is the bigram count divided by the total count of bigrams beginning with the current word.
  • The same idea extends to trigrams, using the preceding two words as context.
• The instructor may provide the counts directly rather than ask students to extract all n-grams from a passage.
• In the demonstration, every instance of “it” was followed by “was,” so the estimated probability of “was” given “it” was 100%. Another example used the different words following “of” to illustrate how their counts form a conditional distribution.

The instructor also said that answers to the second assignment would be posted on Canvas for review.

Speakers and Sources

• Instructor: The main speaker, reviewing exam scope, format, study expectations, and example computations. The subtitles do not identify the instructor by name.
• Student(s): Unnamed students ask questions about exam format, number of questions, notes, calculators, neural-network examples, and partial credit.
• Course materials mentioned: A review PDF and assignment answers available through Canvas. These are referenced as study materials, not as additional speakers.