Probability and Bayes Nets
Consider a joint probability distribution with four binary variables: A, B, C, and D. Suppose A is an influence on B, C, and D, but B, C, and D are all conditionally independent of each other given A. Here are a few useful facts:
1. [1 pt] What is P(b|a)?
○ 0.1
○ 0.12
○ 0.2
○ 0.48
○ 0.8
○ 0.9
2. [1 pt] Is B independent from C?
○ Yes
○ No
3. [4 pts] Recall that many Bavesian networks, given the appropriate CPTs, could all represent the same joint probability distribution. Select all of the following diagrams that could represent this joint probability distribution.
4. [3 pts] Calculate P(a | b,c, d). You may leave your answer exact, or round your answer to three decimal places.
Linear Regression
Setup. We are modeling the number of UEFA Champions League trophies (UCL trophies) a club has won using two features:
x1 = appearances, x2 = avg_Squad_val (in millions).
We use the standard linear regression hypothesis with a bias term:
Cost over a set of m examples:
Unless otherwise stated, take the initial parameters to be
(Units: θ2 matches the “millions” scale of r2.)
Q1. Compute the average cost J(θ(0)) only over Real Madrid and Arsenal (i.e., the first two rows).
Q2. Do one stochastic gradient descent update using only the Juventus point with learning rate α = 10-6, Use the SGD rule
Q3. From the same starting θ(0) and α = 10-6, do one batch gradient descent update using all three examples. The vectorized rule (from lecture) is:
Q4. For the initial θ(0) = [-3, 0.6, 0.01]T, compute (exclude the bias θo):
Data and Training
1. What is the main benefit of k-fold cross-validation compared to a single validation split?
(select one)
A. It uses less data overall
B. It requires less computation
C. It provides a more reliable estimate of model performance
D. It guarantees higher model accuracy
2. You’re working with features that roughly follow a normal distribution and have no major outliers. You plan to use logistic regression. Which preprocessing method would be optimal?
(select one)
A. Min–max scaling
B. Z-score standardization
C. Robust scaling
D. None of the above
3. You train a neural net on 1,000 examples. Training accuracy = 99%, Validation accuracy = 65%. Which of the following is the most likely cause? (select one)
A. Underfitting
B. Overfitting
C. Data leakage
D. Insufficient epochs
4. Which situation most likely indicates overfitting?
A. High training loss and high validation loss
B. Low training loss but high validation loss
C. High training loss but low validation loss
D. Both training and validation losses high and equal
5. You notice that your model’s performance across k-fold cross-validation is highly variable: some folds yield near-perfect accuracy, others barely above chance. Which of the following is the most plausible explanation?
A. The model’s learning rate is too small.
B. The folds are not representative of the data distribution (violating the i.i.d. assumption).
C. Cross-validation always produces high variance by design; this is expected.
D. The number of folds k is too large.
6. After z-score scaling all features, your linear regression model performs worse.
Which subtle mistake could explain this?
A. Some features were nominal, so scaling imposed meaningless distances.
B. The model was under-regularized, so scaling increased variance.
C. Scaling always reduces model interpretability.
D. Z-scoring hides collinearity among features.
Logistic Regression
1. In binary logistic regression, the sigmoid function σ(z) = 1/(1+e^(-z)) is used to transform. the logit into a probability. What is the primary reason we don't use this transformation in linear regression?
A. The sigmoid is computationally expensive
B. Linear regression predicts continuous values that can fall outside [0,1]
C. The sigmoid makes the optimization problem non-convex
D. Linear regression doesn't use gradient descent
2. In softmax regression, why do we subtract the maximum logit value before computing exponentials (the numerical stability trick)?
A. To speed up computation
B. To prevent exponential overflow while maintaining the same probability distribution
C. To ensure all probabilities are exactly equal
D. To make the gradients larger for faster convergence
3. You're using mini-batch gradient descent with batch size b=32 for binary logistic regression on 1,000 examples with 10 features. What is the shape of the gradient ∇_θJ(θ) for one mini-batch update?
A. (32, 1)
B. (11, 1)
C. (11, 32)
D. (1, 32)
4. Your binary logistic regression model predicts ŷ=0.51 for an example with true label y=1, and ŷ=0.49 for another example with true label y=0. Which statement is true?
A. Both examples contribute equally to the loss
B. The first example has lower loss than the second
C. The second example has lower loss than the first
D. Both examples have zero loss since they're correctly classified
5. You train two logistic regression models on the same dataset: Model A uses Mean Squared Error (MSE) as the loss function, Model B uses binary cross-entropy. After training, which outcome is most likely?
A. Model A converges faster due to simpler gradients
B. Model B achieves better performance because cross-entropy is convex for logistic regression
C. Both models achieve identical performance
D. Model A performs better because MSE penalizes large errors more heavily
6. In softmax regression with K=3 classes, you observe the following predicted probability distribution for one example: ŷ = [0.6, 0.3, 0.1]. If the true class is c=2 (second class), what is the approximate cross-entropy loss for this example?
A. 0.3
B. 1.2
C. 0.7
D. 2.3