Homework #1: MLP Forward Pass, Convexity, and Monte Carlo Simulation

EE 541: Fall 2026

Assignment Details

Assigned: 24 August
Due: Sunday, 30 August at 23:59

Gradescope: Homework 1 | Setup | How to Submit

Requirements

Use only Python standard library modules (https://docs.python.org/3/library/) and matplotlib for this assignment, i.e., do not import numpy, scikit, or any other non-standard package.

Overview

Four analytic problems on the forward pass of a small MLP, partial derivatives, hyperplane geometry, and convexity, followed by two Monte Carlo simulations in pure Python.

Getting Started

Download the starter: hw1-starter.zip

unzip hw1-starter.zip
cd hw1-starter

It contains one directory per problem and no code.

Problem 1: MLP Forward Pass

An MLP has two input nodes, one hidden layer, and two outputs. The two sets of weights and biases are given by:

\[ W_1 = \begin{bmatrix} 1 & -2 \\ 3 & 4 \end{bmatrix} \quad b_1 = \begin{bmatrix} 1 \\ 0 \end{bmatrix} \]

\[ W_2 = \begin{bmatrix} 2 & 2 \\ 2 & -3 \end{bmatrix} \quad b_2 = \begin{bmatrix} 0 \\ -4 \end{bmatrix} \]

The non-linear activation for the hidden layer is ReLU (rectified linear unit) – that is \(h(x) = \max(x, 0)\). The output layer is linear (i.e., identity activation function). The output for layer \(l\) is given by \(a^{(l)} = h_l(W_l a^{(l-1)} + b_l)\).

What is the output activation for input \(x = \begin{bmatrix} +1 \\ -1 \end{bmatrix}\)?

Deliverables

See Submission. q1.pdf contains your calculations and the final output activation.

Problem 2: Partial Derivatives and Minimization

Let \(f(x, y) = 4x^2 + y^2 - xy - 13x\)

Find \(\frac{\partial f}{\partial x}\), the partial derivative of \(f\) with respect to \(x\). Find \(\frac{\partial f}{\partial y}\).
Find \((x, y) \in \mathbb{R}^2\) that minimizes \(f\).

Deliverables

See Submission. q2.pdf contains your worked solution, typeset or scanned. Show all work and box final answers.

Problem 3: Distance to a Hyperplane

A hyper-plane in \(\mathbb{R}^n\) is the set, \(\{ \mathbf{x} : \mathbf{x} \in \mathbb{R}^n, w^T x + b = 0 \}\), where \(w \in \mathbb{R}^n\) and \(b\) is a real scalar.

The solution of the following optimization problem describes the distance between a point \(x_0 \in \mathbb{R}^n\) and the hyperplane \(w^T x + b = 0\): \[ \min_x \|x_0 - x\|_2 \quad \text{s.t.} \quad w^T x + b = 0. \] Derive an analytic solution for the distance between \(x_0\) and \(w^T x + b = 0\).
What is the distance between two hyperplanes, \(w^T x + b_1 = 0\) and \(w^T x + b_2 = 0\)?

Deliverables

See Submission. q3.pdf contains your worked solution, typeset or scanned. Show all work and box final answers.

Problem 4: Convexity

A function \(f(x)\) is convex if \[ f(\lambda x + (1 - \lambda)y) \leq \lambda f(x) + (1 - \lambda)f(y) \] for all \(x, y\) and \(0 < \lambda < 1\).

Use this definition to prove that \(f(x) = x^2\) is a convex function. Verify that \(f(x) = x^3\) is not a convex function.
An \(n \times n\) matrix \(A\) is positive semi-definite if \(\mathbf{x}^T A \mathbf{x} \geq 0\) for all \(\mathbf{x} \in \mathbb{R}^n\). Prove that the function \(f(\mathbf{x}) = \mathbf{x}^T A \mathbf{x}\) is convex if \(A\) is positive semi-definite.

Deliverables

See Submission. q4.pdf contains your worked solution, typeset or scanned. Show all work and box final answers.

Problem 5: Biased Coin Simulation

Simulate tossing a biased coin (a Bernoulli trial) where \(P[\text{HEAD}] = 0.70\).

Count the number of heads in 50 trials. Record the longest run of heads.
Repeat the 50-flip experiment 20, 100, 200, and 1000 times. Use matplotlib to generate a histogram showing the observed number of heads for each case. Comment on the limit of the histogram.
Simulate tossing the coin 500 times. Generate a histogram showing the heads run lengths.

Deliverables

See Submission. coin_sim.py is your simulation. q5.pdf contains the histograms and your comments for each part.

Problem 6: Sum of Uniforms Exceeding a Threshold

Define the random variable \(N = \min \{ n: \sum_{i=1}^{n} X_i > 4\}\) as the smallest number of standard uniform random samples whose sum exceeds four. Generate a histogram using 100, 1000, and 10000 realizations of \(N\). Comment on the expected value \(E[N]\).

Deliverables

See Submission. uniform_sum.py is your simulation. q6.pdf contains the histograms and your comment on \(E[N]\).

Submission

README.md
.gitignore
q1/
└── q1.pdf
q2/
└── q2.pdf
q3/
└── q3.pdf
q4/
└── q4.pdf
q5/
├── coin_sim.py
└── q5.pdf
q6/
├── uniform_sum.py
└── q6.pdf