Mathematical Formulation

Empirical Risk Minimization

Many supervised-learning workflows can be written as

\min_\theta
\frac{1}{n}\sum_{i=1}^{n}
\ell\left(y_i,f_\theta(x_i)\right)
+\lambda\Omega(\theta).

The regularization term controls complexity and mitigates overfitting.

Logistic Regression

For a binary classification problem,

P(y=1\mid x)=\sigma(\theta^T x)
=\frac{1}{1+e^{-\theta^T x}}.

The corresponding cross-entropy loss is

\ell(y,\hat y)=
-y\log(\hat y)-(1-y)\log(1-\hat y).

Optimization Methods

The laboratory sequence includes gradient descent and Newton’s method:

\theta_{k+1}=\theta_k-\alpha_k\nabla f(\theta_k)

\theta_{k+1}=\theta_k-
\left[\nabla^2 f(\theta_k)\right]^{-1}\nabla f(\theta_k).

Engineering Relevance

The repository demonstrates transferable skills for computational engineering: