Disentanglement or disentangled representation learning in the context of statistical machine learning is the deriving or uncovering of independent generative factors given an input observation.

In this post, we will cover the fundamentals of unconstrained optimization of smooth functions, formulate and implement a few popular optimization algorithms in Python, and end with a comparative analysis and discussion.

The bias-variance tradeoff is critical for performance evaluation and model selection. It relates statistical quantities of bias and variance to two central modeling deficiencies: underfitting and overfitting.

A model fitted on a specific set of (training) data is expected to perform better on this data compared to another set of (test) data. This post will try to prove this statement mathematically.