Lasso regression modifies the cost function of ordinary least squares by adding a penalty term equal to the absolute value of the magnitude of the coefficients. The optimization problem can be stated as:
Minimize: (1/2n) Σ(y_i - ŷ_i)^2 + λ Σ|β_j|
Here, `λ` is the tuning parameter that controls the strength of the penalty. When `λ` is zero, lasso regression produces the same coefficients as ordinary least squares. As `λ` increases, the coefficients of less important predictors shrink to zero, effectively performing feature selection.
