A way to fit a model's parameters by choosing the values that make the observed data most probable: maximize the likelihood L(θ) = Π p(xᵢ | θ), in practice its logarithm. Least squares and logistic regression's log loss are both maximum likelihood under particular assumptions.
Card 1 of 244. Answer: A way to fit a model's parameters by choosing the values that make the observed data most probable: maximize the likelihood L(θ) = Π p(xᵢ | θ), in practice its logarithm. Least squares and logistic regression's log loss are both maximum likelihood under particular assumptions.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Maximum likelihood treats the data as fixed and the parameters as the unknowns. Write the probability of the observed data as a function of the parameters, take the log so the product becomes a sum, and find the maximum, analytically by setting the derivative to zero or numerically with gradient methods.
Example: a coin lands heads 7 times in 10 flips. The likelihood is p⁷(1 − p)³, its log is 7 ln p + 3 ln(1 − p), and the derivative 7/p − 3/(1 − p) is zero at p = 0.7, the sample proportion. After a single flip that lands heads, the estimate is p = 1, which shows how maximum likelihood overfits tiny samples.
Connections interviewers expect: minimizing squared error is maximum likelihood under Gaussian noise, and minimizing cross-entropy is maximum likelihood for a Bernoulli or categorical outcome. Under regularity conditions the estimator is consistent and asymptotically normal, but it can be biased: the MLE of a normal variance divides by n, not n − 1. Adding a prior gives the MAP estimate, which is where L2 and L1 regularization come from. Ronald Fisher developed the method between 1912 and 1922.
