Logistic Regression Isn't Regression: It's Classification Through a Probability Trick

The name is confusing. Logistic Regression sounds like it predicts continuous values the way Linear Regression does, but it's actually a classification algorithm. The "regression" part refers to the math underneath, not what it outputs. What it actually does is predict the probability that something belongs to a class, then threshold that probability into a decision.

The problem with using regular linear regression for classification: it can output any number, including negatives and values above 1. Probabilities have to live between 0 and 1. So Logistic Regression runs the linear output through a function that squashes everything into that range: the sigmoid function:

$$\sigma(z) = \frac{1}{1 + e^{-z}}$$

This produces an S-curve. For very large positive $z$, the output approaches 1. For very large negative $z$, it approaches 0. At $z = 0$, you get exactly 0.5. The model learns weights so that the linear combination of inputs produces a $z$-value that, after the sigmoid, gives the right probability.

The standard decision rule: if $\sigma(z) \geq 0.5$, predict class 1. If $\sigma(z) < 0.5$, predict class 0. You can adjust this threshold: lower it to catch more positives (higher recall), raise it to be more conservative (higher precision).