Logistic regression is a statistical method that estimates the probability that an observation belongs to a particular category. Despite its name, it is widely used for classification rather than predicting a continuous numerical value. It helps answer questions such as whether an email is spam, whether a customer is likely to cancel a subscription, or whether a medical test result indicates a higher probability of a particular condition.
The method works by combining information from one or more input variables, transforming that information into a probability, and using a decision threshold to assign a class when a categorical prediction is needed. Its value lies in this combination of mathematical simplicity, interpretability, and practical usefulness.
What logistic regression does
Many real-world problems involve deciding between categories. A bank may want to estimate whether a loan applicant will default. A software service may want to identify users at risk of leaving. A researcher may want to investigate which characteristics are associated with the presence or absence of a condition.
In each case, the available information consists of input variables, also called features. These might include income, account activity, age, transaction patterns, or measurements collected during an experiment. The outcome is a category, often represented as a binary variable with two possible values: 0 and 1.
Logistic regression uses the input features to estimate the probability of the outcome coded as 1. For example, a model might estimate that a particular transaction has a 0.85 probability of being fraudulent, given the patterns it has learned from historical data.
A probability is not the same as a final classification. If the system uses a threshold of 0.5, it would classify the transaction as fraudulent because its estimated probability exceeds that threshold. A different threshold could produce a different decision.
This distinction is central to understanding logistic regression. The model estimates a probability; a separate decision rule determines how that probability becomes a class label.
Why logistic regression uses a special mathematical function
A straightforward approach to classification might be to calculate a weighted sum of the input features. Each feature receives a coefficient that determines how strongly it contributes to the result. The model adds these weighted values together with an intercept, which accounts for the baseline level of the prediction.
For example, a simplified model predicting whether a customer will cancel a subscription might use the number of days since the customer last logged in, the frequency of recent support requests, and changes in usage. Each variable contributes according to its learned coefficient.
The resulting weighted sum, however, can take any real value, including negative numbers and numbers greater than 1. Such a value cannot directly serve as a probability, which must fall between 0 and 1.
Logistic regression solves this problem by passing the weighted sum through the logistic function, also called the sigmoid function. The function converts any real-valued input into a number strictly between 0 and 1.
The model can be expressed as:
p=11+e−zp = \frac{1}{1+e^{-z}}p=1+e−z1
where ppp is the predicted probability, eee is the mathematical constant used in exponential functions, and zzz is the weighted sum of the input features.
That weighted sum is calculated as:
z=β0+β1×1+β2×2+⋯+βnxnz = \beta_0+\beta_1x_1+\beta_2x_2+\cdots+\beta_nx_nz=β0+β1×1+β2×2+⋯+βnxn
Here, x1,x2,…,xnx_1, x_2, \ldots, x_nx1,x2,…,xn represent the input features. The coefficients β1,β2,…,βn\beta_1, \beta_2, \ldots, \beta_nβ1,β2,…,βn represent their learned contributions, and β0\beta_0β0 is the intercept.
When the weighted sum is strongly negative, the predicted probability approaches zero. When it is strongly positive, the probability approaches one. When the weighted sum equals zero, the probability is 0.5.
The logistic function therefore gives the model a smooth way to translate a combination of evidence into a probability. Small changes in the weighted sum can change the predicted probability, although the size of that change depends on where the current prediction lies on the curve.
How logistic regression learns from data
A logistic regression model does not begin with a reliable understanding of which features matter. It must learn its coefficients from examples for which the outcomes are already known.
Suppose a company has historical records indicating which customers canceled their subscriptions and which continued using the service. Each record includes information about customer behavior before the outcome occurred. Logistic regression uses these examples to estimate coefficients that make its predictions fit the observed outcomes.
This process is called model training. The model starts with an initial set of coefficients, calculates predicted probabilities, evaluates how well those probabilities correspond to the actual labels, and adjusts the coefficients to improve its performance.
The most common approach uses a mathematical objective called log loss, or binary cross-entropy. This objective penalizes predictions according to the probability assigned to the observed outcome. A model receives a larger penalty when it assigns a very low probability to an outcome that actually occurs.
For example, if a customer cancels, a predicted probability of cancellation of 0.9 is more consistent with that observation than a prediction of 0.1. Conversely, if the customer stays, a high predicted probability of cancellation receives a larger penalty than a low one.
Training seeks coefficients that minimize the total loss across the training examples, sometimes with additional penalties designed to discourage unnecessarily large coefficients. Numerical optimization methods, such as gradient-based algorithms, can perform the adjustments.
The goal is not simply to reproduce the training data. A useful model must also work on new observations it has not seen before. To assess that ability, data scientists typically evaluate the trained model on separate validation or test data. A model that performs well on its training examples but poorly on new data may have learned patterns that do not generalize.
How predicted probabilities become classifications
Once logistic regression estimates a probability, a classification rule can convert it into a categorical prediction.
In binary classification, a common rule uses a threshold of 0.5. Predictions at or above the threshold are assigned to the positive class, while predictions below it are assigned to the negative class. The positive class is simply the outcome designated as the event of interest; it does not necessarily mean something desirable.
Consider a model that estimates the probability that an email is spam. If it predicts a probability of 0.82, a threshold of 0.5 leads to a spam classification. If it predicts 0.18, the email is classified as not spam.
The threshold is a choice, not a fixed property of logistic regression. In some applications, a threshold of 0.5 is appropriate. In others, it can lead to costly mistakes.
A medical screening system, for example, may prioritize identifying as many people with a condition as possible. Lowering the threshold can increase the number of true cases detected, but it may also increase false alarms. A fraud detection system may instead use a threshold that balances missed fraudulent transactions against the inconvenience of flagging legitimate purchases.
The appropriate threshold depends on the consequences of different errors, the prevalence of the event, and the system’s operational requirements. It should be selected using relevant validation data and a clear understanding of the decision being made.
It is also important to distinguish a model’s probability estimate from certainty about an individual case. A prediction of 0.8 does not mean that an outcome is guaranteed, nor does it necessarily mean that the model is correct 80 percent of the time for every type of observation. Whether predicted probabilities correspond closely to observed frequencies is a property known as calibration, which must be assessed rather than assumed.
How to interpret logistic regression coefficients
One of the major advantages of logistic regression is that its coefficients can be interpreted in terms of how input features relate to the predicted outcome.
A positive coefficient means that, holding the other modeled features constant, increasing that feature increases the predicted log-odds of the outcome coded as 1. A negative coefficient means that increasing the feature decreases those log-odds. The intercept represents the log-odds when all numerical features equal zero and categorical features are at their reference levels.
The term odds describes the ratio of the probability that an event occurs to the probability that it does not. If an event has a probability of 0.8, its odds are 0.8 divided by 0.2, or 4 to 1. If its probability is 0.2, the odds are 1 to 4.
Logistic regression models the logarithm of these odds as a linear combination of the input features. This is why the model can use a linear equation internally while still producing probabilities between zero and one.
For a feature with coefficient β\betaβ, increasing that feature by one unit multiplies the modeled odds by eβe^\betaeβ, assuming the other features remain unchanged. This quantity is called an odds ratio.
If a coefficient is positive, its odds ratio exceeds 1. If it is negative, the odds ratio is below 1. A coefficient of zero corresponds to an odds ratio of 1, meaning that the feature has no modeled effect on the odds when the other features are held constant.
An odds ratio is not the same as a change in probability. The probability change associated with a feature depends on the starting probability as well as the coefficient. The same coefficient can produce different absolute probability changes for observations that begin with different predicted probabilities.
These interpretations also require care. A coefficient describes a relationship within the model, conditional on the other included features. It does not, by itself, establish that a feature causes the outcome. Confounding variables, biased sampling, measurement errors, and model misspecification can all affect the relationship between a feature and the observed outcome.
What logistic regression can and cannot represent
Despite its flexibility, standard logistic regression has an important structural limitation: its log-odds are a linear combination of the input features.
This does not mean that its predicted probabilities must change in a straight line. The logistic function makes the relationship between a feature and the probability nonlinear. However, unless the model includes additional terms, each numerical feature contributes linearly to the log-odds.
That structure works well when the relationship between the predictors and the outcome can be represented reasonably by this form. It can become inadequate when the underlying patterns are more complex.
For instance, the relationship between age and the probability of an outcome might rise and then fall. A basic logistic regression model with age as a single numerical feature cannot directly represent that curved pattern in its log-odds. The model may need additional features, such as age squared, or a more flexible representation using splines.
Interactions can also matter. An interaction occurs when the relationship between one feature and the outcome depends on another feature. The effect of a marketing offer, for example, might differ depending on how frequently a customer uses a service. Including an interaction term allows the model to represent that dependence.
Logistic regression also depends on the quality and representativeness of the training data. If a dataset systematically excludes certain groups, contains unreliable labels, or reflects historical biases, the resulting model can reproduce those problems. A mathematically well-fitted model is not necessarily a fair or appropriate decision-making system.
Another important issue is the relationship among input features. When two or more features contain strongly overlapping information, their individual coefficients can become unstable or difficult to interpret. The model may still make useful predictions, but attributing a distinct contribution to each correlated feature becomes more challenging.
Regularization, which penalizes large coefficient values during training, can help reduce overfitting and improve stability. Two common approaches are L1 regularization, which can drive some coefficients to zero, and L2 regularization, which discourages large coefficients without typically forcing them to zero. These methods can improve generalization, although they do not automatically solve problems involving biased data or a poorly specified model.
How logistic regression performance is evaluated
No single performance measure fully describes a classification model. Different metrics answer different questions, and the appropriate choice depends on the task.
A confusion matrix compares predicted classifications with actual outcomes. In binary classification, it counts true positives, true negatives, false positives, and false negatives. These categories make it easier to understand the types of errors a model produces.
Accuracy is the proportion of all predictions that are correct. It is intuitive but can be misleading when one class is much more common than the other. If only a small fraction of transactions are fraudulent, a model that labels every transaction legitimate could achieve high accuracy while failing to detect any fraud.
Precision measures the proportion of positive predictions that are actually positive. Recall, also called sensitivity in many contexts, measures the proportion of actual positive cases the model correctly identifies. Increasing one can come at the expense of the other, depending on the model and threshold.
The F1 score combines precision and recall using their harmonic mean. It can be useful when both matter, although it does not account for every operational cost and ignores true negatives in its formula.
For probability-based evaluation, log loss assesses the quality of predicted probabilities and penalizes confident errors. The area under the receiver operating characteristic curve, commonly called ROC AUC, measures how well the model ranks positive cases above negative cases across possible thresholds. Neither metric replaces the need to examine the errors and trade-offs that matter in a particular application.
Evaluation should use data that was not used to fit the model. If many model choices are made using the same test set, the resulting performance estimate can become overly optimistic. For reliable assessment, model development and final testing should be kept sufficiently separate.
Where logistic regression is useful
Logistic regression remains useful because many classification problems benefit from a model that is relatively efficient, interpretable, and capable of producing probabilities.
In health research, it can estimate the probability of an outcome based on patient characteristics or measured risk factors. In finance, it can support models that estimate the likelihood of default or suspicious activity. In business, it can help estimate customer retention or the probability that a prospective customer will complete a purchase. In scientific research, it can analyze relationships between explanatory variables and binary outcomes.
These applications differ in their goals. A researcher may be interested in estimating associations and understanding how variables relate to an outcome. A business may prioritize predictive accuracy. A high-stakes screening system may need well-calibrated probabilities and carefully chosen thresholds. The same modeling technique can serve these purposes, but its development and evaluation should reflect the intended use.
Logistic regression can also be extended to outcomes with more than two categories. Multinomial logistic regression models outcomes with several categories that do not have a natural order, while ordinal logistic regression is designed for ordered categories, such as low, medium, and high. These extensions use related statistical principles but require assumptions suited to the structure of the outcome.
How logistic regression compares with other classification methods
Logistic regression is not the only way to classify data. Decision trees, random forests, support vector machines, and neural networks can all be used for classification, but they represent relationships differently.
A decision tree divides observations into groups using a sequence of rules. It can capture certain nonlinear patterns and interactions without requiring them to be specified explicitly, although a large tree can be difficult to generalize or interpret. Random forests combine many decision trees to improve predictive performance and reduce some of the instability associated with individual trees.
Support vector machines seek a boundary that separates classes according to a margin-based objective. Depending on the version and kernel, they can represent linear or nonlinear decision boundaries. Neural networks learn layered transformations of input features and can model highly complex patterns, but often require more data, computational resources, and careful tuning.
Logistic regression is often a strong starting point when a problem has a binary outcome, the relationships are reasonably represented by its mathematical form, and interpretable coefficients or probability estimates are important. It is also useful as a baseline against which more complex methods can be compared.
More complex models are not automatically better. Their advantage depends on the data, the structure of the problem, the costs of different errors, and the evaluation method. A simpler model that performs reliably and can be understood by its users may be more useful than a more complicated model whose decisions are difficult to explain.
Ultimately, logistic regression classifies data by learning how input features relate to the log-odds of an outcome, converting those log-odds into probabilities, and applying a decision rule when a categorical prediction is needed. Its lasting importance comes from this clear statistical foundation: it provides a practical way to turn observed patterns into estimates of uncertainty while making many of the assumptions behind a prediction explicit.