The Hosmer–Lemeshow test is a statistical test for goodness of fit and calibration for logistic regression models. It is used frequently in risk prediction models. The test assesses whether or not the observed event rates match expected event rates in subgroups of the model population. The Hosmer–Lemeshow test specifically identifies subgroups as the deciles of fitted risk values. Models for which expected and observed event rates in subgroups are similar are called well calibrated. The test was named after its developers, statisticians David Hosmer and Stanley Lemeshow, and it was popularized by their textbook on logistic regression.
Introduction
Motivation Logistic regression models provide an estimate of the probability of an outcome, usually designated as a "success". It is desirable that the estimated probability of success be close to the true probability. Consider the following example. A researcher wishes to know if caffeine improves performance on a memory test. Volunteers consume different amounts of caffeine from 0 to 500 mg, and their score on the memory test is recorded. The results are shown in the table below.
The table has the following columns.
group: identifier for the 11 treatment groups, each receiving a different dose caffeine: mg of caffeine for volunteers in a treatment group n.volunteers: number of volunteers in a treatment group A.grade: the number of volunteers who achieved an A grade in the memory test (success) proportion.A: the proportion of volunteers who achieved an A grade The researcher performs a logistic regression, where "success" is a grade of A in the memory test, and the explanatory (x) variable is dose of caffeine. The logistic regression indicates that caffeine dose is significantly associated with the probability of an A grade (p < 0.001). However, the plot of the probability of an A grade versus mg caffeine shows that the logistic model (red line) does not accurately predict the probability seen in the data (black circles).
The logistic model suggests that the highest proportion of A scores will occur in volunteers who consume zero mg caffeine, when in fact the highest proportion of A scores occurs in volunteer consuming in the range of 100 to 150 mg. The same information may be presented in another graph that is helpful when there are two or more explanatory (x) variables. This is a graph of observed proportion of successes in the data and the expected proportion as predicted by the logistic model. Ideally all the points fall on the diagonal red line.
The expected probability of success (a grade of A) is given by the equation for the logistic regression model:
p ( success ) = 1 1 + e − ( b 0 + b 1 x 1 ) {\displaystyle p({\text{success}})={\frac {1}{1+e^{-(b_{0}+b_{1}x_{1})}}}}
where b0 and b1 are specified by the logistic regression model:
b0 is the intercept b1 is the coefficient for x1 For the logistic model of P(success) vs dose of caffeine, both graphs show that, for many doses, the estimated probability is not close to the probability observed in the data. This occurs even though the regression gave a significant p-value for caffeine. It is possible to have a significant p-value, but still have poor predictions of the proportion of successes. The Hosmer–Lemeshow test is useful to determine if the poor predictions (lack of fit) are significant, indicating that there are problems with the model. There are many possible reasons that a model may give poor predictions. In this example, the plot of the logistic regression suggests that the probability of an A score does not change monotonically with caffeine dose, as assumed by the model. Instead, it increases (from 0 to 100 mg) and then decreases. The current model is P(success) vs caffeine, and appears to be an inadequate model. A better model might be P(success) vs caffeine + caffeine2. The addition of the quadratic term caffeine2 to the regression model would allow for the increasing and then decreasing relationship of grade to caffeine dose. The logistic model including the caffeine2 term indicates that the quadratic caffeine^2 term is significant (p = 0.003) while the linear caffeine term is not significant (p = 0.21). The graph below shows the observed proportion of successes in the data versus the expected proportion as predicted by the logistic model that includes the caffeine^\2 term.
The Hosmer–Lemeshow test can determine if the differences between observed and expected proportions are significant, indicating model lack of fit.
Pearson chi-squared goodness of fit test The Pearson chi-squared goodness of fit test provides a method to test if the observed and expected proportions differ significantly. This method is useful if there are many observations for each value of the x variable(s). For the caffeine example, the observed number of A grades and non-A grades are known. The expected number (from the logistic model) can be calculated using the equation from the logistic regression. These are shown in the table below.
… excerpt ends here. Continue reading the full article.
