In statistics and numerical analysis, isotonic regression or monotonic regression is the technique of fitting a free-form line to a sequence of observations such that the fitted line is non-decreasing (or non-increasing) everywhere, and lies as close to the observations as possible.
Applications Isotonic regression has applications in statistical inference. For example, one might use it to fit an isotonic curve to the means of some set of experimental results when an increase in those means according to some particular ordering is expected. A benefit of isotonic regression is that it is not constrained by any functional form, such as the linearity imposed by linear regression, as long as the function is monotonic increasing. Another application is nonmetric multidimensional scaling, where a low-dimensional embedding for data points is sought such that order of distances between points in the embedding matches order of dissimilarity between points. Isotonic regression is used iteratively to fit ideal distances to preserve relative dissimilarity order. Isotonic regression is also used in probabilistic classification to calibrate the predicted probabilities of supervised machine learning models. Isotonic regression for the simply ordered case with univariate x , y {\displaystyle x,y} has been applied to estimating continuous dose-response relationships in fields such as anesthesiology and toxicology. Narrowly speaking, isotonic regression only provides point estimates at observed values of x . {\displaystyle x.} Estimation of the complete dose-response curve without any additional assumptions is usually done via linear interpolation between the point estimates. Software for computing isotone (monotonic) regression has been developed for R, Stata, and Python.
Problem statement and algorithms Let ( x 1 , y 1 ) , … , ( x n , y n ) {\displaystyle (x_{1},y_{1}),\ldots ,(x_{n},y_{n})} be a given set of observations, where the y i ∈ R {\displaystyle y_{i}\in \mathbb {R} } and the x i {\displaystyle x_{i}} fall in some partially ordered set. For generality, each observation ( x i , y i ) {\displaystyle (x_{i},y_{i})} may be given a weight w i ≥ 0 {\displaystyle w_{i}\geq 0} , although commonly w i = 1 {\displaystyle w_{i}=1} for all i {\displaystyle i} . Isotonic regression in the L 2 {\displaystyle L_{2}} metric seeks a weighted least-squares fit y ^ i ≈ y i {\displaystyle {\hat {y}}_{i}\approx y_{i}} for all i {\displaystyle i} , subject to the constraint that y ^ i ≤ y ^ j {\displaystyle {\hat {y}}_{i}\leq {\hat {y}}_{j}} whenever x i ≤ x j {\displaystyle x_{i}\leq x_{j}} . This gives the following quadratic program (QP) in the variables y ^ 1 , … , y ^ n {\displaystyle {\hat {y}}_{1},\ldots ,{\hat {y}}_{n}} :
min ∑ i = 1 n w i ( y ^ i − y i ) 2 {\displaystyle \min \sum _{i=1}^{n}w_{i}({\hat {y}}_{i}-y_{i})^{2}} subject to y ^ i ≤ y ^ j for all ( i , j ) ∈ E {\displaystyle {\hat {y}}_{i}\leq {\hat {y}}_{j}{\text{ for all }}(i,j)\in E}
… excerpt ends here. Continue reading the full article.
