In statistics, the knockoff filter, or simply knockoffs, is a framework for variable selection. It was originally introduced for linear regression by Rina Barber and Emmanuel Candès, and later generalized to other regression models in the random design setting. Knockoffs has found application in many practical areas, notably in genome-wide association studies.
Fixed-X knockoffs Consider a linear regression model with response vector y {\displaystyle \mathbf {y} } and feature matrix X {\displaystyle \mathbf {X} } , which is treated as deterministic. A matrix X ~ {\displaystyle {\tilde {\mathbf {X} }}} is said to be knockoffs of X {\displaystyle \mathbf {X} } if it does not depend on y {\displaystyle \mathbf {y} } and satisfies X i ⊤ X j = X i ⊤ X ~ j = X ~ i ⊤ X j = X ~ i ⊤ X ~ j {\displaystyle \mathbf {X} _{i}^{\top }\mathbf {X} _{j}=\mathbf {X} _{i}^{\top }{\tilde {\mathbf {X} }}_{j}={\tilde {\mathbf {X} }}_{i}^{\top }\mathbf {X} _{j}={\tilde {\mathbf {X} }}_{i}^{\top }{\tilde {\mathbf {X} }}_{j}} for i ≠ j {\displaystyle i\neq j} . Barber and Candès showed that, equipped with a suitable feature importance statistic, fixed-X knockoffs can be used for variable selection while controlling the false discovery rate (FDR).
Model-X knockoffs Consider a general regression model with response vector y {\displaystyle \mathbf {y} } and random feature matrix X {\displaystyle \mathbf {X} } . A matrix X ~ {\displaystyle {\tilde {\mathbf {X} }}} is said to be knockoffs of X {\displaystyle \mathbf {X} } if it is conditionally independent of y {\displaystyle \mathbf {y} } given X {\displaystyle \mathbf {X} } and satisfies a subtle pairwise exchangeable condition: for any j {\displaystyle j} , the joint distribution of the random matrix [ X , X ~ ] {\displaystyle [\mathbf {X} ,{\tilde {\mathbf {X} }}]} does not change if its j {\displaystyle j} th and ( j + p ) {\displaystyle (j+p)} th columns are swapped, where p {\displaystyle p} is the number of features. While it is less clear how to create model-X knockoffs compared to their fixed-X counterpart, various algorithms have been proposed to construct knockoffs. Once constructed, model-X knockoffs can be used for variable selection following the same procedure as fixed-X knockoffs and control the FDR.
… excerpt ends here. Continue reading the full article.
