“A Simple Measure of Conditional Dependence”, 2019-10-27 (; backlinks):
We propose a coefficient of conditional dependence between two random variables Y and Z given a set of other variables X1,…,Xp, based on an iid sample.
The coefficient has a long list of desirable properties, the most important of which is that under absolutely no distributional assumptions, it converges to a limit in [0,1], where the limit is 0 if and only if Y and Z are conditionally independent given X1,…,Xp, and is 1 if and only if Y is equal to a measurable function of Z given X1,…,Xp. Moreover, it has a natural interpretation as a nonlinear generalization of the familiar partial R2 statistic for measuring conditional dependence by regression.
Using this statistic, we devise a new variable selection algorithm, called Feature Ordering by Conditional Independence (FOCI), which is model-free, has no tuning parameters, and is provably consistent under sparsity assumptions.
A number of applications to synthetic and real datasets are worked out.