On bandwidth selection in empirical risk minimization

Michael Chichignoud (ETH Zürich)
Wednesday, May 28, 2014 - 10:00am
Mohrenstraße 39, Erhard-Schmidt-Hörsaal

The well-known Goldenshluger-Lepski method (GLM) allows to select multi-dimensional bandwidths (possibly anisotropic) of kernel estimators and provides optimal results in this setting. However, GLM requires some linearity property, which is not satisfied in empirical risk minimization (where a bandwidth is involved in the empirical risk). One typically deals with this issue in local M-estimation such as local median estimate or local maximum-likelihood estimate; and in statistical learning with noisy data such as in quantile and moment estimation, in discriminant analysis and in clustering. Many of these studies lead to data-driven procedures selecting isotropic bandwidths, however, none of them allows anisotropic bandwidth selection. We present a novel data-driven selection of anisotropic bandwidths in the large setting of empirical risk minimization. The selection consists of comparing gradient empirical risks (instead of comparing estimators). It can be viewed as a non-trivial improvement of GLM to non-linear estimators. This method allows us to derive excess risk bounds - with fast rates of convergence - in noisy clustering as well as adaptive minimax results for pointwise and global estimation in nonparametric regression.