The homotopy method revisited: Computing solution paths of ℓ1-regularized problems

Bringmann B, Cremers D, Krahmer F, Moeller M (2018)

Publication Type: Journal article

Publication year: 2018

Journal

Mathematics of Computation American Mathematical Society

Book Volume: 87

Pages Range: 2343-2364

Journal Issue: 313

DOI: 10.1090/mcom/3287

Abstract

ℓ1-regularized linear inverse problems frequently arise in signal processing, image analysis, and statistics. The correct choice of the regularization parameter t ∈ R ≥0 is a delicate issue. Instead of solving the variational problem for a fixed parameter, the idea of the homotopy method is to compute a complete solution path u(t) as a function of t. In their celebrated paper A new approach to variable selection in least squares problems [IMA J. Numer. Anal. 20 (2000), no. 3, 389-403], Osborne, Presnell, and Turlach showed that the computational cost of this approach is often comparable to the cost of solving the corresponding least squares problem. Their analysis relies on the one-at-a-time condition, which requires that different indices enter or leave the support of the solution at distinct regularization parameters. In this paper, we introduce a generalized homotopy algorithm based on a nonnegative least squares problem, which does not require such a condition, and prove its termination after finitely many steps. At every point of the path, we give a full characterization of all possible directions. To illustrate our results, we discuss examples in which the standard homotopy method either fails or becomes infeasible. To the best of our knowledge, our algorithm is the first to provably compute a full piecewise linear and continuous solution path for an arbitrary combination of a measurement matrix and a data vector.

Involved external institutions

University of California Los Angeles (UCLA)

United States (USA) (US) Technische Universität München (TUM)

Germany (DE) Universität Siegen

Germany (DE)

How to cite

APA:

Bringmann, B., Cremers, D., Krahmer, F., & Moeller, M. (2018). The homotopy method revisited: Computing solution paths of ℓ1-regularized problems. Mathematics of Computation, 87(313), 2343-2364. https://doi.org/10.1090/mcom/3287

MLA:

Bringmann, Bjoern, et al. "The homotopy method revisited: Computing solution paths of ℓ1-regularized problems." Mathematics of Computation 87.313 (2018): 2343-2364.

BibTeX: Download