Citation
The 2021 Lagrange Prize is awarded to Léon Bottou, Frank E. Curtis, and Jorge Nocedal for their paper, "Optimization Methods for Large-Scale Machine Learning", SIAM Review 60(2), 2018, which provides a foundational and insightful review of optimization methods for large-scale machine learning, including a new perspective for the simultaneous consideration of noise reduction and ill-conditioning and the foundations and analysis of second-order stochastic optimization methods for machine-learning.
Selection Committee
Sven Leyffer (Chair)
Xiaojun Chen
Etienne de Klerk
Philip Gill
Citation
The 2018 Lagrange Prize is awarded to Francis Bach, Nicolas Le Roux, and Mark Schmidt for their paper, "Minimizing finite sums with the stochastic average gradient" (NIPS, 2012; Mathematical Programming, 2017).
This paper is the first in a series of significant advances in the design and analysis of stochastic gradient methods applied to finite-sum problems. In particular, it establishes that in this setting, the proposed variant of a stochastic gradient method achieves a linear convergence rate. This novel approach has resulted in a surge of interest in variance reduction methods that achieve superior performance compared to other first-order methods. A wide range of applications benefit from this methodology, including linear least squares, principal component analysis, and L1-regularization problems. In summary, this work represents a paradigm shift in theoretical analysis of stochastic methods applied to finite-sum optimization problems.
Selection Committee
Katya Scheinberg (Chair)
Etienne de Klerk
Philip Gill
Andreas Griewank
Citation
The 2015 Lagrange Prize is awarded to Andrew R. Conn, Katya Scheinberg, and Luis Nunes Vicente for their paper "Introduction to Derivative-Free Optimization", MPS-SIAM Series on Optimization, SIAM, 2009.
This monograph represents a significant contribution in understanding the formulation of surrogate models, the construction of derivative free optimization (DFO) algorithms, and their convergence properties. It includes a groundbreaking trust region framework for convergence that has made DFO both principled and practical. A large number of key optimization studies have used these results substantively in both practical and conceptual ways. This work has not only influenced new DFO algorithms; its results have also enabled the solution of numerous optimization applications in science and engineering. A small sampling of the direct impact of their work is seen in aerospace engineering, urban transport systems, adaptive meshing for partial differential equations, and groundwater remediation.
Selection Committee
Mihai Anitescu (Chair)
Kurt M. Anstreicher
Lorenz Biegler
Werner Roemisch
Citation
The 2012 Lagrange Prize is awarded to Emmanuel J. Candès and Benjamin Recht for their paper, "Exact matrix completion via convex optimization", Foundations of Computational Mathematics 9 (2009), 717-772.
The paper of Candès and Recht was selected because of its exposition excellence, the current importance of the topic and the impressive number of citations in three years. It also opens Semidefinite Optimization to a fascinating new field of applications and introduces a very clever mathematical approach for proving probabilistic tractability of certain NP hard problems.
Selection Committee
Tamas Terlaky (Chair)
Kurt M. Anstreicher
Donald Goldfarb
Selection Committee
Adrian S. Lewis (Chair)
Jorge J. More
Philippe L. Toint
Margaret H. Wright
Thomas M. Liebling
Selection Committee
Michael J. Todd (Chair)
John E. Dennis Jr.
Nicholas I. Gould
Adrian S. Lewis
Selection Committee
C. Tim Kelley
Claude Lemarechal
Michael J. Todd
Stephen J. Wright