Metodi di ottimizzazione non vincolata, L. Grippo, M. Sciandrone, Springer-Verlag, 2011
Additional Lecture Notes
On line video streaming of each lecture
Learning Objectives
This course gives the student a theoretical background on non linear optimization. It has the objective of giving the student a sufficiently deep knowledge on continuous optimization theory, optimization algorithms and their main characteristics. Specific attention is devoted to optimization methods for Machine Lerning
CA2: Applying knowledge and understanding related to the analysis and optimization of systems, as well as to their innovation also through the development and improvement of design methods, constantly confronting with the rapid evolution of engineering.
CA3: Applying knowledge and understanding related to the choice and application of appropriate analytical and modelling methods, based on mathematical and numerical analysis, in order to better simulate the behavior of components and plants in order to predict and improve their performance.
CA6: Applying knowledge and understanding related to the identification, location and retrieval of data and information necessary for the assessment.
CA8: Applying knowledge and understanding related to the appropriate interpretation of the results of experimental tests, verification calculations and complex theoretical simulation processes, through the use of the computer, applying the acquired experimental, modeling, mathematical and informatics bases.
CA12: Applying adequate knowledge and understanding to understand English texts.
CC1: In-depth knowledge and understanding of the theoretical-scientific aspects of engineering, in which students are able to identify, formulate and solve, even in an innovative way, complex and/or interdisciplinary problems. The ability to understand a multidisciplinary context in the engineering field and to work with a problem solving approach
Prerequisites
Elementary knowledge of calculus (Taylor expansions, gradients, Hessian matrix)
Linear algebra
A course on operations Research / linear programming might prove useful
Teaching Methods
Front lectures. Lectures are video recorded and made availavable through Moodle
Further information
Type of Assessment
Written or oral exam on all the course subjects
The exam consists in checking, through theooretical questions:
- knowledge of the theory of optimization (optimality conditions)
- knowledge of non linear optimization algorithms and their theoretical properties
- knowledge of optimization methods applied to machine learning
Course program
Introduction; optimization models and examples
Basic definitions
Optimality conditions for constrained optimization (KKT conditions)
Introduction to machine learning
Convergence of algorithms
One-dimensional optimization
Gradient descent methods
Newton methods
Conjugate direction methods
Quasi-Newton methods
Trust Region methods
Constrained optimization methods.
Optimization Methods for Machine Learning