Course: Numerical Methods of Optimization

« Back
Course title Numerical Methods of Optimization
Course code KMA/NMO
Organizational form of instruction Lecture + Exercise
Level of course Master
Year of study not specified
Semester Summer
Number of ECTS credits 3
Language of instruction Czech
Status of course Compulsory-optional, Optional
Form of instruction Face-to-face
Work placements This is not an internship
Recommended optional programme components None
  • Burkotová Jana, Mgr. Ph.D.
  • Andrášik Richard, Mgr.
  • Netuka Horymír, RNDr. Ph.D.
  • Machalová Jitka, RNDr. Ph.D.
Course content
1. Unconstrained optimization - subject, applications, examples. Introductory definitions and basic conceptions. 2. First-order necessary optimality conditions. Second-order necessary optimality conditions. Second-order sufficient optimality conditions. 3. Univariate minimization. Minimization without using derivatives (comparative method, Fibonacci search method, golden section search method). Methods using derivatives (bisection, Newton method). 4. Derivative-free minimization of functions of several variables. Nelder-Mead method. Hooke-Jeeves method. 5. Minimization of quadratic functions using gradient methods - part I. Quadratic function and its properties. Descent methods basics. Method of steepest descent for quadratic functions. 6. Minimization of quadratic functions using gradient methods - part II. Conjugate gradient method. Convergence analysis. 7. Line search methods - part I. Fundamental principles. Step length selection using backtracking line search. Armijo condition. Backtracking-Armijo line search algorithm. Convergence analysis. 8. Line search methods - part II. Wolfe conditions. Inexact line search using Wolfe conditions. Convergence analysis. 9. Line search methods - part III. Method of steepest descent for nonquadratic function. Conjugate gradient method for nonquadratic function and its two main versions. 10. Newton's method and its modifications. Classical Newton method. Modifications of Newton's method (damped Newton's method, finite-difference Newton's method). 11. Quasi-Newton methods. Principles of quasi-Newton methods. General scheme with B matrices. General scheme with G matrices. Broyden's method, DFP method, BFGS method. 12. Solution of systems of nonlinear equations. Multivariate minimization and its connection with nonlinear algebraic equations. Newton's method. Broyden's method.

Learning activities and teaching methods
Lecture, Monologic Lecture(Interpretation, Training), Demonstration
  • Attendace - 39 hours per semester
  • Homework for Teaching - 15 hours per semester
  • Preparation for the Exam - 40 hours per semester
Learning outcomes
Gain knowledge about theory and algorithms required to solve unconstrained optimization problems.
Knowledge Gain useful knowledge about theory and algorithms in order to study and solve unconstrained optimization problems.
Standard knowledge from mathematical analysis and linear algebra. Information concerning numerical methods are welcomed, but not necessary. Elemental experience with computation on PC.

Assessment methods and criteria
Oral exam, Seminar Work

Credit: the student has to compute given examples. Exam: the student has to understand the subject and be acquainted with theory and computational methods.
Recommended literature
  • J. E. Dennis, R. B. Schnabel. (1983). Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall Englewood Cliffs N. J.
  • J. Machalová, H. Netuka. (2013). Numerické metody nepodmíněné optimalizace. Olomouc.
  • J. Nocedal, S. J. Wright. (1999). Numerical Optimization. Springer.
  • L. Lukšan. (2011). Numerické optimalizační metody. Nepodmíněná minimalizace. Technical report no. 1152. Praha.
  • M.S. Bazaraa, H.D. Sherali, C.M. Shetty. (2006). Nonlinear Programming. Theory And Algorithms..
  • O. Došlý. (2005). Základy konvexní analýzy a optimalizace v R^n. Brno.
  • S. Míka. (1997). Matematická optimalizace. FAV ZČU, Plzeň.
  • Z. Dostál, P. Beremlijski. (2012). Metody optimalizace. Ostrava.

Study plans that include the course
Faculty Study plan (Version) Branch of study Category Recommended year of study Recommended semester
Faculty of Science Applied Statistics (2015) Mathematics courses 3 Summer