Course: Numerical Methods of Optimization

« Back
Course title Numerical Methods of Optimization
Course code KMA/NMOM
Organizational form of instruction Lecture + Exercise
Level of course Bachelor
Year of study not specified
Semester Summer
Number of ECTS credits 4
Language of instruction Czech, English
Status of course Compulsory, Compulsory-optional
Form of instruction Face-to-face
Work placements This is not an internship
Recommended optional programme components None
Course availability The course is available to visiting students
  • Burkotová Jana, Mgr. Ph.D.
  • Machalová Jitka, RNDr. Ph.D.
  • Andrášik Richard, Mgr.
  • Pastor Karel, doc. Mgr. Ph.D.
  • Netuka Horymír, RNDr. Ph.D.
Course content
1. Unconstrained optimization - subject, applications, examples. Introductory definitions and basic conceptions. 2. First-order necessary optimality conditions. Second-order necessary optimality conditions. Second-order sufficient optimality conditions. 3. Univariate minimization. Minimization without using derivatives (comparative method, Fibonacci search method, golden section search method). Methods using derivatives (bisection, Newton method). 4. Derivative-free minimization of functions of several variables. Nelder-Mead method. Hooke-Jeeves method. 5. Minimization of quadratic functions using gradient methods - part I. Quadratic function and its properties. Descent methods basics. Method of steepest descent for quadratic functions. 6. Minimization of quadratic functions using gradient methods - part II. Conjugate gradient method. Convergence analysis. 7. Line search methods - part I. Fundamental principles. Step length selection using backtracking line search. Armijo condition. Backtracking-Armijo line search algorithm. Convergence analysis. 8. Line search methods - part II. Wolfe conditions. Inexact line search using Wolfe conditions. Convergence analysis. 9. Line search methods - part III. Method of steepest descent for nonquadratic functon. Conjugate gradient method for nonquadratic functon and its two main versions. 10. Newton's method and its modifications. Classical Newton method. Modifications of Newton's method (damped Newton's method, finite-difference Newton's method). 11. Quasi-Newton methods. Principles of quasi-Newton methods. General scheme with B matrices. General scheme with G matrices. Broyden's method, DFP method, BFGS method. 12. Solution of systems of nonlinear equations. Multivariate minimization and its connection with nonlinear algebraic equations - Gauss-Newton method. Newton method. Broyden method.

Learning activities and teaching methods
Lecture, Monologic Lecture(Interpretation, Training), Demonstration
  • Attendace - 52 hours per semester
  • Homework for Teaching - 20 hours per semester
  • Preparation for the Exam - 50 hours per semester
Learning outcomes
Gain knowledge about theory and algorithms required to solve unconstrained optimization problems.
Knowledge Gain useful knowledge about theory and algorithms in order to study and solve unconstrained optimization problems.
Standard knowledge from mathematical analysis and linear algebra. Information concerning numerical methods are welcomed, but not necessary. Elemental experience with computation on PC.

Assessment methods and criteria
Oral exam, Seminar Work

Credit: the student has to compute assigned examples. Exam: the student has to understand the subject and be acquainted with theory and computational methods.
Recommended literature
  • J. E. Dennis, R. B. Schnabel. (1983). Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall Englewood Cliffs N. J.
  • J. Machalová, H. Netuka. (2013). Numerické metody nepodmíněné optimalizace. Olomouc.
  • J. Nocedal, S. J. Wright. (1999). Numerical Optimization. Springer.
  • L. Lukšan. (1990). Metody s proměnnou metrikou. Academia, Praha.
  • S. Míka. (1997). Matematická optimalizace. FAV ZČU, Plzeň.

Study plans that include the course
Faculty Study plan (Version) Branch of study Category Recommended year of study Recommended semester
Faculty of Science Applied Mathematics (2014) Mathematics courses - Summer
Faculty of Science Mathematics and Applications (1) Mathematics courses 3 Summer
Faculty of Science General Physics and Mathematical Physics (2014) Physics courses 1 Summer
Faculty of Science Applications of Mathematics in Economy (2015) Mathematics courses 1 Summer