CS5321 Numerical Optimization

Classes: M7 M8 R6 at EECS 129 (Fall 2008)

Textbooks: Numerical optimization, Jorge Nocedal and Stephen J. Wright (http://www.mcs.anl.gov/otc/Guide)

References:

1. Linear and Nonlinear Programming, Stephen G. Nash and Ariela Sofer (1996, 2005)
2. Numerical Methods for Unconstrained Optimization and Nonlinear Equations, J. Dennis and R. Schnabel

External links: Many useful notes/references can be found in the following links

• Class webpage by Dianne P. O'Leary
• Convex optimization, semidefinie programming by Anthony So.
• Linear programming by W.W.Lin
• SVM, Large-scale Nonlinear Optimization by C.J. Lin.
• Mathematical Optimization

Instructor: Cherung Lee

TA:

Prerequisites: Caculus, Linear Algebra, Numerical methods

Announcements

• The schedule of presentations has been posted. Please checkout here.
• The due date of classnote is postponed to 4/23
• Latex lecture here
• Please review the multivariable calculus and linear algebra.

Lecture:

1. Introduction ppt pdf
2. Unconstrained optimization ppt pdf
   • Line search methods ppt pdf examples
   • Trust region methods ppt pdf example updated
   • Conjugate gradient methods ppt pdf
   • Quasi-Newton methods ppt pdf example
   • Methods for large scaled problems ppt pdf example
   • Least square problems ppt pdf
3. Linear programming
   • The simplex method pptpdf example
   • The interior point method ppt pdf example
4. Constrained optimization ppt pdf example
   • Algorithm overview ppt pdf example
   • Quadratic programming ppt pdf example
   • Penalty and augmented Largrangian methods ppt pdf example
   • Active set methods pptpdf example
   • Interior point methods ppt pdf example
5. Matlab optimization toolbox introduction

Course requirements

• Attendance (10%)

• Class note (50%)
  
  • Write a class note (30%)
    You need to write a short tutorial paper on one topic covered in this class. It can be an algorithm, a problem, a concept, or a theorem. You should fully understand the topic that you choose and explain it to the potential readers, which are, in this case, your classmates. (details)

  • Review other's work (20%)
    You also need to review your classmates' note. Since one article will be reviewed by three persons, one student will review at least three notes. (details)

  • Important dates
    3/12: decide the topic
    4/16 4/23: submit the note
    5/14 5/21: submit the review
    6/11 6/18: submit revised note
    

• Project (40%)

  In this assignment, you need to find a related topic for independent study. The topic can be an optimization application, a problem formulation, software survey, or implementations. (details)

  • Proposal (10%): due on 3/26
    
  • Presentation (10%): schedule
    
  • Final report (20%): before the end of this semester (6/18)