ECE490: Introduction to Optimization

Course Information

  • Office Hours: Tu/Th 3-4pm, 145 CSL

  • Lectures: Tu/Th 11-12:20, Room 2017 ECEB

  • Parisa's Office Hours: Tu 9-10am in 5034 ECEB, Th 9-10am in 4034 ECEB

  • For a complete syllabus, see here.

Course Description

Basic theory and methods for the solution of optimization problems; iterative techniques for unconstrained minimization including gradient descent method, Nesterov’s accelerated method, and Newton’s method; convergence rate analysis via dissipation inequalities; constrained optimization algorithms including penalty function methods, primal and dual methods, penalty and barrier method; Lagrangian multiplier theory; duality theory

Required Materials

There is no required textbook for the class. All course material will be presented in class and/or provided online as notes. The following resources may be used as references.

  • D. Bertsekas. Nonlinear Programming, Athena Scientific, 2016.

  • J. Nocedal, S. Wright. Numerical Optimization, Springer, 2006.

  • S. Bubeck. Convex Optimization: Algorithms and Complexity, 2015.

Grading

  • Homework: There are roughly biweekly homework assignments (about 7 total). Homework will be due on at the end of the lecture that is given a week later. In-class and afterclass discussions are strongly encouraged. However, copying of others’ homework is not allowed. Homework problems will include mathematical derivations as well as coding tasks. Late homework will not be accepted, unless there is a valid reason. Examples of valid reasons include illnesses, job interviews and travel to conferences to present research papers. In all such cases, you have to provide proof that you missed the homework deadline for a valid reason.

  • Two Midterm Exams: There will be two in-class midterm exams; in roughly the 6th and 12th week, respectively. In both midterms, you are allowed to use notes handwritten on one 8.5"x11" sheet of paper (you can write on the front and back of the paper).

  • Final Exam: There will be one final exam that will be comprehensive. You are allowed to use notes handwritten on three 8.5"x11" sheet of paper (you can write on the front and back of the paper).

  • Grades for the students in Section P3 (3 credits) will be weighted as follows: Homework (35%), two midterm exam (20% each), and Final Exam (25%).

  • Grades for the students in Section P4 (4 credits) will be weighted as follows: Homework (28%), two midterm exam (15% each), Final Exam (20%), and Final Project (22%).

  • Final Project : The students in Section P4 (4 credits) are required to work independently on an optimization project and submit a final report. One may use techniques developed in this course but are also encouraged to learn and apply new techniques. For example, one can use the dissipation inequality technique developed in the course to investigate the performance of an optimization algorithm that is not covered in the course. We will provide candidate algorithms for you to investigate. You can also explore other ideas you have. More details will be given later in the semester.

  • Bonus Points: You will earn bonus points if you solve bonus problems in the homework. When working on bonus problems, one has to be completely independent.