Optimization for CS
Graduate · CS / Programming
Syllabus focus
Standard syllabus · STEM / applied
Pricing
Graduate-level rates are set on consultation. See the pricing page for K–12 and undergraduate rates.
Topics typically covered
Standard syllabus
Convex optimization
- Convex sets and functions; optimality conditions
- Linear programming and simplex overview
- Gradient descent and projected gradient
- Lagrangian duality and KKT conditions
- Second-order cone and semidefinite programs (intro)
Discrete and stochastic
- Integer programming and branch-and-bound (intro)
- Network flow as optimization
- Subgradient methods for non-smooth problems
- Stochastic gradient descent and mini-batching
- Online convex optimization (intro)
STEM / applied
ML and engineering applications
- Regularized regression as convex programs
- ADMM for distributed optimization (intro)
- Hyperparameter optimization and Bayesian opt (survey)
- Optimal control and model predictive control (intro)
- Solver tooling: CVXPY, Gurobi, MOSEK (survey)
Large-scale methods
- Coordinate descent and proximal algorithms
- Variance reduction methods (SVRG-style intro)
- Low-rank matrix completion applications
- GPU acceleration for optimization workloads
- Case studies from ML, robotics, and OR
Notes
Cross-listed with OR/EE at some schools. Linear algebra and multivariable calculus expected.