Stochastic systems courses

From Murray Wiki
Jump to navigationJump to search

This page collects some information about stochastic systems courses offered at Caltech. This page was prepared in preparation for a faculty discussion on the current stochastic systems sequence (ACM/EE 116, ACM 216, ACM 217/EE 164).

2018 course offerings

Primary courses in probability and stochastic processes

ACM/EE 116 Introduction to Probability Models. 9 units (3-1-5); first term. Prerequisites: Ma 2, Ma 3.This course introduces students to the fundamental concepts, methods, and models of applied probability and stochastic processes. The course is application oriented and focuses on the development of probabilistic thinking and intuitive feel of the subject rather than on a more traditional formal approach based on measure theory. The main goal is to equip science and engineering students with necessary probabilistic tools they can use in future studies and research. Topics covered include sample spaces, events, probabilities of events, discrete and continuous random variables, expectation, variance, correlation, joint and marginal distributions, independence, moment generating functions, law of large numbers, central limit theorem, random vectors and matrices, random graphs, Gaussian vectors, branching, Poisson, and counting processes, general discrete- and continuous-timed processes, auto- and cross-correlation functions, stationary processes, power spectral densities.

CMS/ACM/EE 117. Probability and Random Processes. 12 units (3-0-9); first term. Prerequisites: ACM 104 and ACM/EE 116. The course will start with a quick reminder on probability spaces, discrete and continuous random variables. It will cover the following core topics: branching processes, Poisson processes, limit theorems, Gaussian variables, vectors, spaces, processes and measures, the Brownian motion, Gaussian learning, game theory and decision theory (finite state space), martingales (concentration, convergence, Doob’s inequalities, optional/optimal stopping, Snell’s envelope), large deviations (introduction, if time permits).

Ma/ACM 144 ab. Probability. 9 units (3-0-6); first, second terms. Prerequisites: For 144 a, Ma 108 b is strongly recommended. Overview of measure theory. Random walks and the Strong law of large numbers via the theory of martingales and Markov chains. Characteristic functions and the central limit theorem. Poisson process and Brownian motion. Topics in statistics. Instructors: Tamuz, Ivrii.

ACM 159. Inverse Problems and Data Assimilation. 9 units (3-0-6); first term. Prerequisites: Basic differential equations, linear algebra, probability and statistics: ACM 104, ACM/EE 106 ab, ACM/EE 116, ACM/CS 157 or equivalent. Models in applied mathematics often have input parameters that are uncertain; observed data can be used to learn about these parameters and thereby to improve predictive capability. The purpose of the course is to describe the mathematical and algorithmic principles of this area. The topic lies at the intersection of fields including inverse problems, differential equations, machine learning and uncertainty quantification. Applications will be drawn from the physical, biological and data sciences.

ACM 216. Markov Chains, Discrete Stochastic Processes and Applications. 9 units (3-0-6); second term. Prerequisite: ACM/EE 116 or equivalent. Stable laws, Markov chains, classification of states, ergodicity, von Neumann ergodic theorem, mixing rate, stationary/equilibrium distributions and convergence of Markov chains, Markov chain Monte Carlo and its applications to scientific computing, Metropolis Hastings algorithm, coupling from the past, martingale theory and discrete time martingales, rare events, law of large deviations, Chernoff bounds.

ACM/EE 217. Advanced Topics in Stochastic Analysis. 9 units (3-0-6); third term. Prerequisite: ACM 216 or equivalent. The topic of this course changes from year to year and is expected to cover areas such as stochastic differential equations, stochastic control, statistical estimation and adaptive filtering, empirical processes and large deviation techniques, concentration inequalities and their applications. Examples of selected topics for stochastic differential equations include continuous time Brownian motion, Ito’s calculus, Girsanov theorem, stopping times, and applications of these ideas to mathematical finance and stochastic control.

Additional courses with some content

CDS 112. Control System Design. 9 units (3-2-4); second term. Prerequisites: CDS 110. Optimization-based design of control systems, including optimal control and receding horizon control. Robustness and uncertainty management in feedback systems through stochastic and deterministic methods. Introductory random processes, Kalman filtering, and norms of signals and systems.

ACM 257. Special Topics in Financial Mathematics. 9 units (3-0-6); third term. Prerequisite: ACM 95/100 or instructor’s permission. A basic knowledge of probability and statistics as well as transform methods for solving PDEs is assumed. This course develops some of the techniques of stochastic calculus and applies them to the theory of financial asset modeling. The mathematical concepts/tools developed will include introductions to random walks, Brownian motion, quadratic variation, and Ito-calculus. Connections to PDEs will be made by Feynman-Kac theorems. Concepts of risk-neutral pricing and martingale representation are introduced in the pricing of options. Topics covered will be selected from standard options, exotic options, American derivative securities, term-structure models, and jump processes.

SS/Ma 214. Mathematical Finance'. 9 units (3-0-6); second term. Prerequisites: Good knowledge of probability theory and differential equations. Some familiarity with analysis and measure theory is helpful. A course on pricing financial derivatives, risk management, and optimal portfolio selection using mathematical models. Students will be introduced to methods of Stochastic, Ito Calculus for models driven by Brownian motion. Models with jumps will also be discussed. Instructor: Cvitanic.

2009 Notes

Introduction

Background

The current sequence of courses (ACM/EE 116, ACM 216, ACM 217/EE 164) were first offered in the 2005-06 academic year following discussions between Emmanuel Candes, Babak Hassibi, Jerry Marsden, Richard Murray and Houman Ohwadi about how to integrate some of the course offerings in ACM and EE, with an eye toward applications in CDS. As a consequence of these changes, EE 162 (Random Processes for Communication and Signal Processing) was eliminated and replaced by ACM/EE 116.

There are three drivers for evaluating the course sequence at this time:

  • It's been a while since we set this up and it would be good to get together and see what we think about how its been going.
  • CDS is about to require ACM/EE 116 as part of its PhD requirements (in place of CDS 140b) and this may increase the enrollment in both ACM/EE 116 and the followon courses => it would be nice to think through any implications that has.
  • As part of the slow ramp up to creating a new PhD program in "systems mathematics and engineering" (or whatever we call the XYZ program), we are planning on a "stochastic systems" sequence that would probably be based on this set of courses.

In addition, in looking through Caltech's current course offerings, it appears that there are several courses in statistics and stochastic systems that might benefit from better integration. These include:

  • Courses on statistical modeling and analysis:
    • ACM/ESE 118 - Methods in Applied Statistics and Data Analysis
    • Ec 122 - Econometrics
    • Ma 112 - Statistics
    • SS 222 - Econometrics
    • SS 224 - Applied Data Analysis for the Social Sciences
  • Additional advanced courses (ala ACM 217/EE 164):
    • ACM 257 - Special Topics in Financial Mathematics
    • Ma 193 a - Random Matrix Theory (special topics course)
    • SS/Ma 214 - Mathematical Finance
    • CS 101-2 - Active Learning and Optimized Information Gathering
    • CS/CNS/EE 156 ab - Learning Systems
    • CS 286 - Performance Modeling (of computing systems)
  • Other courses that may have partial overlap with the main topics in ACM 116/216/217:
    • Ae 115ab - Spacecraft navigation: includes statistical orbit determination problem (batch and sequential Kalman filter implementations)
    • CDS 110b - Control systems: includes noise as disturbances as random processes, Kalman filtering and sensor fusion
    • CDS270 - Stochastic System Analysis and Bayesian Updating (special topics course): includes Bayesian modeling, Markov Chain stochastic simulation, Bayesian sequential estimation, Kalman filtering
    • CS 101-2 - Active Learning and Optimized Information Gathering: includes Markov decision processes, Bayesian search
    • CS 286 - Performance Modeling (of computing systems): includes probability theory, Markov chains, Queueing theory, heavy-tailed distributions
    • EE 163 ab - Communication Theory: has ACM/EE 116 as a pre-requisite; includes signals and noise as random processes

Agenda

  1. Introductions
  2. Goals, agenda and background (Richard)
  3. Review of current courses (Houman, Emmanuel, Babak)
  4. Discussion: how well is the current course sequence working
  5. Discussion: other possible linkages, additional material, etc?
  6. Action items to be taken (if any)

Overview of current course sequence

ACM/EE 116: Introduction to Stochastic Processes and Modeling

Catalog listing Introduction to fundamental ideas and techniques of stochastic analysis and modeling. Random variables, expectation and conditional expectation, joint distributions, covariance, moment generating function, central limit theorem, weak and strong laws of large numbers, discrete time stochastic processes, stationarity, power spectral densities and the Wiener-Khinchine theorem, Gaussian processes, Poisson processes, Brownian motion. The course develops applications in selected areas such as signal processing (Wiener filter), information theory, genetics, queuing and waiting line theory, and finance.

Dependent courses:

Partially overlapping courses

Topics (Winter 2009)

  1. Probability spaces.
  2. Sigma algebras.
  3. Independence, Bayes formula.
  4. Continuous random variables
  5. Strong Law of Large Numbers. Monte Carlo Simulations.
  6. Modes of Convergence.
  7. Central Limit Theorem.
  8. Large Deviations (basic concepts)
  9. Conditional expectation. Filtrations.
  10. Martingales (definition, limit theorems, optimal stopping times, inequalities)
  11. Concentration of Measure (basic concepts, proof of McDiarmid's inequality as a martingale inequality).
  12. Poisson processes.
  13. Markov chains (basic concepts).
  14. Branching processes.
  15. Gaussian processes.
  16. Kalman/Wiener filters.
  17. Brownian Motion.
  18. Stochastic Differential Equations. Langevin processes. (basic concepts)

ACM 216: Markov Chain, Discret Stochastic Processes and Applications

Catalog listing Stable laws, Markov chains, classification of states, ergodicity, von Neumann ergodic theorem, mixing rate, stationary/equilibrium distributions and convergence of Markov chains, Markov chain Monte Carlo and its applications to scientific computing, Metropolis Hastings algorithm, coupling from the past, martingale theory and discrete time martingales, rare events, law of large deviations, Chernoff bound

Dependent courses

Partially overlapping courses

Topics (Winter 2008)

  1. Markov Models.
  2. Transition matrices and Markov Chains.
  3. Kernels and Markov chains on arbitrary spaces.
  4. Finite Markov Chains.
  5. Markov Chains on countable state spaces.
  6. Simulations with Markov Chains.
  7. MCMC algortithms.
  8. Simulated annealing.
  9. The Propp-Wilson algorithm.
  10. Sandwiching.
  11. Rate of convergence

Advanced courses

There are several advanced courses that build on ACM 116/216 and are offered on a semi-regular basis:

  • EE 164 - Stochastic and Adaptive Signal Processing
  • ACM 217 - Stochastic Differential Equations and Applications
  • ACM 217 - Stochastic Control Theory (taught once)
  • ACM 256 - Large Deviation Theory and Concentration Inequalities (special topics course)

Additional stochastic systems courses at Caltech

The following table lists all of the courses that I was able to find that have been taught in the last four years. Enrollments (when given) are for 2005-2008, based on data from the registrar.

Course Title Enroll 2005-06 2006-07 2007-08 2008-09
ACM/EE 116 Introduction to Stochastic Processes and Modeling 30-50 Owhadi Owhadi Owhadi Owhadi
ACM/ESE 118 Methods in Applied Statistics and Data Analysis 40-50 Schneider Schneider Tropp Candes
ACM 216 Markov Chains 15-20 Owhadi Owhadi Candes Owhadi
ACM 217 Advanced Topics in Stochastic Analysis 2-12 Owhadi Von Handel Hassibi N/O
ACM 257 Special Topics in Financial Mathematics 20 N/O Hill N/O N/O
Ae 115a Spacecraft Navigation (Kalman filters) 3-6 Watkins Watkins Watkins N/O
CDS 110b Introductory Control Theory (Kalman filters) 20-30 Murray Murray Murray MacMynowski
EE 163 Communications Theory 5-10 Arabshahi Quirk Quirk Quirk
Ec 122 Econometrics N/A Bossaerts Sherman Sherman Sherman
Ma 112ab Statistics N/A Lorden Lorden Lorden Lorden
Ma/ACM 144ab Probability (including Markov chains) N/A Strahov N/O Kang N/O
Ma 193 Advanced Topics - Random Matrix Theory N/A N/O N/O N/O Borodin
SS/Ma 214 Mathematical Finance N/A Cvitanic Cvitanic N/O N/O
SS 122 Econometrics N/A Sherman, Lee Sherman, Matzkin Sherman, Staff Sherman, Staff
SS 228 Applied Data Analysis for the Social Sciences N/A Katz Katz Katz Katz

Course listings

The course listings below are from the Caltech catalog, mainly to serve as a reference for the rest of the information on this page.

ACM/EE 116. Introduction to Stochastic Processes and Modeling. 9 units (3-0-6); first term. Prerequisite: Ma 2 ab or instructor’s permission.Introduction to fundamental ideas and techniques of stochastic analysis and modeling. Random variables, expectation and conditional expectation, joint distributions, covariance, moment generating function, central limit theorem, weak and strong laws of large numbers, discrete time stochastic processes, stationarity, power spectral densities and the Wiener-Khinchine theorem, Gaussian processes, Poisson processes, Brownian motion. The course develops applications in selected areas such as signal processing (Wiener filter), information theory, genetics, queuing and waiting line theory, and finance.

ACM/ESE 118. Methods in Applied Statistics and Data Analysis. 9 units (3-0-6); first term. Prerequisite: Ma 2 or another introductory course in probability and statistics. Introduction to fundamental ideas and techniques of statistical modeling, with an emphasis on conceptual understanding and on the analysis of real data sets. Multiple regression: estimation, inference, model selection, model checking. Regularization of ill-posed and rank-deficient regression problems. Cross-validation. Principal component analysis. Discriminant analysis. Resampling methods and the bootstrap.

ACM 216. Markov Chains, Discrete Stochastic Processes and Applications. 9 units (3-0-6); second term. Prerequisite: ACM/EE 116 or equivalent. Stable laws, Markov chains, classification of states, ergodicity, von Neumann ergodic theorem, mixing rate, stationary/equilibrium distributions and convergence of Markov chains, Markov chain Monte Carlo and its applications to scientific computing, Metropolis Hastings algorithm, coupling from the past, martingale theory and discrete time martingales, rare events, law of large deviations, Chernoff bounds.

ACM 217. Advanced Topics in Stochastic Analysis. 9 units (3-0-6); third term. Prerequisite: ACM 216 or equivalent. The topic of this course changes from year to year and is expected to cover areas such as stochastic differential equations, stochastic control, statistical estimation and adaptive filtering, empirical processes and large deviation techniques, concentration inequalities and their applications. Examples of selected topics for stochastic differential equations include continuous time Brownian motion, Ito’s calculus, Girsanov theorem, stopping times, and applications of these ideas to mathematical finance and stochastic control. Not offered 2008–09.

ACM 257. Special Topics in Financial Mathematics. 9 units (3-0-6); third term. Prerequisite: ACM 95/100 or instructor’s permission. A basic knowledge of probability and statistics as well as transform methods for solving PDEs is assumed. This course develops some of the techniques of stochastic calculus and applies them to the theory of financial asset modeling. The mathematical concepts/tools developed will include introductions to random walks, Brownian motion, quadratic variation, and Ito-calculus. Connections to PDEs will be made by Feynman-Kac theorems. Concepts of risk-neutral pricing and martingale representation are introduced in the pricing of options. Topics covered will be selected from standard options, exotic options, American derivative securities, term-structure models, and jump processes. Instructor: Hill.

Ae 115 ab. Spacecraft Navigation. 9 units (3-0-6); first, second terms. Prerequisite: CDS 110 a. This course will survey all aspects of modern spacecraft navigation, including astrodynamics, tracking systems for both low-Earth and deep-space applications (including the Global Positioning System and the Deep Space Network observables), and the statistical orbit determination problem (in both the batch and sequential Kalman filter implementations). The course will describe some of the scientific applications directly derived from precision orbital knowledge, such as planetary gravity field and topography modeling. Numerous examples drawn from actual missions as navigated at JPL will be discussed.

CDS 110 ab. Introductory Control Theory. 12 units (3-0-9) first, 9 units (3-0-6) second terms. Prerequisites: Ma 1 and Ma 2 or equivalents; ACM 95/100 may be taken concurrently. An introduction to analysis and design of feedback control systems, including classical control theory in the time and frequency domain. Modeling of physical, biological, and information systems using linear and nonlinear differential equations. Stability and performance of interconnected systems, including use of block diagrams, Bode plots, the Nyquist criterion, and Lyapunov functions. Robustness and uncertainty management in feedback systems through stochastic and deterministic methods. Introductory random processes, Kalman filtering, and norms of signals and systems. The first term of this course is taught concurrently with CDS 101, but includes additional lectures, reading, and homework that is focused on analytical techniques for design and synthesis of control systems.

Ec 122. Econometrics. 9 units (3-0-6); fi rst term. Prerequisite: Ma 112a. The application of statistical techniques to the analysis of economic data.

EE 163 ab. Communication Theory. 9 units (3-0-6); second, third terms. Prerequisite: EE 111; ACM/EE 116 or equivalent. Least mean square error linear filtering and prediction. Mathematical models of communication processes; signals and noise as random processes; sampling and quantization; modulation and spectral occupancy; intersymbol interference and synchronization considerations; signal-to-noise ratio and error probability; optimum demodulation and detection in digital baseband and carrier communication systems.

EE 164. Stochastic and Adaptive Signal Processing. 9 units (3-0-6); third term. Prerequisite: ACM/EE 116 or equivalent. Fundamentals of linear estimation theory are studied, with applications to stochastic and adaptive signal processing. Topics include deterministic and stochastic least-squares estimation, the innovations process, Wiener filtering and spectral factorization, state-space structure and Kalman filters, array and fast array algorithms, displacement structure and fast algorithms, robust estimation theory and LMS and RLS adaptive fields.

Ma 112 ab. Statistics. 9 units (3-0-6); first, second terms. Prerequisite: Ma 2 a probability and statistics or equivalent. The first term covers general methods of testing hypotheses and constructing confidence sets, including regression analysis, analysis of variance, and nonparametric methods. The second term covers permutation methods and the bootstrap, point estimation, Bayes methods, and multistage sampling.

Ma/ACM 144 ab. Probability. 9 units (3-0-6); second, third terms. Overview of measure theory. Random walks and the Strong law of large numbers via the theory of martingales and Markov chains. Characteristic functions and the central limit theorem. Poisson process and Brownian motion.

Ma 193 a. Random Matrix Theory. 9 units (3-0-6); first term. Prerequisite: Ma 108. Wigner matrices, Gaussian and circular ensembles of random matrices. Dyson's threefold way: orthogonal, unitary, and symplectic ensembles. Correlation functions; determinantal and Pfaffi an random point processes. Scaling limits. Fredholm determinant approach to gap probabilities.

SS/Ma 214. Mathematical Finance. 9 units (3-0-6); second term. A course on fundamentals of the mathematical modeling of stock prices and interest rates, the theory of option pricing, risk management, and optimal portfolio selection. Students will be introduced to the stochastic calculus of various continuous-time models, including diffusion models and models with jumps.

SS 222 abc. Econometrics. 9 units (3-0-6); first, second, third terms. Introduction to the use of multivariate and nonlinear methods in the social sciences.

SS 228. Applied Data Analysis for the Social Sciences. 9 units (3-0-6); third term. The course covers issues of management and computation in the statistical analysis of large social science databases. Maximum likelihood and Bayesian estimation will be the focus. This includes a study of Markov Chain Monte Carlo (MCMC) methods. Substantive social science problems will be addressed by integrating programming, numerical optimization, and statistical methodology.

Retrieved from "https://murray.cds.caltech.edu/index.php?title=Stochastic_systems_courses&oldid=21937"