Statistical learning theory
is playing an increasingly important role in diverse areas such as communication,
bioinformatics, computer vision, neuroscience, and economics. The goals of this
course are twofold: first, provide a solid foundation on the fundamental
concepts in statistical learning theory so that graduate students can apply
them in their own research; second, discuss recent results and new applications
in this area so as to stimulate new research ideas. The course will meet three
times a week, discussing papers, book chapters, and homework problems. The
meetings will tentatively be held MWF 4:30-5:30pm at MIA Lab in the
Graphical model, sum-product and junction tree
Linear and generalized linear models
Exponential family, sufficiency, conjugacy
Density estimation, kernel methods, mixture
Expectation-Maximization algorithm for parameter
Hidden Markov models (HMM)
Factor analysis, principal component analysis
analysis (CCA) and independent component analysis (ICA)
Approximate inferencing I: Markov-chain
Monte-Carlo (MCMC) and particle filtering
Model selection, marginal likelihood, AIC, BIC
Vapnik Chervonenkis theory and risk bounds
Kernel methods and Support Vector Machine (SVM)
Ensemble methods: bagging and boosting
Nonparametric Bayes, Dirichlet processes
Decision networks, Markov decision processes and
Grades will be assigned based on participation (50%)
and a substantialfinal project (50%).
Titles and scopes of final projects will be jointly determined by the
instructor and students. Each topic should involve substantial amount of
LITERATURE SURVEY AND EXPERIMENTAL RESULTS. The prerequisite are basic
statistics, probability and stochastic systems.
No text required. Copies of
papers and book chapters will be provided. The following books are recommended:
The Elements of Statistical Learning by T. Hastie
All of Statistics by L. Wasserman
The Nature of Statistical Learning Theory by V.
Learning with Kernels by B. Schlkopf and A. J.
Probabilistic Networks and Expert Systems by R.
G. Cowell et al.
Independent Component Analysis by A. Hyvarinen et
Pattern Recognition by S. Theodoridis and K.
Artificial Intelligence: A Modern Approach by S.
Russell and P. Norvig