EE 639 Advanced Topics in Signal Processing and Communication

Intelligent Visual Surveillance

Fall 2010


[Home] [Syllabus]

News z


Visit course website at and the course site in Blackboard.


If you have never used Blackboard, visit to create an account and take the online training as instructed in

Instructor: Dr. Sen-ching Cheung (cheung at



Room 831 VisCenter (257-1257 ext. 80299)

By appointment or try your luck


MWF 10-11am


Right after class



First two lectures will be Wednesday and Friday 2:00pm – 2:50pm at FPAT 267

Subsequent lectures will be tentatively moved to WF 1:00pm-2:15pm (Vis Center Conference Room)


Course Description

Video surveillance has become part of our daily lives. Closed-circuit cameras are mounted in countless shopping malls for deterring crimes, at toll booths for assessing tolls, and at traffic intersections for catching speeding drivers. The combination of ubiquitous cameras, wireless connectivity, and powerful recognition algorithms has turned the once labor intensive surveillance monitoring process into a powerful automated system with applications extended far beyond security. In this course, we will survey the state-of-the-art key technologies behind the new field of Intelligent Video Surveillance, including background subtraction, crowd and traffic analysis, object tracking, activities and event detection, multi-camera calibration, placement and planning, as well as security and privacy. Our focus will be on the core mathematical concepts such as probabilistic graphical models, image features, multi-view geometry that enable these new technologies.  This course is primary for master and PhD students with good background in probability and image processing. The grade will be based on homework assignments, one midterm and an extensive final project.

Tentative Topics

  1. Probabilistic graphical model: unifying framework for density estimation, classification and time series
  2. Background Subtraction for Foreground Extraction
  3. Crowd density estimation
  4. Image and Motion Features
  5. Multi-Object Tracking
  6. Articulated Object Tracking
  7. Recognition of human biometrics, activities and events
  8. Calibration: from single to multiple cameras
  9. System issues of camera network: placement, control and communication
  10. Security and Privacy of visual surveillance


Your grade will be based on:




Homework Critique




Project Proposal


Project Presentation


Project Report



  1. Homework and Critique

Four homework assignments will be given throughout the semester. Most of the assigned problems are open-ended research questions and usually have many different solutions. To encourage peer learning, your homework solution must be submitted to Blackboard and will be released to the entire class. Each student will study the work of each other and prepare a critique to be discussed in class.

  1. Midterm

There will be one midterm. The midterm is closed-book and will be conducted in class.

  1. Project

50% of the grade will depend on a final research project. The topic must be selected from the class project page and you are required to develop a novel solution to the technical challenge(s) as described. There are three components to the overall project: (1) a proposal that summarizes current state-of-the-art, outlines the proposed solutions with supportive arguments on why they should perform better; (2) a presentation that motivates the design, describes the implementation and experiments, as well as explains and interprets the results with proper analysis; (3) a final report that is effectively a combination of the proposal and the written form of the presentation with more details and suggestions for future directions. The LaTeX templates with additional information on format can be downloaded here: proposal.tex and report.tex.

  1. Grade Assignment

-        The letter grade assignment is based on the following scale: from 100 to 90 pts => A, from 89 to 80 pts. => B, from 79 to 70 pts => C, from 60 to 69 pts. => D, from 59 to 0 pts. => E.   

5.      Plagiarism

-        I have a zero-tolerance policy for all forms of plagiarism, from cheating in the exam to copying a sentence from an un-cited source. Scenarios of possible plagiarism are discussed in this IEEE article. Not only you will lose all the points for that assignment, the incident will also be reported to the Department Chair who will determine the appropriate disciplinary action.

Required Text

There is no required text. The lectures will be based on selected papers from major conferences and journals in image processing, vision and signal processing. We will also use selected chapters from an to-be-published book “Computer Vision: Algorithms and Applications” by Dr. Richard Szeliski. An early draft of this book can be downloaded from this page.

Optional Reference: there are a number of recently published collections of papers that cover many of the same topics discussed in class.

1.     Y. Ma and G. Qian (Ed.), Intelligent Video Surveillance: Systems and Technology, CRC Press, 2009.

2.     H. Aghajan and A. Cavallaro (Ed.), Multi-Camera Network: Principles and Applications, Elsevier, 2009.

3.     A. Senior (Ed.), Privacy Protection in Video Surveillance, Elsevier, 2009.

Public-domain Software Required:

1.      OpenCV is the most commonly used library in computer vision. It is a C/C++ library for real-time computer vision and image processing. While other scientific computing package like Matlab or Mathematica provide some image processing routines, you are strongly encouraged to use OpenCV for its comprehensive set of routines and optimized performance.

-        SourceForge:

-        OpenCV Wiki:

2.      LaTeX is a typesetting language used in scientific community and BibTeX is a companion language for bibliography. Project proposal and report must be typeset with LaTeX and BibTeX, and submitted in pdf format. Homework solution is encouraged but not required. Templates will be provided to you. LaTeX and BibTeX are supported in just above every computing platform. In the Windows environment, the following public-domain software package is widely used:

-        Lyx ( is a WYSIWYG frontend of LaTeX.

-        MiKTeX ( is a LaTeX “compiler” that typesets a LaTeX document.

-        JabRef ( is a bibliography and citation manager based on BibTeX.



1.      EE 635 or good working knowledge in either image processing or computer graphics

2.      A solid background on discrete and continuous probability is necessary. Knowledge of stochastic processes is desirable but not necessary. You may want to brush up on your background with the following review.

-        Probability Review by Randall Berry:

3.      A good working knowledge of C and C++

-        A quick introduction to C Programming by Lewis Girod :

-        Programming in C – A Tutorial by Brian Kernighan:

Sen-ching Samson Cheung

Last modified: Wednesday, August 25, 2010