NME130/Information theory

From Murray Wiki
Revision as of 19:27, 27 May 2009 by Murray (talk | contribs)
Jump to navigationJump to search


  • Tried to figure out what people wanted to see
  • Decided that the way to go is to pull out a small piece that can be done in its entirety, but gives a sense of the point of view


  1. Assumptions underlying information theory
    • Convenient versus critical
  2. Heart of the matter
    • Long sequences of random variables are "easy" to predict (weak law, AEP)
      • This piece current takes 3.5 lectures * 1.5 hours = ~ 6 hours
    • Example: achievability (in sketch form) of the channel coding theorem
      • Can probably be done in 1-2 lectures of 1.5 hours each
  • Entropy will have be introduced, but probably not entropy rate