NME130/Information theory

Michelle

• Tried to figure out what people wanted to see
• Decided that the way to go is to pull out a small piece that can be done in its entirety, but gives a sense of the point of view

Outline

1. Assumptions underlying information theory
   • Convenient versus critical
2. Heart of the matter
   • Long sequences of random variables are "easy" to predict (weak law, AEP)
     • This piece current takes 3.5 lectures * 1.5 hours = ~ 6 hours
   • Example: achievability (in sketch form) of the channel coding theorem
     • Can probably be done in 1-2 lectures of 1.5 hours each

• Entropy will have be introduced, but probably not entropy rate