NME130/Information theory - Revision history

Murray: /* Discussion */

2009-05-27T19:57:10Z

Discussion

← Older revision		Revision as of 19:57, 27 May 2009
Line 49:		Line 49:
	** Chernoff theory		** Chernoff theory
	** Could also link to ACM/EE 116a		** Could also link to ACM/EE 116a

			* "Shape" of the course
			** There are conceptual things that take a while to sink in
			** Probably better spread out, but could be taught quickly

			* Some key concepts
			** The notion of typical (and how to think about that)
			** Design oriented optimization

			* Textbooks
			** Cover and Thomas for the information theory part
			** McEliece, McKay, A & Richardson for coding theory part

Murray: /* Discussion */

2009-05-27T19:47:06Z

Discussion

← Older revision		Revision as of 19:47, 27 May 2009
Line 45:		Line 45:
	** Eg, design of a large network		** Eg, design of a large network
	*** Layer as optimization versus information theory		*** Layer as optimization versus information theory

			* Linkages to CS
			** Chernoff theory
			** Could also link to ACM/EE 116a

Murray: /* Tracey */

2009-05-27T19:45:34Z

Tracey

← Older revision		Revision as of 19:45, 27 May 2009
Line 35:		Line 35:
	# Networking coding and its relation to network information theory, coding thoery and networking optimization (2-3 hr)		# Networking coding and its relation to network information theory, coding thoery and networking optimization (2-3 hr)
	# Connections with other fields (learning, cryptography) (2-3 hr)		# Connections with other fields (learning, cryptography) (2-3 hr)

			=== Discussion ===
			* Linkage to optimization
			** How could we tune optimization or information theory to streamline the two
			** In information theory, we engineer the optimization problem so that certain techniques will be able to find a solution
			** So: talk more about the ''design'' of optimization problems (could also hit mechanism design, protocol design)

			* Another approach: set a big goal that requires all of the tools
			** Eg, design of a large network
			*** Layer as optimization versus information theory

Murray: /* Tracy */

2009-05-27T19:34:22Z

Tracy

← Older revision		Revision as of 19:34, 27 May 2009
Line 35:		Line 35:
	# Networking coding and its relation to network information theory, coding thoery and networking optimization (2-3 hr)		# Networking coding and its relation to network information theory, coding thoery and networking optimization (2-3 hr)
	# Connections with other fields (learning, cryptography) (2-3 hr)		# Connections with other fields (learning, cryptography) (2-3 hr)

	~~=== Tracy ===~~

Murray: /* = Topics */

2009-05-27T19:34:12Z

= Topics

← Older revision		Revision as of 19:34, 27 May 2009
Line 25:		Line 25:
	* Want to impart a basic knowledge of what are some connections between them and other fields, so that students will have a basis for deciding if they want to go deeper		* Want to impart a basic knowledge of what are some connections between them and other fields, so that students will have a basis for deciding if they want to go deeper

	==== Topics ===		==== Topics ====
	# Framework and assumptions (1 hr)		# Framework and assumptions (1 hr)
	#* Differences between information theory and coding theory		#* Differences between information theory and coding theory

Murray: /* = Tracey */

2009-05-27T19:34:04Z

= Tracey

← Older revision		Revision as of 19:34, 27 May 2009
Line 15:		Line 15:
	* Should also be able to talk about stochastic versus worst case		* Should also be able to talk about stochastic versus worst case

	=== Tracey ==		=== Tracey ===

	==== Error correction coding ====		==== Error correction coding ====

Murray: /* Outline */

2009-05-27T19:33:53Z

Outline

@@ Line 12: / Line 12: @@
 #** Can probably be done in 1-2 lectures of 1.5 hours each
 * Entropy will have be introduced, but probably not entropy rate
 === Tracy ===

Murray at 19:27, 27 May 2009

2009-05-27T19:27:08Z

← Older revision		Revision as of 19:27, 27 May 2009
Line 11:		Line 11:
	#* Example: achievability (in sketch form) of the channel coding theorem		#* Example: achievability (in sketch form) of the channel coding theorem
	#** Can probably be done in 1-2 lectures of 1.5 hours each		#** Can probably be done in 1-2 lectures of 1.5 hours each
			* Entropy will have be introduced, but probably not entropy rate
	#* Entropy will have be introduced, but probably not entropy rate

	=== Tracy ===		=== Tracy ===

Murray at 19:26, 27 May 2009

2009-05-27T19:26:59Z

New page

=== Michelle ===
* Tried to figure out what people wanted to see
* Decided that the way to go is to pull out a small piece that can be done in its entirety, but gives a sense of the point of view

==== Outline ====
# Assumptions underlying information theory
#* Convenient versus critical
# Heart of the matter
#* Long sequences of random variables are "easy" to predict (weak law, AEP)
#** This piece current takes 3.5 lectures * 1.5 hours = ~ 6 hours
#* Example: achievability (in sketch form) of the channel coding theorem
#** Can probably be done in 1-2 lectures of 1.5 hours each

#* Entropy will have be introduced, but probably not entropy rate

=== Tracy ===