||Introduction/history; basic information measures - entropy,
relative entropy, mutual information. ||[slides] |
| 4/1/04 ||Convexity, Jensen's inequality and consequences,
convexity properties of entropy and mutual information.
| 4/6/04 ||The Data Processing Inequality, Fano's
Inequality, the Asymptotic Equipartition Property.
| 4/8/04 || Typical sets, Shannon's Source Coding theorem and
converse, Entropy rate of a stochastic process.
| 4/13/04 ||Entropy rate of a stochastic process,
| 4/15/04 || Lossless source codes, uniquely decodable
codes, instantaneous codes, Kraft inequality, optimal codes.
| 4/20/04 || Kraft inequality for uniquely decodable
codes, Shannon codes, Huffman Codes.
| 4/22/04 || Optimality of Huffman Codes, universal
source codes, Lempel-Ziv Compression
| 4/27/04 || Introduction to channel coding,
discrete-time/discrete alphabet channels,
discrete memoryless channels, information capacity.
| 4/29/04 ||Calculating capacity- symmetric channels,
convex optimization; Joint typicality
| 5/4/04 || MID-TERM EXAM.
| 5/7/04 ||Joint typicality; block codes, random coding argument.
| 5/11/04 || Proof of the direct part of channel coding
theorem for DMC; expurgated codes; implementation issues.
| 5/13/04 ||Strong Coding theorems, error exponents and
reliability functions; Converse to the coding theorem.
| 5/18/04 || Converse to the coding theorem for bit
error probability, the joint source channel coding
theorem, feedback capacity
| 5/20/04 || Feedback capacity, continuous-valued
random variables and differential entropy.
| 5/25/04 || More on Differential entropy, Gaussian
RV's, capacity of the
discrete-time additive white Gaussian noise channel.
| 5/27/04 ||Coding theorem and converse for additive
white Gausian noise channel, High SNR and low SNR regimes,
parallel Gaussian channels and water-filling.
| 6/1/04 || Continuous-time (waveform) channel models,
sampling and equivalent discrete-time channels, colored
Gaussian noise channels.
| 6/3/04 || Rate-Distortion theory, quantization,
Lloyd-Max algorithm, vector quantizers,
rate-distortion function, Gaussian sources