Information and Coding Theory
Prerequisite: A third-year undergraduate course in Probability Theory and some familiarity with digital communications.
3 credits
Course Outline:
1. Introduction; channel models; source and channel coding
2. Preliminaries and review: coding gain, soft input soft output (SISO) decoding.
3. Information theory; measure of information; properties of the entropy function.
4. Source coding; variable and fixed-length source coding theorems; Huffman and L-Z coding.
5. Channel capacity theorem.
6. Block codes; error-detecting and error-correcting capability.
7. Linear block codes; generator and parity-check matrices; syndrome decoding.
8. Cyclic codes; generator and parity-check polynomials and matrices; encoding/decoding.
9. Convolutional codes; maximum likelihood decoding.
10. Network information theory.
11. Physical-layer security, secrecy capacity.
12. Rate distortion theory (as time permits).
13. Introduction to near-capacity codes: low density parity check (LDPC) and polar codes (as time permits).
14. Term project presentations.
Textbooks (recommended):
- T.M. Cover and J.A. Thomas, Elements of Information Theory, second edition, Wiley, 2006.
- S. Lin and D.J. Costello, Error Control Coding, second edition, Pearson/Prentice-Hall, 2004.
Other books:
- D.J.C. Mackay, Information Theory, Inference and Learning Algorithms, Cambridge, 2014.
- J.G. Proakis and M. Salehi, Digital Communications, 5th Edition, McGraw-Hill, 2008.
- S.B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice-Hall, 1995.
- R. Gallager, Information Theory and Reliable Communication, J. Wiley, 1968.