http://www.info620.ece.mcgill.ca/Eng.gif           3480 University Street, Montreal, Quebec, CANADA

McGill University

Department of Electrical and Computer Engineering

Academic Integrity

 McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the code of students conduct and disciplinary procedures (see academic integrity for more information).
 
 

ECSE-620B   INFORMATION THEORY AND CODING

Winter 2015

 

 General Information:

 

 

Instructor:

Prof. H. Leib, Tel. 398-8938,   Room 757 ENGMC
email : harry.leib@mcgill.ca
office hours : Thursday, Friday 14:30 - 16:30

 

 

Text book:

Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, 2nd ed., John Wiley & Sons Inc. 2006

 

 

References:

Raymond W. Yeung, Information Theory and Network Coding, Springer 2008

 

 

R. G. Gallager, Information Theory and Reliable Communication, John Wiley & Sons Inc. 1968

 

 

James Gleick, The information: A History, A Theory, A Flood, Pantheon Books 2011

 

Selected papers.

 

 

Pre-requisites :

A course on random processes (such ECSE-509 or ECSE-510) and a course on communication systems (such as ECSE-411, CSE-511, ECSE-521)

 

 

Final mark composition :

Assignments (20%), one midterm test (2h, open books, 30%), term project (50%)

 

 

Schedule / Location :

Wednesdays and Fridays, 11:30 - 13:00, room ENGTR 1090

 

 

First Class: 

Wednesday, January 7, 2015

 

 

 

Course Outline:

 

Information Theory is a mathematical discipline that seeks to quantify the notion of information and its transmission capabilities. Information Theory was introduced in 1948 by Claude Shannon, and provides the foundations for modern communication systems. Subsequently, Information Theory found applications in many other fields, such as computer science and statistics. Nowadays, Information Theory is an active research discipline, attracting talented researchers from a variety of fields.

This is a first course in Information Theory. Its main objectives are to provide a solid basis in this field, as well as an ability of using Informatic Theoretic techniques for a variety of applications. Students will be encouraged to choose term projects  in Information Theory that are related to their own research area. Guidelines for term projects

The course covers the following main topics :

1)     Introduction and overview.

2)     Information sources and channel models, entropy and mutual information.

3)     Lossless source coding (data compression).

4)     Source coding with a fidelity criterion (rate distortion theory).

5)     Performance limits for channel coding (channel capacity)

6)     Multiple-Input Multiple-Output (MIMO) channels.

7)     An introduction to multi-user information theory (if time permits).

 

 

 
Useful Links:

Shannon's pioneering work

Information Theory - Britannica online

Introduction to Probability Theory - book

Probability web

Entropy on the web

Entropy and Information Theory on the web

 

Messages

TERM TEST

 Date : 

TBD

 Time : 

TBD

 Location : 

TBD

 Material :

1) Class lectures: from the beginning of the term up to TBD

.

2) Text book : TBD

.

3) Assignments : TBD

 Type of test : 

Open books, open notes, all hand-held calculators allowed, notebook computers or any other computers not allowed.