1 Introduction. - 2 Preliminaries of Information Theory and Neural Networks. - 2. 1 Elements of Information Theory. - 2. 2 Elements of the Theory of Neural Networks. - I: Unsupervised Learning. - 3 Linear Feature Extraction: Infomax Principle. - 4 Independent Component Analysis: General Formulation and Linear Case. - 5 Nonlinear Feature Extraction: Boolean Stochastic Networks. - 6 Nonlinear Feature Extraction: Deterministic Neural Networks. - II: Supervised Learning. - 7 Supervised Learning and Statistical Estimation. - 8 Statistical Physics Theory of Supervised Learning and Generalization. - 9 Composite Networks. - 10 Information Theory Based Regularizing Methods. - References. Language: English