FOUNDATIONS OF INFORMATION THEORY AND COMPUTATIONAL NEUROSCIENCES - MOD. 1

The course is one of the mandatory educational activities of the Master of Science in Engineering Physics, Physics of the Brain curriculum, and enables the student to gain knowledge and understanding of the fundamental and applied concepts of probability and information theory.

1. Knowledge and understanding skills
To know and understand the laws of probability and information theory, and their contribution within the scientific method in the study of stochastic phenomena. The course also aims to develop critical thinking in students.

2. Ability to apply knowledge and understanding
Use the mathematics needed to describe stochastic phenomena.

3. Autonomy of judgment
Know how to evaluate the logical consistency of results, both in theory and in inference from experimental or empirical data.
Know how to recognize possible errors through critical analysis of the applied method

4. Communication skills
Know how to communicate the knowledge learned using appropriate terminology, both orally and in writing
Know how to interact with the lecturer and course colleagues in a respectful and constructive manner, particularly during work carried out in groups

5. Learning skills
Know how to take notes, selecting and collecting information according to its importance and priority
Know how to be sufficiently autonomous in collecting data and information relevant to the problem investigated

The course takes for granted many of the concepts covered in Mathematical Analysis I and Mathematical Analysis II courses (derivatives and integrals to one and more variables), Linear Algebra (vector spaces and operations between vectors, equations to eigenvalues).

1. Definitions of probability:
Kolmogorov's axioms and probability as an extension of logic to plausibility estimates
2. Stochastic independence, conditional probability, Bayes' theorem
3. Causal variables
4. Classical probability models: urn models, distributions of particles in states, random marches
5. Generator functions for integer causal variables and their application to random marches and cascade processes.
6. Borel-Cantelli lemma. Limits in probability. Law of large numbers.
7. Limit laws for sums (central limit theorem and Levy's laws) and for extremes (Random Energy model). Bounds on the validity of the central limit theorem.
8. Information, entropy, Shannon's theorem, asymptotic equipartition. Mutual and relative information, maximum entropy distributions.
9. Large deviation theory for fine-tailed and fat-tailed distributions for independent variables. Examples of correlated variables, Garner-Ellis theorem and phase transitions.
10. Applications to statistics: hypothesis testing and Stein's lemma, Fisher information and parameter estimation, Bayesian model selection and complexity estimates. Minimum description length theory.

W. Feller, An Introduction to Probability Theory and its Applications (J.Wiley & Sons 1968).
Cover and Thomas, Elements of Information Theory (J. Wiley & Sons 2006).
E. T. Jaynes, Probability Thoery: the logic of science, (Cambridge U. Press 2003).
M. Mezard, A. Montanari, Information, Physics and Computation (Oxford Univ. Press
2009).
C.W. Gardiner, Handbook of stochastic methods (Springer-Verlag, 1985).

Achievement of learning objectives is assessed through participation in question-and-answer (Q&A) sessions, an intermediate written test, and a final oral exam.

The written exam consists of problems similar to those done in class during the Q&A sessions.

Pre-recorded lectures and interactive question-and-answer sessions.

English

written and oral

This subject deals with topics related to the macro-area "Human capital, health, education" and contributes to the achievement of one or more goals of U. N. Agenda for Sustainable Development