Agenda

Antonietta Mira
Università della Svizzera Italiana and Università degli Studi dell'Insubria

Abstract

Real-world datasets tend to show a high degree of (possibly) non-linear correlations and constraints between their features. This means that, despite a very large embedding dimensionality, data typically lie on a manifold characterized by a much lower intrinsic dimension (ID) which, in the presence of noise, may depend on the scale at which the data is analysed. This fact raises interesting questions: How many variables, or combinations there of, are necessary to describe a real-world data set without significant information loss? What is the appropriate scale at which one should analyze and visualize the data? These two issues, which are often considered unrelated, are actually strongly entangled, and can be addressed within a unified framework.

We introduce an approach in which the optimal number of variables and the optimal scale are determined self-consistently, recognizing and bypassing the scale at which the data are affected by noise. To this aim we estimate the data ID in an adaptive way, and exploit it as a summary statistics in Approximate Bayesian Computation for inference in network type data. Sometimes, within the same dataset, it is possible to identify more than one ID meaning that different subsets of the data points lie onto manifolds with different IDs. Identifying these manifold provides a clustering of the data, and in many real world applications a simple topological feature, like the ID, allows to uncover a rich data structure and improves our insight into subsequent statistical analysis.

Examples of these applications range from gene expression to protein folding, pandemic evolution, FMRI, all the way to finance, sport data and the analysis of the representations of neural networks.

- - -

This lecture is included in the programme of the 2024 ISBA World Meeting

For further information please contact isba2024@unive.it