Multimodal Data Analysis & FusionOne of the long-standing challenges in signal processing and data analysis is the fusion of information acquired by multiple, multimodal sensors. The problem of information fusion has become particularly central in the wake of recent technological advances, which have led to extensive collection and storage of multimodal data. Nowadays, many devices and systems, e.g., cell-phones, laptops, and wearable-devices, incorporate more than one sensor, often of different types. Of particular interest in the context of our research are the massive data sets of medical recordings and healthcare-related information, acquired routinely, for example, in operation rooms, intensive care units, and clinics. The availability of such distinct and complementary information calls for the development of new theories and methods, leveraging it toward achieving concrete objectives such as analysis, filtering, and prediction, in a broad range of fields.
In our research, we address the problem from a manifold learning/geometric analysis standpoint. While most existing manifold learning methods consider only a single data set arising from a single manifold, we extend this basic setting and develop techniques for several data sets arising from multiple manifolds. In our recent work, we have introduced a new notion of a common latent manifold, which naturally arises in settings where different types of devices are used to measure the same activity/physical phenomenon. We developed a method of alternation diffusion, based on a product of diffusion operators, and showed that it extracts the common component underlying several data sets in an unsupervised manner. Based on the notion of common latent manifold, in another recent work, we further proposed two new composite diffusion operators that allow to isolate, enhance and attenuate the hidden components of multi-modal data with prior model knowledge. Fundamental algebraic concepts, involving the adjoint, the commutator and the symmetric and anti-symmetric parts of known diffusion operators were incorporated in the design of this new class of diffusion operators. In addition, the mathematical properties induced by these operators were investigated and asymptotic analysis in a regime where the number of data samples approaches infinity and the kernel scale is small was carried out. Particularly, we showed that the associated asymptotic limit operators of the new diffusion operators demonstrate a differential Laplace-like nature. These new operators facilitate the construction of efficient low-dimensional representations of data, which characterize the common structures and the differences between the manifolds underlying the different modalities.
A class of signal processing problems that arguably lag behind with respect to modern data analysis tools, where we have identified significant gaps in their existing models, arises from biomedicine and neuroscience applications. Currently, we are involved in collaborative projects concerning epilepsy seizure prediction, early detection of Alzheimer’s disease, fetal heart rate detection, automatic sleep stage identification, to name just a few.
