Research

Our research focuses on core foundational challenges integrating mathematical tools with real-world objectives to advance the state-of-the-art. We pursue ambitious use-inspired research, targeting frontier perceptual tasks in video, imaging and navigation.

See all Research Projects

Research Thrusts

  • Advanced Algorithms for Deep Learning

    Advanced Algorithms for Deep Learning

    We create fast, provably efficient tools for training neural networks and searching parameter spaces. We develop new theories to rigorously explain successful heuristics.

  • Learning with Dynamic Data

    Learning with Dynamic Data

    Since datasets are constantly evolving, we research new algorithms and models that can incorporate context and changes at training and test time, including robustness to perturbations.

  • Exploiting Structure in Data

    Exploiting Structure in Data

    What characteristics of a dataset help with training and inference? We define and uncover rich mathematical structures in datasets to improve downstream modeling and optimization.

  • Optimizing Real-World Objectives

    Optimizing Real-World Objectives

    We develop principled methods for automatically satisfying complex constraints and handling interactive feedback from users in real-world situations as is needed for safe robot navigation.

Use Inspired Applications 

The foundational research thrusts of IFML all have broad potential impact and feed directly into real-world applications. We selected three use-inspired research areas: video, imaging, and navigation. We work with industrial partners to redesign the whole video pipeline from recognition, compression/decompression to training and model design; we collaborate to improve the imaging pipeline and create novel priors for MRI and circuit quality control; and we develop new methods for autonomous navigation in highly unstructured environments while maintaining safe operation with high confidence.

Publications

  • Knowledge-Augmented Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop

    Yan Han, Chongyan Chen, Ahmed Tewfik, Benjamin Glicksberg, Ying Ding, Yifan Peng, Zhangyang Wang

    arXiv, v5, 2022

  • Sequential Online Chore Division for Autonomous Vehicle Convoy Formation

    Harel Yedidsion, Shani Alkoby, and Peter Stone

    arXiv, 2021

  • Towards Long-Form Video Understanding

    Chao-Yuan Wu, Philipp Krähenbühl

    CVPR, 2021