IFML Seminar: 11/07/25 - Model Self-improvement via Optimal Retraining

Abstract: Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model’s performance. While prior works have demonstrated the benefits of specific heuristic retraining schemes, the question of when retraining is useful and how to optimally combine the model’s predictions and the provided labels remains largely open. In this talk, I will discuss this fundamental question in the context of binary classification tasks. We develop a principled framework based on approximate message passing (AMP) to analyze iterative retraining procedures for two ground truth settings: Gaussian mixture model (GMM) and generalized linear model (GLM). We also quantify the performance of this optimal retraining strategy over multiple rounds. We complement our theoretical results by proposing a practically usable version of the theoretically-optimal aggregator function for linear probing with the cross-entropy loss, and demonstrate its superiority over baseline methods in the high label noise regime. Time permitting, I will discuss extensions in large language models.
 

Bio: Adel Javanmard is a Professor of Data Sciences and Operation at USC Marshall School of Business, where he also serves as an Executive member on the Outlier Research in Business (iORB) program and on the Marshall Leadership fellow program. He is also a research scientist at Google. Prior to joining USC in 2015, he was a NSF postdoctoral research fellow at UC Berkeley. He completed his PhD at Stanford University in 2014. His research interests are broadly in the area of uncertainty-aware machine learning, high-dimensional inference, optimization, and personalized decision-making. Adel is the recipient of several awards and fellowships, including the Alfred P. Sloan Research Fellowship in Mathematics, the IMS Tweedie Researcher award, the NSF CAREER award, Dean’s associate chair title, Dean's award for Research Excellence, Golden Apple Teaching Awards, Thomas Cover dissertation award from the IEEE Society, and several faculty awards from Google Research, Adobe Research, and Amazon.

Zoom link: https://utexas.zoom.us/j/84254847215