Events

IFML Seminar

IFML Seminar: 12/4/26 - Foundations of few-step generation in diffusion language models

Sitan Chen, Harvard University, Assistant Professor of Computer Science

-

The University of Texas at Austin
2317 Speedway
Gates-Dell Complex (GDC) 6.302
Austin, TX 78712
United States

ifml_seminar_120426

Title: Foundations of few-step generation in diffusion language models


Abstract: A popular selling point of diffusion large language models (dLLMs) is their capacity for parallelism: the ability to generate sequences of text far more efficiently than autoregressive models, which require one forward pass per token. Yet among the many competing paradigms for dLLMs, from masked to uniform to Gaussian diffusion, principled understanding of how these different proposals compare in parallelism remains limited. In this talk, I will show that (1) all three can sample in a number of forward passes that scales with the dual total correlation of the underlying distribution, yet (2) there are natural families of distributions for which uniform and Gaussian diffusion quadratically outperform masked diffusion. Contrary to popular intuition that masked diffusions are harder to parallelize because they must commit to token values, the latter separation instead comes from the fact that the "critical windows" / "speciation transitions" in masked diffusion sampling are quadratically sharper than those in uniform and Gaussian diffusion sampling.


Bio: Sitan Chen is an Assistant Professor of Computer Science at Harvard SEAS, where he is a member of the Theory of Computation and the Harvard Quantum Initiative, and an associate faculty at the Kempner Institute. Previously, he was an NSF math postdoc at UC Berkeley, after completing his PhD in EECS at MIT in 2021. His work has been recognized with an NSF CAREER award, an ICML Outstanding Paper Award, and the Harvard Dean’s Competitive Fund for Promising Scholarship. Homepage: sitanchen.com 

Zoom link: https://utexas.zoom.us/j/89110047398