Speaker
Dr. Mengxin (Maxine) Yu, Assistant Professor of Statistics & Data Science, Department of Statistics and Data Science, Washington University in St. Louis
Title
Statistics Seminar Series
Subtitle
Uncertainty Quantification for Modern Machine Learning Predictions
Physical Location
Allen 411
Abstract:
Quantifying the uncertainty of black-box machine learning predictions is a core problem in modern statistics. Methods for predictive inference have been developed under a variety of assumptions, often -- for instance, in standard conformal prediction -- relying on the invariance of the distribution of the data under special groups of transformations such as permutation groups. Moreover, many existing methods for predictive inference aim to predict unobserved outcomes in sequences of feature-outcome observations. Meanwhile, there is interest in predictive inference under more general observation models (e.g., for partially observed features) and for data satisfying more general distributional symmetries beyond exchangeability (e.g., network, rotationally invariant, or data with hierarchical structure). Here, we propose SymmPI, a unified methodology for predictive inference when data distributions have general group symmetries in arbitrary observation models. Our methods leverage the novel notion of distributional equivariant transformations, which process the data while preserving their distributional invariances. We show that SymmPI has valid coverage under distributional invariance and characterize its performance under distribution shift, recovering recent results as special cases. These methodologies are particularly relevant for cluster-randomized trials in clinical settings, where prediction reliability is essential.
If time permits, I will also briefly present our recent work on evaluating uncertainty and confidence measures in large language models. We introduce a novel method called rank calibration, which enables the identification of reliable uncertainty measures across a range of tasks and LLM models.
About the Speaker:
Dr. Mengxin (Maxine) Yu is an Assistant Professor in the Department of Statistics and Data Science at Washington University in St. Louis. Prior to joining WashU, she was a Postdoctoral Research Fellow in the Department of Statistics and Data Science at the Wharton School, University of Pennsylvania, working with Professor Dylan S. Small. She received her Ph.D. in Operations Research and Financial Engineering from Princeton University in 2023, advised by Professor Jianqing Fan. Earlier, she graduated summa cum laude (Guo Moruo Scholarship, <1%) from the University of Science and Technology of China (USTC) in 2018.
Her work has appeared in leading journals such as The Annals of Statistics, Journal of the Royal Statistical Society: Series B, Journal of the American Statistical Association, Journal of Machine Learning Research, and Operations Research. Dr. Yu has received several honors, including the IMS New Researcher Travel Award, the ASA Best Student Paper Award, and the SEAS Award for Excellence—the highest distinction for advanced graduate students at Princeton University.
Note:
Contact Prof. JZ at jzhang@math.msstate.edu for additional information.