publications
2026
- arXivFidelity-Diversity Metrics for TextAmanda Wang, Tudor Manole, Florentina Bunea, and 1 more authorarXiv preprint arXiv:2607.04563, 2026
As language modeling technology matures, there is an increasing research focus on the composition and curation of datasets used to train these models. For instance, practitioners commonly seek to augment high-quality datasets with additional text to enhance the performance of models trained on that data. However, informed decisions about data augmentation require more nuanced assessments about data quality. We build on work measuring the precision and recall of generative models to develop a pair of metrics that quantify (1) fidelity, capturing how closely candidate text resembles reference data, and (2) diversity, capturing how well it covers the modes of the reference dataset. Our metrics are based on optimal transport divergence functionals between discrete text summaries. In experiments on M2D2 text datasets, we show that these metrics are able to disentangle a lack of fidelity from a lack of diversity in deficient candidate text. In further experiments, our metrics detect diversity deficits in synthetic GSM8K-style math datasets, which correlate with degradations in downstream accuracy of language models finetuned on this synthetic data.
@article{wang2026fidelity, title = {Fidelity-Diversity Metrics for Text}, author = {Wang, Amanda and Manole, Tudor and Bunea, Florentina and Thickstun, John}, journal = {arXiv preprint arXiv:2607.04563}, year = {2026}, }
2025
- arXivPrincipled Out-of-Distribution Generalization via SimplicityJiawei Ge, Amanda Wang, Shange Tang, and 1 more authorarXiv preprint arXiv:2505.22622, 2025
Modern foundation models exhibit remarkable out-of-distribution (OOD) generalization, solving tasks far beyond the support of their training data. However, the theoretical principles underpinning this phenomenon remain elusive. This paper investigates this problem by examining the compositional generalization abilities of diffusion models in image generation. Our analysis reveals that while neural network architectures are expressive enough to represent a wide range of models – including many with undesirable behavior on OOD inputs – the true, generalizable model that aligns with human expectations typically corresponds to the simplest among those consistent with the training data. Motivated by this observation, we develop a theoretical framework for OOD generalization via simplicity, quantified using a predefined simplicity metric. We analyze two key regimes: (1) the constant-gap setting, where the true model is strictly simpler than all spurious alternatives by a fixed gap, and (2) the vanishing-gap setting, where the fixed gap is replaced by a smoothness condition ensuring that models close in simplicity to the true model yield similar predictions. For both regimes, we study the regularized maximum likelihood estimator and establish the first sharp sample complexity guarantees for learning the true, generalizable, simple model.
@article{ge2025principled, title = {Principled Out-of-Distribution Generalization via Simplicity}, author = {Ge, Jiawei and Wang, Amanda and Tang, Shange and Jin, Chi}, journal = {arXiv preprint arXiv:2505.22622}, year = {2025}, } - AAMASMaximizing Truth Learning in a Social Network is NP-hardFilip Úradník, Amanda Wang, and Jie GaoIn Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2025
Sequential learning models situations where agents predict a ground truth in sequence, by using their private, noisy measurements, and the predictions of agents who came earlier in the sequence. We study sequential learning in a social network, where agents only see the actions of the previous agents in their own neighborhood. The fraction of agents who predict the ground truth correctly depends heavily on both the network topology and the ordering in which the predictions are made. A natural question is to find an ordering, with a given network, to maximize the (expected) number of agents who predict the ground truth correctly. In this paper, we show that it is in fact NP-hard to answer this question for a general network, with both the Bayesian learning model and a simple majority rule model. Finally, we show that even approximating the answer is hard.
@inproceedings{uradnik2025maximizing, title = {Maximizing Truth Learning in a Social Network is NP-hard}, author = {Úradník, Filip and Wang, Amanda and Gao, Jie}, booktitle = {Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)}, year = {2025}, }