Alice submits a number of papers to a machine learning conference and has knowledge of the quality of her papers. Given raw grades rated by independent reviewers, can Bob obtain accurate estimates of the grades of the papers by asking Alice questions about the ground truth? In this talk, we address this problem in the case where the raw grades are generated by an additive model and the payoff of Alice is additive convex utility over all her papers. First, we prove that, if Alice is truth-telling in the sense that her payoff is maximized when she truthfully answers the questions about the underlying grades, then the questions must be formulated as pairwise comparisons between her papers. Next, we show that if Alice is required to provide a ranking of her papers, which is the most refined question via pairwise comparisons, she would be truth-telling. By leveraging the true ranking provided by Alice, moreover, we demonstrate that the grades provided by Bob have the optimal estimation performance in certain regimes among all truth-telling ways of information elicitation. In particular, the adjusted grades are indeed significantly more accurate than the raw grades when the number of papers is large and the raw grades are very noisy. Finally, we conclude the talk with several extensions and some refinements for practical considerations. This is based on an ongoing work and arXiv:2110.14802. 

4月22日
10:30am - 11:30am
地点
https://hkust.zoom.us/j/5616960008 (Passcode: hkust)
讲者/表演者
Prof. Weijie SU
University of Pennsyvania
主办单位
Department of Mathematics
联系方法
付款详情
对象
Alumni, Faculty and staff, PG students, UG students
语言
英语
其他活动
3月24日
研讨会, 演讲, 讲座
IAS / School of Science Joint Lecture - Pushing the Limit of Nonlinear Vibrational Spectroscopy for Molecular Surfaces/Interfaces Studies
Abstract Surfaces and interfaces are ubiquitous in Nature. Sum-frequency generation vibrational spectroscopy (SFG-VS) is a powerful surface/interface selective and sub-monolayer sensitive spect...
11月22日
研讨会, 演讲, 讲座
IAS / School of Science Joint Lecture - Leveraging Protein Dynamics Memory with Machine Learning to Advance Drug Design: From Antibiotics to Targeted Protein Degradation
Abstract Protein dynamics are fundamental to protein function and encode complex biomolecular mechanisms. Although Markov state models have made it possible to capture long-timescale protein co...