On Thursday, January 21, 2016, Thomas Steinke presented "The Power of Adaptivity in Data Analysis" at The Chinese University of Hong Kong's Institute of Theoretical Computer Science and Communications ITCSC - CSE Joint Seminar. Thomas' abstract is listed below.
Abstract
If a dataset is only used once, a rich theory exists for ensuring that the conclusions are valid. But what happens if the same dataset is reused for multiple analyses? Since each analysis may now depend onthe outcome of previous analyses, the danger of over fitting the dataset is increased. For example, if the same dataset is used to select a learning model and then fit that model, the resulting model may appear to explain the data better thanit should.In this talk, I will discuss a recent line of research on adaptive data analysis. I will show that there aresophisticated techniques--using tools from information theory and differential privacy–that enable us to ensure that adaptive analysis provides sound conclusions. I will also discuss how adaptive data analysis is inherently morepowerful than non-adaptive data analysis