Strategic Sample Selection Benefitting a Decision Maker

Economic data are often nonrandomly selected, due to choices made by subjects under investigation or sample inclusion decisions by data analysts. Are selected data more or less informative than the same amount of random data? For example, fixing a number of objects for sale, are the winning bids in a more competitive auction more or less revealing of market demand? When a new treatment is given to the healthiest patients rather than to random patients in a group, does inference improve or worsen? When testing a candidate, should the examiner ask questions at random or allow the candidate to select the most preferred questions out of a larger batch? And how does the common‐law right of peremptory challenge—by which the attorney on each side of a trial can strike down a number of jurors—affect judgment quality?

Peter Norman Sørensen and his co-authors Alfredo Di Tillio and Marco Ottaviani develop multivariate accuracy for interval dominance ordered preferences and show that sample selection always benefits (or always harms) a decision maker if the reverse hazard rate of the data distribution is log‐supermodular (or log‐submodular), as in location experiments with normal noise.

Peter, Alfredo and Marco find nonpathological conditions under which the information contained in the winning bids of a symmetric auction decreases in the number of bidders. Exploiting extreme value theory, we quantify the limit amount of information revealed when the presample size (number of bidders) goes to infinity.

In a model of equilibrium persuasion with costly information, they derive implications for the optimal design of selected experiments when selection is made by an examinee, a biased researcher, or contending sides with the peremptory challenge right to eliminate a number of jurors.

There is a lot more to the work than we share here - read the full research article 'Strategic Sample Selection' in Econometrica.