At the heart of almost every economic theory are preferences. Preferences identify what we want. What choices we’d make if given the chance. As economists, knowing people’s preferences allows us to make welfare statements, such as whether a given policy helps or harms a given person. Knowing their preferences also helps us predict their behavior. Which insurance policies they’ll choose, how they’ll bid in an auction, and whether they’ll refinance their home next year.
When we run economic experiments in the laboratory, we have actual participants engage in simulated economic decisions. Essentially, we measure their preferences. And to do that, we pay subjects in a way that mimics the real-world decisions we’re interested in studying. For example, if we run a simulated auction in the lab, we tell each subject (privately) that if they win the item, it’s worth $X dollars to them (meaning, we pay them $X if they win), but of course they must pay their bid if they win. Just like in the real world, they’re not told the (private) valuations of other bidders. We then observe their bidding behavior—which bidding strategies they prefer—and test whether our theoretical models accurately predict those choices.
The key to this methodology is providing real incentives. In the auction example, the incentives come in the form of dollars earned by the winning bidder. But providing incentives can sometimes backfire if we’re not careful! Here’s a contrived example to illustrate. Suppose we ask a subject two questions: “1. Would you rather have an apple or a left shoe? 2. Would you rather have a banana or a right shoe?” And we’ll incentivize the subject by paying them the things they choose. Since single shoes are useless, we would expect almost every subject to prefer the apple and the banana. But, if both questions are paid, a clever subject should realize that by answering “left shoe” and “right shoe” they will ultimately be paid a pair of shoes, which is far more valuable to them than a couple of pieces of fruit. In other words, they have a strategic incentive to lie to us about their preferences.
The problem isn’t limited to shoes. For example, if I ask you to make several choices between a safe treasury bill and a risky stock, you might pick the stocks every time (even if you hate bearing risk) because the portfolio of stocks may actually be quite low-risk. In this way, the stocks act like shoes: the bundle of stocks is more valuable than the sum of their parts.
In their paper “Incentives in Experiments,” published in the Journal of Political Economy in 2018, OSU Profs. Yaron Azrieli and Paul J. Healy, along with their coauthor Prof. Christopher Chambers from Georgetown, analyze this problem. They argue in favor of a system of “paying one randomly,” meaning the subject makes many decisions, but only one of their choices is actually chosen for payment. In this way they can only receive a single shoe, or a single stock. This breaks the complementarities between their choices.
But this raises a new concern: subjects are now technically receiving a sort of “meta-gamble” over the apples, bananas, and shoes. A lottery over these objects, which itself is risky. Previous critics of the “pay one randomly” scheme have argued that this may require strong assumptions on how subjects view such meta-gambles. Profs. Azrieli, Chambers, and Healy show, however, that this method will lead to truth-telling under very weak assumptions about how people view these meta-gambles. Indeed, the logic is fairly simple: If you pick “apple, banana” then you get a coin flip that pays “apple if heads” and “banana if tails”. If you lie and say “apple, right shoe” then you get the same outcome if heads is flipped, and you get a strictly worse outcome if tails comes up. In that way, “apple, right shoe” is dominated by the truthful announcement “apple, banana.” So, as long as subjects never make these sorts of dominated announcements, the “pay one randomly” method will work flawlessly. In fact, they prove that it is the only way to pay subjects to avoid these problems.
Because of this article, the “pay one randomly” system has become the accepted standard for incentivizing laboratory experiments. The paper has over 300 citations and has spawned a new theoretical literature on the link between incentives in experiments and the behaviors they generate.
