I attended an interesting meeting today where I had the opportunity to speak to a policy analyst who works for an elected official. I mentioned how important it is to have a control group in order to evaluate the effectiveness of a given policy and to infer the counter-factual outcome (i.e what would have happened if the policy in question had not been adopted).
She argued that it is often politically difficult to have a control group. How does the elected official explain why 1/2 the population received a treatment while the other 1/2 didn't?
I tried to argue that without a good control group that it is very hard to know whether a specific policy is cost-effective. She countered that with many "Big Data" public policies that the cost of treatment is relatively low. One example is building energy efficiency labels. Such labels (similar to a new car's MPG sticker) indicate whether a given building is energy efficient or not. In such cases, she argued that any observed improvement in outcomes (for example investments in energy efficiency) can be embraced by government officials as a sign that their policies are effective.
This discussion reminded me of the old paper that CEOs are paid for luck. If good things happen to a company during a boom, people call the CEO a genius and he gets even richer. To an economist, we seek to understand whether he caused his firm's good outcome but it may be the case that other decision makers such as voters are not so sophisticated and elected officials are willing and eager to claim credit for any observed progress.
So, the point of this blog post is that voters claim they want cost effective public policies but then rebel against standard research method to pinpoint whether such policies actually exist.