There are many treatments in this world. I bet that reading this blog has a small negative treatment effect on your IQ while brushing your teeth has a positive effect on reducing cavities.

Many economists are attempting to measure such treatments because we care about what is "good public policy" and we care about causality. If deworming children in a LDC CAUSES an increase in school attendance, then this suggests that there are benefits from such a policy that may exceed the costs of the policy.

Treatment effects researchers face at least 2 challenges. First, they must establish that the treatment they are studying such as "attending Harvard" is not correlated with key variables that a researcher may not observe such as "intrinsic aptitude". If there is a positive correlation between the key causal variable and such unobservables, then a researcher will over-estimate the impact of the treatment (in this case attending harvard) because it partially proxies for high unobserved ability. The second challege a researcher faces is extrapolation. If a treatment works in one setting, how do you know that it will work in a different nation and a different economic environment. This raises the issue of "heterogeneous treatment effects" which Deaton talks about below.


Angus Deaton's quote

"But there are limits. Take Banerjee’s example of flip charts. The effectiveness of flip charts clearly depends on many things, of which the skill of the teacher and the age, background, and previous training of the children are only the most obvious. So a trial from a group of Kenyan schools gives us the average effectiveness of flip charts in the experimental schools relative to the control schools for an area in western Kenya, at a specific time, for specific teachers, and for specific pupils. It is far from clear that this evidence is useful outside of that situation. This qualification also holds for the much more serious case of worms, where the rate of reinfection depends on whether children wear shoes and whether they have access to toilets. The results of one experiment in Kenya (in which there was in fact no randomization, only selection based on alphabetical order) hardly prove that deworming is always the cheapest way to get kids into school, as Banerjee suggests."


source http://bostonreview.net/BR31.4/deaton.html


Jim Heckman's recent work has pushed further on this point. If you would like to read some hard papers take a look at these:

jenni.uchicago.edu/underiv/documentation_2006_03_20.pdf

www.nuff.ox.ac.uk/users/nielsen/res/Heckman/Heckman-Urzua-Vytlacil_IV.pdf

If people differ with respect to what they will gain from taking a treatment and if THEY KNOW what their personal benefit would be, then there will be self-selection--- those who gain the most from treatment will be more likely to take treatment. A researcher who ignores this point will tend to over-estimate the gains from treatment for a random person assigned treatment from observing the treatment effects for the treated.

It will be interesting to see how radomization researchers handle the challenge that Heckman has posed. For example, If i know that i would gain greatly from treatment but I've been assigned to the control group in your trial will I pursue efforts to gain treatment? In this case, this would "contaminate" the experiment because you would code me as part of the control group while I really took treatment. In this case, the treatment effect estimate would under-estimate the true treatment effect because the control group represents a mixture of people who did not take the treatment and those (like me) who figured out a way to take it.