1、QuestionsThe relationship of interestThe ideal experimentThe identification strategyThe mode of inference隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果1QuestionsThe relationship of iThe relation of interestMost questions are about casual and effectExampleSmaller classes are better for the learning of the studentsLower co

2、payment encourages the health utilizations隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果2The relation of interestMost qCoefficient EquationsSlopey-interceptPrediction Equation隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果3Coefficient EquationsSlopey-inThe estimates of regression shows the extent of “correlation”, but we are interested to know

3、 “causation” Key issue in empirical analysis is separating causation from correlation.Correlated means that two economic variables move together.Casual means that one of the variables is causing the movement in the other.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果4The estimates of regression shTHE IMPORTANT DISTINCTIO

4、N BETWEEN CORRELATION AND CAUSATIONThere are many examples where causation and correlation get confused.It is critical for government policy to understand the difference; otherwise policy may not have the intended impact.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果5THE IMPORTANT DISTINCTION BETWTHE IMPORTANT DISTINCTIO

5、N BETWEEN CORRELATION AND CAUSATIONOne example concerns SAT preparation courses.In 1988, Harvard interviewed its freshmen and found those who took SAT “coaching” courses scored 63 points lower than those who did not.One dean concluded that the SAT courses were unhelpful and “the coaching industry is

6、 playing on parental anxiety.”隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果6THE IMPORTANT DISTINCTION BETWThe ProblemIn both examples, there is a common problem: an attempt to interpret a correlation as a causal relationship, without sufficient thought to the underlying data generating process.For any correlation betwee

7、n two variables A and B, there are three possible explanations for a correlation:A is causing B.B is causing A.Some other factor is causing both.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果7The ProblemIn both examples, tThe ProblemIn the Harvard SAT example, the possibilities could be:SAT prep courses worsen preparatio

8、n for the SATs.Those with poorer test taking ability take prep courses to try to catch up.Those who are generally nervous both like to take prep courses and do the worst on standardized exams.Harvard dean thought the first possibility was correct.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果8The ProblemIn the Harvard SA

9、T The ProblemAlthough the peasants or the Harvard dean could actually be correct, odds are they are misinterpreting the underlying process at work.For policy purposes, what we care about is causation.Knowing that two factors are correlated gives you no predictive power.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果9The P

10、roblemAlthough the peasanThe Problem of BiasIn this case, the assignment of the intervention was not random.This means the treatment and control groups are not identical.Non-random assignment, in turn, could cause bias.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果10The Problem of BiasIn this cas The Problem of BiasBias

11、represents any source of difference between treatment and control groups that is correlated with the treatment, but not due to the treatment.In the SAT example, the impact of SAT courses is biased by the fact that those who take the prep course are likely to do worse on the SAT for other reasons.隨機控

12、制實驗了解接受遠端祈禱心臟病病人的治療效果11 The Problem of BiasBias repreThe Problem of BiasBy definition, such differences do not exist in a randomized trial, since the groups are not different in any consistent fashion.As a result, randomized trials have no bias, and it is for this reason they are the “gold standard”

13、 for empirically estimating causal effects.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果12The Problem of BiasBy definitiThe ideal experimentWhat sort of experiment can ideally be used to capture the casual effect?Golden standard: controlled trialsRandomize subjects into the treatment and control group, then compare thei

14、r outcome difference between controlled and treatment groupCommonly used to answer questions in natural science, but difficult to implement to answer questions in social science for various issues隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果13The ideal experimentWhat sort MEASURING CAUSATION WITH DATA WED LIKE TO HAVE:

15、RANDOMIZED TRIALSWith random assignment, the assignment of the intervention is not determined by anything about the subjects.As a result, the treatment group is identical to the control group in every facet but one: the treatment group gets the intervention.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果14MEASURING CAUSAT

16、ION WITH DATA Example I: does the payer work?Example (Harris et al.): “隨機控制實驗：瞭解接受遠端祈禱心臟病病人的治療效果 “隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果15Example I: does the payer workHarris et el.實驗設計原則：“隨機, 控制, 雙盲,事前，同時實驗. “隨機：病人隨機分配到禱告與否控制：有些病人沒有禱告雙盲：病人或醫師不知道為實驗或對照組事前：在治療前隨機分配同時：實驗同時進行隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果16Harris et el.實驗

17、設計原則：“隨機, 控制, 雙Harris et el. 的設計隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果17Harris et el. 的設計隨機控制實驗了解接受遠端祈隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果18隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果18Harris et el. 的結論“ 結論：遠端禱告有效”隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果19Harris et el. 的結論“ 結論：遠端禱告有效”TANF and labor supply among single mothersTANF is “Temporary Assistance for

18、Needy Families.”Cash welfare for poor families, mainly single mothers.For example, in New Mexico, family of three receives $389 per month.Assume the two “goods” in utility maximization problem are leisure and food consumption.Whatever time is not devoted to leisure is spent working and earning money

19、.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果20TANF and labor supply among siRandomized Trials in the TANF ContextIt is believed that an increases in labor supply when TANF benefits are cut, but the magnitude of the effect is unclear.One could design a randomized trial to learn about the elasticity of employment with r

20、espect to TANF benefits.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果21Randomized Trials in the TANF Randomized Trials in the TANF ContextImagine a large group (say, 2000) of single mothers were randomly assigned to one of two groups with a coin flip:The “control” group continues to receive a guarantee of $5,000.The “tr

21、eatment” group now has their TANF benefit cut to $3,000.Follow groups for a period of time, and measure the work effort.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果22Randomized Trials in the TANF Randomized Trials in the TANF ContextIn an experiment like this in California in 1992, the elasticity of employment with res

22、pect to welfare benefits was estimated to be -0.67.Thus, a 10% decrease in benefits resulted in a 6.7% increase in employment.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果23Randomized Trials in the TANF Why We Need to Go Beyond Randomized TrialsRandomized trials present some problems:They can be expensive.They can take

23、a long time to complete.They may raise ethical issues (especially in the context of medical treatments).The inferences from them may not generalize to the population as a whole.Subjects may drop out of the experiment for non-random reasons, a problem known as attrition.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果24Why

24、We Need to Go Beyond RandoWhy We Need to Go Beyond Randomized TrialsFor these reasons (especially the first one about randomized trials being expensive), economists often take different approaches to try to assess causal relationships in empirical research.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果25Why We Need to Go

25、 Beyond RandoWhat is your identification strategyHow can you obtain the casual effect using your observational data?Drawing inference from the observational data needs to be very carefulLikely to suffer from the selection bias 隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果26What is your identification stExample I根據9999泛亞

26、人力銀行調查顯示，有六成八的大學應屆畢業生求職處處碰壁，到現在還找不到工作，而這些失業青年目前生活經費的主要來源，有七成以上是在家靠父母親友養，更令人憂心的是，有一成的社會新鮮人是以借貸、舉債過日。9999泛亞人力銀行營運長楊肯誠認為，八月份二技、四技二專考試放榜後，部分為被錄取的高職、專科生也將投入就業市場，預估九月份的應屆畢業生失業率更為嚴重，有可能超越去年九月的七成二。該人力銀行昨天在臺北的喜來登酒店舉行記者會，公布八月三日至十四日所進行今年找全職工作的應屆畢業生就業狀況調查，針對跟人力銀行資料庫符合條件的九萬二千一百五十二名應屆畢業生發出問卷，有效樣本共一萬九千三百七十八份，回收率百分

27、之廿一，當信心水準為百分之九十五時，誤差值為正負百分之零點八。受訪樣本集中在大專以上學歷為主，占全體百分之八十七點五。試就上述論點一一評論下列敘述：隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果27Example I根據9999泛亞人力銀行調查顯示，有六成八Do you believe the number?Before you believe in the observational data, you should Examine the numbers with your personal experiencesExamine the numbers with the governme

28、nt dataExamine the numbers with other studiesMore than half times these numbers are incorrect, so betting on disbelieving these numbers is usually smarter隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果28Do you believe the number?隨機控制What is the problem?Provide basic survey statistics: sample, response rate, sample size, c

29、onfidence intervalThe majority of the studies have the response rate is lower than 70%For confidential questions, the response rate could be even lower than 30%隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果29What is the problem?隨機控制實驗了解接受Where does the bias come from?Those who have found the job is unlikely to respond to

30、 the emailTherefore, the majority of them should be the ones who cannot find the jobsIf we do the reweighing this factor, then the unemployment rate would drop to less than 15%隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果30Where does the bias come from?What is the correct estimates隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果31What is the c

31、orrect estimates隨Example IINHIS asks about the health status (1 best and 5 worst) of individuals admitted and not admitted for hospitals Hospital makes you sicker!隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果32Example IINHIS asks about the How to solve the problem?Quasi-random assignmentInstrumental variable methodPrope

32、nsity score matching methodRegression隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果33How to solve the problem?QuasiQuasi-ExperimentsEconomists typically cannot set up randomized trials for many public policy discussions. Yet, the time-series and cross-sectional approaches are often unsatisfactory.Quasi-experiments are ch

33、anges in the economic environment that create roughly identical treatment and control groups for studying the effect of that environmental change.This allows researchers to take advantage of randomization created by external forces.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果34Quasi-ExperimentsEconomists tyQuasi-Experi

34、mentsBasic approach is to let outside forces do the randomization for us. In some cases, the situation happens naturally.Suppose, for example, that Arkansas cut its TANF benefit by 20% in 1997, and that we had a large sample of single mothers in Arkansas in 1996 and 1998.At the same time, imagine th

35、at Louisianas benefits remained unchanged.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果35Quasi-ExperimentsBasic approacQuasi-ExperimentsIn principle, the alteration in the states policies has essentially performed our randomization for us.The women in Arkansas who experienced the decrease in benefits are the treatment g

36、roup.The women in Louisiana whose benefits were unchanged are the control.By computing the change in labor supply across these groups, and then examining the difference between treatment (Arkansas) and control (Louisiana), we can obtain an estimate of the impact of benefits on labor supply that is f

37、ree from bias.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果36Quasi-ExperimentsIn principle,Quasi-ExperimentsImagine we simply studied single mothers in Arkansas alone.Arkansas has essentially performed an “experiment” where single mothers in 1996 are the control group, and those in 1998 are the treatment group.In practi

38、ce, this comparison runs into the criticisms that confront us with time series analysis.For example, the national economy was growing exceptionally fast during this period.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果37Quasi-ExperimentsImagine we siQuasi-ExperimentsBecause of these concerns about national trends, the qu

39、asi-experimental approach includes the extra step of comparing the treatment group for whom the policy changed to a control group for whom it did not.Single mothers in Louisiana did not experience the TANF cut, yet benefit from the growth in the economy.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果38Quasi-ExperimentsBec

40、ause of thQuasi-ExperimentsThat is, by examining hours of work in Arkansas, we obtain:HOURSAR,1998-HOURSAR,1996This contains both the treatment effect and the bias from the economic boom.In contrast, by examining hours of work in Louisiana, we obtain:HOURSLA,1998-HOURSLA,1996This contains only the e

41、ffect of the economic boom.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果39Quasi-ExperimentsThat is, by eQuasi-ExperimentsBy subtracting the change in hours of work in Louisiana from that in Arkansas, we control for the bias caused by the economic boom.We obtain a causal estimate of the effect of TANF benefits on hours o

42、f work.An example is given in Table 1, first focusing on Arkansas alone.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果40Quasi-ExperimentsBy subtractinTable 1Using Quasi-Experimental VariationArkansas19961998DifferenceBenefit Guarantee$5,000$4,000-$1,000Hours of Work Per Year1,0001,200200CPS data shows actual hours of wor

43、k for single mothers in Arkansas.At the same time benefits were being cut, hours of work increased for single mothers in Arkansas.Benefits fell by $1,000 in Arkansas during 1997.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果41Table 1Using Quasi-ExperimentaQuasi-ExperimentsWhile benefits fell by 20%, hours of work increas

44、ed by 20%; the implied elasticity of labor supply with respect to benefits levels is -1.This is larger than the -0.67 elasticity estimate found in the randomized trial in California.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果42Quasi-ExperimentsWhile benefitQuasi-ExperimentsThere is likely to be bias in this “first-dif

45、ference,” because there was major economic growth during this period.Thus, single mothers in Arkansas may have increased their work effort even if TANF benefits had not fallen.We examine single mothers in the neighboring state of Louisiana, in the bottom panel of Table 1.隨機控制實驗了解接受遠端祈禱心臟病病人的治療效果43Qu

46、asi-ExperimentsThere is likeTable 1Using Quasi-Experimental VariationArkansas19961998DifferenceBenefit Guarantee$5,000$4,000-$1,000Hours of Work Per Year1,0001,200200Louisiana19961998DifferenceBenefit Guarantee$5,000$5,000$0Hours of Work Per Year1,0501,10050We can gather the same kind of data for Louisiana.Benefit