Mustard and Lott (1997) Replication

The content is written for pdf viewer. please refer to the attachment.

L and M replication paper.pdf

0.65MB

 

 

 

 

 

 

 

 

 

Introduction

 

             Nobody would like to be victimized by criminals. Naturally, we are concerned about our security issues to be alive by our instinct as human beings. In this respect, Mustard and Lott’s (1997) study shows that allowing citizens to carry concealed handguns deters violent crimes effectively. However, as the law and the methodology have changed since 1997, this study has become controversial. This is attributed to the fact that the study used county-level data while the law is applied at the state level in reality. This can be a substantial measurement error that falls short of vindicating the research point. Therefore, this paper is designed to redeem the weakness of the Mustard and Lott study.

 

 

 

Background and Economic Theory

 

             “Shall issue” law means that individuals must have a license to carry a concealed handgun, and the granting of such licenses is subject only to meeting specific criteria[1]. Namely, the law made concealed handguns legal. Mustard and Lott investigated the deterrence effect of “shall issue” right-to-carry concealed handgun laws. The methodology used new data, annual cross-sectional time-series county-level crime data of the US from 1977 to 1992.

 

Table  SEQ Table \* ARABIC 1: The Handgun Laws state by state between 1977 and 1992

Already had the laws	Newly adopted laws	Special case
Alabama Connecticut Indiana, New Hampshire North Dakota South Dakota Vermont Washington	Florida (1987) Georgia (1989) Idaho(1990) Maine (1985) Mississippi (1990) Montana (1991) Oregon(1990) Pennsylvania (1989) Virginia (1988) West Virginia (1989)	Pennsylvania

The states in ‘Already had the laws’ indicate the states whose firearm laws existed before 1977, whereas ‘Newly adopted laws’ mean states whose laws are newly applied given years for each. Pennsylvania is an exceptional case in that Philadelphia was exempted from the state law. The rest of the states not listed in this table imply that the laws are not applicable.

---

[1] Shall Issue States 2022, World Population Review, 2022, https://worldpopulationreview.com/state-rankings/shall-issue-states.

---

 

 

 

 

 

             The economic theory of the deterrence effect of the law is simply constructed from the extended concept of opportunity cost. To be more specific, the victim directly encounters the criminal in a circumstance where the victim can do self-defense as a counterattack against firearmed criminals. In this case, the probability of receiving opportunity cost as death to criminals is increased by the victims and police. This concept can be linked to other economic theories. One of the examples is Game Theory. Given two players, the victim(V) and the criminal(C), (V) has two actions between firearms, and none, and (C) has two actions between committing a crime and doing nothing. Even though both players cannot observe the actions of each in advance, (C) knows the “shall issue” law led to an increase in the probability of action firearms by (V) with the lowest payoffs(death) to (C) as an outcome. Naturally, the law deters the action of committing a crime by (C) to avoid the lowest payoffs, death. Furthermore, this concept can also link to another economic concept based on the price of obtaining guns procedure variation by the law and other theories.

             Mustard and Lott argued that previous studies ignored the heterogeneity within

States, and thus, they have biased results. That is, there are unaffected counties by the law that should be excluded to precisely analyze the deterrence effect of the law. This assertion implies that Mustard and Lott suggest a more accurate analysis of the deterrence effect of the law on a county basis with new time-series data, unlike previous papers.

 

 

Data

 

             In this paper, the data set originated from The FBI's Crime Data Explorer (CDE). The data is the number of recorded crimes from 1979 to 2020 by all states and crime types. In order to figure out the effect of the “shall issue” law, this section selects the state which adopted the law in 1989 (Georgia, Pennsylvania, and West Virginia) and 1990 (Idaho, Mississippi, and Oregon) as samples for better understandings to readers with a convenient purpose. Since the original measurement for ‘rape’ was abolished in 2017 and a new measurement for ‘rape’ has been adopted, the data in this paper covers from 1979 to 2016 only for consistency. Highlighted yellow cells indicate the year when the “shall issue” law was adopted. The sample table consists of the sum of three states’ crimes. This section picks homicides as a representative crime type to compare the trend by the states in the Figures. Note that this analysis based on simplified sample data is extended to the entire data in the section from the bacon decomposition.

 

 

 

 

Table 2: Sum of crime types from Georgia, Pennsylvania, and West Virginia

The table shows that the adopted law did not significantly decrease the number of all types of crimes instantly in 1990. Property crime, burglary, and larceny show a slightly decreasing in 1990. Then, consider the plot graph of homicide trends in the separate states.

 

 

 

 

Figure 1: Homicide Trend (law enacted in 1989)

Although the instant effect of the law is not necessarily guaranteed in 1991, the Figure shows the existence of a gradual decrease from 1990 to 2000. 

 

 

 

 

 

Table 3: Sum of crime types from Idaho, Mississippi, and Oregon

In this table, the increasing crimes in 1991 appear after the adopted law in 1990. Like the table above, the instant effect was not necessarily guaranteed.

 

 

 

 

 

Figure 2: Homicide Trend (law enacted in 1990)

The figure shows that the decreasing trend appears after 1994 while increasing from 1990 to 1994.

 

 

Empirical model and Estimation

 

 

This section applies data analysis with regressions in four ways: Two-way fixed effects (TWFE), Bacon decomposition, Callaway and Sant’anna, and Event study.  Note that ‘crime_rate’ is the outcome of regression and is equal to the sum of all crime types divided by population.

 

 

1.       Two-way fixed effects (TWFE)

 

TWFE is applied separately to the group of the law enacted in 1989 and in 1990 as the prime examples. In order to clarify the effect of the law, the regression periods are divided by the year the law was enacted. For example, the first estimate (~1989) from Table 4 indicates the period from 1979 to 1989, while the second estimate (1989~) indicates the period from 1989 to 2016. Both estimates include the year of the enacted law. The detailed information, such as standard errors, corresponds to the new estimates only. Note that ‘motor_vehicle_theft’ and ‘aggravated_assault’ for old estimates are dropped because of multi-collinearity.

 

 

Table 4: TWFE (1989) for Georgia, Pennsylvania, and West Virginia

Term	Estimate (~1989)	Estimate (1989~)	std.error	Statistic	p.value
violent_crime	0.0002231	0.0012206	0.0007228	1.6886581	0.0983611
Homicide	-0.0029738	-0.0095062	0.0016221	-5.8603355	0.0000005
rape_legacy	-0.0001212	-0.0025675	0.0008128	-3.1586808	0.0028633
Robbery	-0.0002060	-0.0004848	0.0007391	-0.6560197	0.5152275
aggravated_assault	N/A	-0.0012449	0.0007273	-1.7117891	0.0939767
property_crime	0.0000296	-0.0000250	0.0000262	-0.9539615	0.3453149
Burglary	0.0000766	0.0000666	0.0000377	1.7638924	0.0846926
Larceny	0.0000541	0.0000793	0.0000310	2.5594881	0.0139957

In the table, all of the observed coefficients of variables are decreased except for larceny after enacting the law in 1989. This implies that ‘crime_rate’ is comparatively low than in the previous period.

 

 

Table 5: TWFE (1990) for Idaho, Mississippi, and Oregon

Term	Estimate (~1990)	Estimate (1990~)	std.error	Statistic	p.value
violent_crime	-0.0000835	0.0011512	0.0008028	1.4339656	0.1583452
Homicide	0.0023399	0.0062640	0.0030716	2.0393563	0.0471830
rape_legacy	0.0017381	0.0002652	0.0009212	0.2878448	0.7747585
Robbery	0.0001856	-0.0013858	0.0007941	-1.7450454	0.0876548
aggravated_assault	N/A	-0.0009862	0.0007902	-1.2481267	0.2183005
property_crime	0.0000047	0.0000133	0.0000239	0.5571714	0.5801115
Burglary	-0.0000125	0.0000134	0.0000353	0.3801357	0.7055943
Larceny	0.0000209	0.0000342	0.0000347	0.9873099	0.3286557

In the table, the positive effect of the law is limited to the crime types of ‘rape_legacy’ and ‘robbery’ only. This result questions the effectiveness of the “shall issue” law.

 

 

2.      Bacon decomposition

 

Unlike the TWFE section above, this section uses the entire data containing all states with the same concept of ‘crime_rate.’ Moreover, the data uses numeric variables of ‘state_name’ and binary treatment variable. The yardstick of the treatment variable is the year when the law was enacted, so that '1' means treated status. For example, enacted year of Georgia, Pennsylvania, and West Virginia is 1989. Then the treatment variable for 1979 to 1988 is ‘0’ whereas the year from 1989 to 2016 is ‘1.’ This treatment variable contains omitted states by Mustard and Lott referenced to the historical facts[2]. Furthermore, it includes the updated fact, such as Illinois adopted the “shall issue” law in 2013. As seen the Table 6 below, the coefficient of treatment variable for TWFE is calculated based on this entire data.

 

 

Table 6: Bacon Decomposition with TWFE = 0.616128

	Type	Weight	Estimate
Earlier vs Later	Treated	0.18044	0.16362
Later vs Always	Treated	0.63998	0.88175
Later vs Earlier	Treated	0.17958	0.12419

 

This bacon decomposition assumes that there is no never-treated group. Instead, there is an always-treated group

 

---

[2] Higgins, P., Owens, S., Woodward, M., & Wayne. (2022, January 29). The ultimate concealed carry guide (2019). GunsToCarry. Retrieved May 26, 2022, from https://www.gunstocarry.com/

---

 

 

 

 

Figure 3: Bacon Decomposition with TWFE = 0.616128

This bacon decomposition shows how much TWFE can be biased by representing the variance weight of the estimate. In this decomposition, the ‘Later vs always treated’ group has the most significant weight. The biggest problem with TWFE is that it uses the treated group as a control group (untreated group), which is the leading cause of bias. Even though this result does not include the case of the sign flipped, considering the fact that TWFE does not have “no sign flip property,” this method can implicitly arise severe misleading results. This problem induces to use of other better estimating methods such as Callaway and Sant’anna. (CS)

 

3.      Callaway and Sant’anna (CS)

 

Using the CS method (‘did’ package in R) estimates the effect on ‘crime_rate.’ The panel time variable is ‘year,’ and the panel id variable is ‘sid’ which is a converted numeric variable from the columns ‘state_name.’ The first treatment period variable is the ‘enacted_year’ for each ‘sid.’ The estimation method is doubly robust and it does not set any covariates. Then the equation for this method would be:

 

 

 

The result from ‘did’ library package in R is below:

 

Call:    att_gt(yname = “crime_rate", tname = "year", idname = "sid", gname = "enacted_year", xformla = NULL, data = crime,            panel = TRUE, control_group = "notyettreated", bstrap = TRUE, biters = 1000, est_method = "dr", print_details = FALSE)   Reference: Callaway, Brantly and Pedro H.C. Sant'Anna.  "Difference-in-Differences with Multiple Time Periods." Journal of Econometrics, Vol. 225, No. 2, pp. 200-230, 2021. <https://doi.org/10.1016/j.jeconom.2020.12.001>, <https://arxiv.org/abs/1803.09015>   Group-Time Average Treatment Effects:
Group Time   ATT(g,t)  Std. Error [95% Simult.  Conf. Band]    1985 1980  -0.5093     0.0892       -1.2381      0.2194    1985 1981  -0.1817     0.0988       -0.9893      0.6258    1985 1982  -0.3257     0.1135       -1.2533      0.6019    1985 1983   0.3515     0.1004       -0.4695      1.1726    1985 1984  -0.0924     0.0544       -0.5368      0.3519    1985 1985  -0.1011     0.1010       -0.9266      0.7244    1985 1986  -1.0315     0.1598       -2.3374      0.2744    1985 1987  -0.9144     0.1792       -2.3796      0.5508    1985 1988  -0.7936     0.2386       -2.7441      1.1569    1985 1989  -0.8279     0.2851       -3.1583      1.5025    1985 1990  -0.9699     0.3023       -3.4406      1.5008    1985 1991  -1.1607     0.2845       -3.4861      1.1646    1985 1992  -1.2145     0.2893       -3.5796      1.1507    1985 1993  -1.7586     0.3056       -4.2569      0.7396    1985 1994  -1.4979     0.3055       -3.9948      0.9989    1985 1995  -1.3384     0.4300       -4.8537      2.1769    1985 1996  -0.5281     0.4840       -4.4847      3.4286    1985 1997  -1.0537     0.5331       -5.4117      3.3044    1985 1998  -0.8650     0.5471       -5.3374      3.6074    1985 1999  -0.4488     0.5564       -4.9972      4.0996    1985 2000  -0.7252     0.5917       -5.5622      4.1118    1985 2001  -1.0922     0.4108       -4.4501      2.2656    1985 2002  -0.9992     0.4115       -4.3632      2.3648    1985 2003  -1.0212     0.5411       -5.4445      3.4020    1985 2004  -0.7788     0.8000       -7.3184      5.7607    1985 2005  -0.5901     0.6600       -5.9852      4.8050  1985 2006   0.6032     0.5702       -4.0576      5.2639	1985 2007   0.5616     0.5873       -4.2392      5.3624    1985 2008   0.8127     0.4899       -3.1916      4.8169    1985 2009   1.0345     0.5441       -3.4127      5.4818    1985 2010   1.3689     0.5239       -2.9137      5.6515    1985 2011   2.6877         NA              NA             NA    1985 2012   2.8231         NA              NA             NA    1987 1980   0.8415     0.0850        0.1469      1.5361 *   1987 1981  -0.6885     0.0982       -1.4912      0.1141    1987 1982  -0.7053     0.1089       -1.5957      0.1850    1987 1983  -0.6931     0.0963       -1.4799      0.0937    1987 1984   0.3083     0.0530       -0.1245      0.7411    1987 1985   1.1546     0.0929        0.3948      1.9144 *   1987 1986   0.8264     0.0763        0.2023      1.4505 *   1987 1987   0.4586     0.0974       -0.3380      1.2552    1987 1988   1.3575     0.1525        0.1108      2.6042 *   1987 1989   1.0657     0.2036       -0.5983      2.7297    1987 1990   0.6753     0.2035       -0.9885      2.3392    1987 1991  -0.1294     0.2232       -1.9540      1.6951    1987 1992  -0.0728     0.2296       -1.9499      1.8043  1987 1993   0.1079     0.2556       -1.9815      2.1973 1987 1994  -0.1012     0.3007       -2.5596      2.3571    1987 1995  -1.2552     0.3157       -3.8355      1.3251    1987 1996  -1.1917     0.3702       -4.2178      1.8344    1987 1997  -1.6436     0.4145       -5.0317      1.7444    1987 1998  -2.0445     0.4495       -5.7190      1.6299    1987 1999  -2.6576     0.4517       -6.3504      1.0351    1987 2000  -3.4454     0.5021       -7.5496      0.6588    1987 2001  -4.1196     0.4085       -7.4591     -0.7800
1987 2002  -4.2509     0.4119       -7.6177     -0.8842 *   1987 2003  -4.6762     0.4486       -8.3435     -1.0089 *   1987 2004  -4.8758     0.5626       -9.4748     -0.2768 *   1987 2005  -5.0607     0.5600       -9.6382     -0.4831 *   1987 2006  -4.3619     0.6509       -9.6829      0.9591    1987 2007  -3.9947     0.6681       -9.4557      1.4664    1987 2008  -3.7394     0.5706       -8.4039      0.9250    1987 2009  -4.1534     0.6248       -9.2609      0.9541    1987 2010  -4.6918     0.6047       -9.6347      0.2510    1987 2011  -3.4688         NA             NA              NA    1987 2012  -3.8046         NA             NA              NA  1989 1980  -0.1964     0.0964       -0.9844      0.5915    1989 1981   0.1092     0.1148       -0.8295      1.0478    1989 1982  -0.0545     0.2038       -1.7207      1.6117    1989 1983  -0.0095     0.3673       -3.0117      2.9926    1989 1984   0.0746     0.0860       -0.6280      0.7772    1989 1985  -0.0577     0.4527       -3.7582      3.6427    1989 1986  -0.2153     0.2102       -1.9336      1.5031    1989 1987   0.1008     0.2652       -2.0669      2.2685    1989 1988   0.4764     0.3318       -2.2355      3.1883    1989 1989   0.6179     0.4187       -2.8051      4.0409    1989 1990   0.1800     0.1474       -1.0248      1.3848    1989 1991  -0.0964     0.1934       -1.6775      1.4846    1989 1992   0.1331     0.2252       -1.7080      1.9742    1989 1993   0.0543     0.2991       -2.3904      2.4989    1989 1994   0.0568     0.4143       -3.3299      3.4436    1989 1995  -0.1137     0.3754       -3.1828      2.9553    1989 1996   0.7159     0.3118       -1.8329      3.2646    1989 1997   0.2770     0.5659       -4.3493      4.9033    1989 1998   0.3749     0.7612       -5.8478      6.5975    1989 1999   0.9227     0.9867       -7.1426      8.9880    1989 2000   0.7340     1.1492       -8.6597     10.1277    1989 2001   0.2580     1.2165       -9.6862     10.2022    1989 2002   0.2377     1.2629      -10.0860     10.5615    1989 2003   0.4958     1.1081       -8.5626      9.5541    1989 2004   0.9382     1.2893       -9.6010     11.4773  1989 2005   1.1009     1.3441       -9.8866     12.0884    1989 2006   1.5398     1.5182      -10.8706     13.9502    1989 2007   1.5537     1.5413      -11.0454     14.1529    1989 2008   1.8862     1.3969       -9.5327     13.3052 1989 2009   1.8250     1.5283      -10.6682     14.3183    1989 2010   1.7298     1.4668      -10.2602     13.7197    1989 2011   3.2094     2.1232      -14.1466     20.5655    1989 2012   3.4390     2.7235      -18.8242     25.7021    1990 1980   0.2891     0.1412       -0.8653      1.4436    1990 1981   0.2419     0.3135       -2.3208      2.8047    1990 1982  -0.1505     0.3302       -2.8495      2.5485	1990 1983   0.1090     0.1265       -0.9252      1.1432 1990 1984  -0.0166     0.1078       -0.8975      0.8644    1990 1985   0.2545     0.1959       -1.3467      1.8557    1990 1986  -0.0259     0.1728       -1.4381      1.3863    1990 1987  -0.1527     0.1411       -1.3058      1.0005    1990 1988   0.0785     0.1970       -1.5319      1.6888    1990 1989  -0.8628     0.5560       -5.4081      3.6825  1990 1990  -0.3260     0.5255       -4.6213      3.9693    1990 1991  -0.1517     0.6534       -5.4928      5.1894    1990 1992   0.2348     0.6207       -4.8387      5.3082    1990 1993   0.3828     0.7714       -5.9230      6.6887    1990 1994   1.2799     0.8073       -5.3189      7.8787    1990 1995   1.2980     0.3906       -1.8951      4.4911    1990 1996   1.0393     0.6805       -4.5234      6.6021    1990 1997   1.2340     0.6768       -4.2986      6.7666    1990 1998   0.8847     0.8161       -5.7866      7.5559    1990 1999   0.7507     1.1217       -8.4181      9.9196  1990 2000   0.7269     1.0276       -7.6735      9.1273    1990 2001   0.6365     1.0864       -8.2440      9.5170    1990 2002   0.7011     1.1369       -8.5925      9.9947  1990 2003   0.7914     0.9517       -6.9877      8.5705    1990 2004   0.7346     0.9682       -7.1795      8.6487    1990 2005   0.5339     0.9715       -7.4077      8.4756    1990 2006   0.4387     1.3824      -10.8617     11.7391    1990 2007   0.3328     1.4352      -11.3992     12.0648    1990 2008   0.1125     1.2403      -10.0263     10.2513    1990 2009   0.2215     1.3487      -10.8034     11.2463    1990 2010   0.3997     1.3357      -10.5186     11.3181    1990 2011   1.9768     2.3983      -17.6276     21.5813    1990 2012   2.0459     2.0911      -15.0475     19.1393    1991 1980   0.5315     0.0910       -0.2127      1.2758    1991 1981   0.0636     0.0995       -0.7496      0.8769    1991 1982  -0.9502     0.1047       -1.8061     -0.0944 *   1991 1983   1.3638     0.0904        0.6249      2.1028 *   1991 1984   0.2425     0.0538       -0.1971      0.6821    1991 1985  -0.6173     0.1020       -1.4508      0.2163    1991 1986  -0.6726     0.0794       -1.3221     -0.0232 *   1991 1987   0.1554     0.1023       -0.6809      0.9917    1991 1988  -0.6574     0.0868       -1.3673      0.0525    1991 1989  -0.6492     0.1184       -1.6170      0.3187   [ reached 'max' / getOption("max.print") -- omitted 287 rows ]   Signif. codes: `*' confidence band does not cover 0 Control Group:  Not Yet Treated,  Anticipation Periods:  0 Estimation Method:  Doubly Robust

This result allows finding overall ATT. The ATT in this section cannot be a group-level ATT because there are not enough states per treatment year for the bootstrapping to provide accurate 95% confidence intervals. Hence, note that this ATT is simple aggregate ATT with a uniformly weighted average.

 

 

 

 

 

 

 

Table 7: ATT comparison with covariate or not

NO covariate	Covariate with population for each state
ATT        Std. Error       [ 95%  Conf. Int.]   0.0688        0.3412         -0.5999      0.7375	ATT         Std. Error     [ 95%  Conf. Int.]  -0.0225        0.2903       -0.5915      0.5464

 

The left result shows that the “shall issue” law could cause a 6.9% increase in the crime rate. The right result is the ATT with covariate ‘population’ of each state. This suggests another research question as to whether the size of the population for each state can be influential on the overall crime rate or not. According to the ATT result, depending on each state’s population, the enacted law can indirectly cause a -2.3% decrease in the crime rate. In light of these results, the right outcome is a comparatively more appropriate model to estimate the effect of the law because of its less standard error.

 

 

 

4.      Event study (SA)

 

The event study is called Sun and Abraham (SA) method, and its estimator is called the interaction weighted estimator. This estimator is compared with any never-treated groups and the last cohort treated.  The method limits the data to use the last cohort and never treated units as controls. Namely, all periods when every state is treated are dropped, unlike TWFE, so that the method can overcome the major bias factor of TWFE. Note that the consistently estimated shares weight coefficients.

 

Figure 4: Comparison between TWFE and SA with crime types

As seen from Figure 4 above, the pre-trend of TWFE is nearby zero and then gradually closed to the negative estimate over time after the treatment. This violates the parallel trend assumption so that TWFE is not a consistent estimate. In contrast, the pre-trend of SA is primarily negative but has an increasing tendency over time despite event time (year = 0, enacted year) passed. This indicates SA estimates satisfy the parallel trend to some extent.

 

Overall, ATT of SA indicates the “shall issue” policy causes a 44% increase in the crime rate. However, when all crime types are included in the control groups, This declines to a 30.12% increase in the crime rate. Specifically, Violent crime, homicide, and property crime are restrained to less than -0.001 (-0.1%) estimate. Crime types with positive coefficients, such as rape legacy, robbery, aggravated assault, burglary, larceny, and motor vehicle theft, are not that significant because all of them are restricted to less than 0.0012 (0.12%) estimate.

 

 

 

 

 

 

 

Conclusion

 

 

Table 8: How much does the "shall issue" law influence the crime rate in the U.S

Methodology	TWFE	CA	CA (with covariate)	SA	SA (with crime types)
ATT	0.616128 (62%)	0.0688 (6.9%)	-0.0225 (-2.3%)	0.439 (44%)	0.298576 (30.12%)

 

This table summarizes the conclusion of the procedure of this paper. First, the TWFE method is comparatively not reliable for this paper due to the violation of the parallel trend assumption demonstrated in Figure 4. The problem is that CA and SA are significantly different from each other. However, the SA method is seemingly more reliable than CA in light of the fact that SA is constructed method to overcome the problematic bias that the traditionally preferred method TWFE has. Therefore, the conclusion is that the “shall issue” policy can contribute to an increase in the crime rate in the U.S by 30% ~ 44% despite individual crime types constrained to some extent. This is presumably because the license for firearms led people to abuse the right to take the law into their own hands.