Scientific authors found racial bias based on fictional media usage of term.
Last week here I wrote a post about a “scientific” study of Florida’s Stand-Your-Ground (SYG) law. That paper was published in the Elsevier journal “Social Science & Medicine.”
The paper: “Race, law, and health: Examination of ‘Stand Your Ground’ (SYG) and defendant convictions in Florida” (Social Science & Medicine, Volume 142, October 2015, pages 194-201; pay-walled )
In that paper the authors applied a Public Health Critical Theory Methodology to the question of whether Florida’s SYG law has a quantifiable racial bias. The key finding of the paper is that a “defendant is two times … more likely to be convicted in a case that involves White victims compared to those involving non-White victims” in the context of Florida’s SYG law.
This finding was the result of statistical analysis conducted on a set of 204 purported SYG cases drawn from a total collection of 237 purported SYG cases collected by the Tampa Bay Times, a regional Florida newspaper.
Sadly, when I examined each of the overall dataset of 237 cases individually (you’re welcome) I found that fully 181 of them simply could not be SYG cases as a matter of legal definition. (I won’t repeat here the detailed reasons why they could not be SYG cases; if you’re interested just go read my previous post.)
That left only 56 possible SYG cases in the whole dataset of 237. Even if all 56 of these were included in the authors’ working data set of 204, that means the other 148 (over 70%) of the purported 204 SYG cases from which they drew their key finding were not, as a simple matter of legal definition, SYG cases at all.
It was this observation that led to last week’s post.
Reaching out to the Journal of Social Science & Medicine
In response to my post several people, including some in the post comments, suggested that I contact the publisher of “Social Science & Medicine” and bring my observations to their attention. Perhaps, it was thought, they would wish to retract the paper once they were made aware of the fundamental defect in its methodology.
Sure, why not. On Wednesday, October 21 I sent the editors what was essentially an email version of my Legal Insurrection post on the paper.
Getting a Response from the Journal!
Really, I didn’t expect to get a reply, so you can imagine my surprise when on Monday, October 26 I in fact received one from Ryan Mowat of “Social Science & Medicine.” Ryan kindly thanked me for my comments, and indicated they would look into the matter.
He did remark, interestingly, that they would not be inclined to withdraw the paper for error, writing “[withdrawal] generally only happens to scientific articles if deliberate malpractice has occurred.” I thought that odd, but then my expertise these days is the law, not the world of scientific publishing.
He also asked if I’d be interested in writing a dissenting commentary. Never having done so previously I replied the same day asking him what would be entailed.
Three days later, on October 29, I had not yet heard back from Ryan. I informed him that I was planning to write a follow-up post on the matter, and asked if he had any substantive response to my prior email.
Color Me Flabbergasted
This time Ryan got back to me the same day:
Sorry for the delay in getting back to you, we were waiting on the authors themselves to respond to you concerns. They did so, and I’ve pasted this below. We have done this in the hope that the matter can be resolved between you and the authors amicably. As such, we would encourage you to engage with the authors before taking further steps. If this cannot be resolved then we will still be happy to publish a dissenting commentary from you. This could take a similar form to your original email, we would probably place it within a scientific commentary format.
From Goodman et al.:
“We are clear in the methods section of the paper that we conduct secondary data analysis of a database put together by the Tampa bay Times (TBT). We state in the methods section “The TBT data set includes 237 cases related to Florida’s ‘Stand Your Ground’ (SYG) law from 2005 to 2013. The TBT included cases that either involved a request for a ‘Stand Your Ground’ immunity hearing or ‘cases where circumstances appeared to reflect the Legislature’s intent when it passed the law’ (Tampa Bay Times, 2013)”. We use the term “Stand Your Ground” as it has been used in the media around highly publicized cases (e.g., Trayvon Martin and George Zimmerman) and not the legal definition provided below [by Branca]. We take the SYG terminology from the TBT who created this database of cases and did preliminary reporting on the results of this data.
We exclude cases from our analysis with missing data which would be dropped from regression models; this is to keep the size of our analytic sample consistent across analyses. We also state in the discussion “As with all secondary data analyses, we acknowledge some limitations to the data we used.””
Let me know what you think.
Uh, That’s Not How Convictions Happen
And here’s my reply, sent immediately after receiving the email above:
Thanks for getting back to me, and no worries about the delay, I’m sure we’re all equally busy. And I certainly have no interest in anything other than amicable communications and outcomes. This is a scientific matter to me, not a personal one.
With respect to engaging with the authors it is most efficient for me to simply cc them on this email to you. Or prior correspondence is threaded below this message, if they’d care to review that for background.
On to the substance: I have to say I’m rather flabbergasted by the authors’ response, especially this sentence:
“We use the term “Stand Your Ground” as it has been used in the media around highly publicized cases (e.g., Trayvon Martin and George Zimmerman) and not the legal definition provided below [by Branca].”
I can only interpret this to mean that the authors perceive that there are two distinct and inconsistent definitions of Stand-Your-Ground (SYG): (1) the definition as “used in the media around highly publicized cases” (“media definition”) versus (2) the actual “legal definition.”
The former is what the authors say they studied in their paper. That they were aware of the existence of the latter when writing the paper is evidenced by the fact that they cite the relevant statutes (§776.012 and §776.031) on page 196.
Does it need to be said that only the latter, the actual SYG law, is applied by the criminal justice system? And that the former, the “media definition” of SYG, is not applied by the criminal justice system?
You will recall that the paper’s key finding is that a “defendant is two times more likely to be convicted in a case that involves White victims compared to those involving non-White victims” in the context of Florida’s SYG laws. (Emphasis added.)
Surely it must be self-evident that whatever the impact of SYG on conviction rates, it can only be the “legal definition” as actually applied by the criminal justice system that could possibly have an effect, and that the “media definition” that is not applied by the criminal justice system (because it is not law) can not have had any effect on conviction rates.
The authors also note: ““As with all secondary data analyses, we acknowledge some limitations to the data we used.”
I’m afraid the matter is more serious than the mere use of data with “some limitations.” The authors have studied something (the “media definition” of SYG) that simply plays no role in the adjudication of guilt, and therefore could not possibly have had an impact of any sort on conviction rates.
The authors’ chosen definition of “Stand-Your-Ground” is quite literally legal nonsense, and could not possibly support any scientifically sound findings re: conviction rates.
I certainly don’t attribute this to any intentional malfeasance on the part of the authors. The authors state that they adopted the SYG terminology from the Tampa Bay Times, and I have no reason to believe they were aware of the intrinsic legal error in the TBT’s terminology. Indeed, I find innocent misunderstanding of SYG to be the rule rather than the exception, even among those whose expertise is the law.
I would also suggest, however, that there is no shame in acknowledging plain error. Indeed, in science it is akin to a duty. We’re all human, mistakes are made, we all move on from them.
Before further considering your kind offer to potentially publish a dissenting commentary by me on this matter, I suggest it would be most appropriate to provide the authors with an opportunity to respond to this email, if they care to. If there is some error in my assessment of the matter I’d be more than happy to have that error corrected without causing what would have been shown to be unwarranted embarrassment to the authors.
I expect that if they do wish to respond directly that they will have done so by the end of business on Monday, November 2.
Many thanks again for your thoughtful communications on this matter.
I did, in fact, cc the authors of “Race, law and health” on the above email, and as yet have not received any communications from any of them, nor any additional communications from Ryan.
Science, am I right?
“SYG-Retreat” versus “SYG-Immunity”
It’s been suggested that perhaps the authors use of the “media definition” of “Stand Your Ground” indicates that they were using “Stand-Your-Ground” (incorrectly) to mean “self-defense immunity” (“SYG-immunity”) rather than to mean (correctly) “no duty to retreat” (“SYG-retreat”). This is practice is actually pretty common, if mentally sloppy. The Tampa Bay Times dataset includes cases of both types without clearly distinguishing between the two. This current post and my prior post were both written in the context of “SYG-retreat” (because I actually know what I’m talking about).
That the authors might have intended to be speaking of “SYG-immunity” is an interesting possibility. The authors never really define exactly what they mean by “Stand-Your-Ground,” and in their paper they cite the relevant statutes for both “SYG-immunity” (§776.032) and “SYG-retreat” (§776.012, and elsewhere). I suppose they could have meant either, or a blend of both (which would be silly, as they’re two entirely different legal doctrines with different legal effects).
If, however, application of the Tampa Bay Times dataset was inappropriate for studying rates of conviction in the context of “SYG-retreat” (and it most certainly was), it is even less appropriate in the context of “SYG-immunity.” I’ll explain why in a follow-up post (this post is already long enough).
Attorney Andrew Branca and his firm Law of Self Defense have been providing internationally-recognized expertise in American self-defense law for almost 20 years in the form of blogging, books, live seminars & online training (both accredited for CLE), public speaking engagements, and individualized legal consultation.
“Law of Self Defense, 2nd Ed.” /Seminars / Instructors Course / Seminar Slides / Twitter / Facebook / Youtube
Donations tax deductible
to the full extent allowed by law.