Fact checking: How to deal with repeat offenders?
I haven't followed the fact-checking industry closely for at least a year. In that time, Fact Checker at WaPo has begun running an article series called "Recidivism Watch", where they cover a false claim that a politician made and then repeated. Here's an example where Rand Paul tries again to convince the public that the Clean Water Act makes dirt a pollutant and your backyard a river, which caused some poor unsuspecting old man to go to prison for ten years.
Recidivism presents a conceptual and practical challenge to the aggregate measurement of a person's soundness. Do you count the falsehood twice against the person who uttered it? If you do, should you provide an additional penalty for repeating the falsehood? What about the repetition of the truth; do we award a bonus for that?
Indeed, Fact Checker doesn't appear to count statements twice in its report cards anymore (if it ever did), focusing on recidivism watchdoggery instead. In contrast, it appears that PolitiFact does count statements twice at least sometimes. This discrepancy will influence the validity of the aggregated fact-check scores I'll put out this election season since these aggregates are based on Fact Checker and PolitiFact report cards. Not only will this phenomenon present a challenge to soundness measurement. It will also affect the width of the uncertainty bands around those averages, which are largely a function of sample size.
Also interesting is the difference in how Fact Checker covers recidivism in this special series versus PolitiFact's coverage of repeated claims, at least in one case. For example, in its coverage of Trump's repeated false claim that most Syrian refugees are men, PolitiFact didn't call it out as recidivism until the fifth paragraph.
Both fact checkers, however, emphasize the repetition of false claims not true ones. That's interesting when you consider the advice of the Debunking Handbook, which is to repeat a false claim as seldom as possible so that people are more likely to remember the truth. The lesson here is that the more you cover "recidivism", the more careful you have to be in the messaging.