Biased, wish-fulfilling, partisan, badly supported news stories abound. (I'm going to try to avoid the f–e n–s term, even ironically.) Has it always been thus? Maybe. But it's never too late to try to improve yourself. Moreover, with a highly controversial and divisive President, such stories will probably multiply. We'll be faced, daily, with more news that tells us what we want or expect to hear, that endorses our dim view of the political and social figures we don't like.
Critical reading is essential. Skepticism of even one's favored sources is important, unless we're looking only to be entertained and affirmed. This is an hourly habit, not an occasional one. It's a task I fail daily and will probably keep failing daily even if I try harder. But maybe I could fail a little less often.
Recently it hit me: what if I reviewed news stories with the skeptical eye I turn towards search warrant applications?
If you're not familiar with them, search warrant applications include a declaration under penalty of perjury from the investigating officer or agent. The declaration and supporting paperwork are supposed to identify the location to be searched, the items to be seized, and the specific facts providing probable cause that those items are evidence of a crime. Federal courts scrutinize search warrants more closely than state courts. That's not the law; that's just reality.
When I was a prosecutor, my job was to review proposed warrant applications from federal agents and make sure that they complied with legal requirements before submitting them for approval to federal magistrate judges. As a criminal defense attorney, my job is to analyze warrant applications that have yielded searches of my clients and scrutinize them for flaws and constitutional failures that I can present carefully and forthrightly to a judge so that the judge can then ignore or rationalize them. The critical eye that prosecutors and judges are supposed to use when reviewing a warrant application — and that defense lawyers use in evaluating whether they can be challenged — comes in handy in assessing the trustworthiness of news. Three doctrines in particular come to mind.
Attribution: Around the time I became a federal prosecutor, thanks to a series of unfavorable Ninth Circuit decisions (which, naturally, I resented at the time as unfairly anti-government), the U.S. Attorney's Office began emphasizing attribution in reviewing search warrant applications and prosecutor training. Put simply, attribution means this: for each fact asserted in the warrant application, how does the affiant know it? if the affiant learned the fact from someone else, how did that person know it?
A good search warrant establishes clean attribution for each fact, even if that attribution involves second, third, or fourth-hand knowledge. For example, a good search warrant would say something like this: "I spoke with Officer Jones of my department on January 15th, 2017. Officer Jones told me the following: she interviewed Mary Smith earlier that day. Smith stated that she was present at the corner of Elm and Oak and saw the car accident. Smith told Officer Jones that she was walking north on Oak when she saw a red SUV travelling at a high rate of speed run a stop sign and crash into the side of a green sedan." A well-drafted affidavit also identifies its factual inferences and its basis for them. "I obtained electricity usage records with an administrative subpoena to Southern California Edison for the subject address. I noted that, starting the month that suspect ROBERTS occupied the residence, energy usage spiked 350%, to a level that was consistently more than three times what the energy usage had been for the same time of year over the last five years at the residence. In my training and experience, I know that indoor marijuana grows often result in substantial spikes in energy usage because of the lights and other equipment used"
Thanks to thorough attribution, the reader knows the ultimate source of the fact and the ultimate source's basis for asserting the fact. A bad search warrant application, by contrast, makes assertions about what happened without any indication of how the affiant knows those facts.
A well-attributed news story might be less stilted. But it would still make clear the basis for the facts asserted in the story. Partial or unclear attribution obscures this. Take yesterday's extremely popular New York Times story about Rick Perry's gig as Energy Secretary. I certainly wanted to believe it. I deplore Donald Trump and, to a lesser extent (mostly thanks to his criminal justice stance) Rick Perry. The slams on Perry were artful and viscerally satisfying. The picture it painted confirmed what I wanted to believe about the administration. But notice how the story's main assertion — that Perry thought he was signing up to lead energy industry policy, when in reality his job would be primarily about nuclear security — comes in the first three paragraphs without any attribution. The fourth paragraph has a quote from a (former) insider, but the paragraphs are structured so it's impossible to determine if that source told the Times what's in the previous three paragraphs, or if he endorses that content (he says he doesn't), or whether he's simply provided a pull quote that the Times can present as consistent with their theme. Is the point of the story the Times' characterization or interpretation of facts, or is it based on something that a source specifically told the Times? If it came from the source, was it all based on direct knowledge or based on the source's own gloss? (Notice how the source switches from "I asked him" to describe one sentiment and the vague and unattributed "now he would say" for the second). We're left to guess.
Particularity: My debut as a prosecutor also coincided with a Ninth Circuit push for more particularity in warrants. That is, the Court pushed back against the habit of general warrants that sought permission to seize whatever the investigating agents felt like seizing.1 Instead, the Court demanded that warrant affidavits not only specify with reasonable particularity what is to be seized, but support the proposition that each thing to be seized is somehow evidence of a crime. "There are things that are evidence of a crime, some of those things are in this house, therefore all things in this house should be seized" doesn't cut it.
Particularly is useful in evaluating news stories too. If a story attributes a stance, or a goal, or a motive to a public figure, does it give specific examples of conduct consistent with stance? If the story offers examples of conduct — specific facts — does it connect them to the thesis of the article? Does it show how those specific examples actually support its thesis, or does it simply regurgitate them and rely on proximity to persuade the reader to assume they are connected? So, for instance, the New York Times' Rick Perry story has a number of paragraphs questioning Perry's qualifications, comparing the better qualifications of a prior Energy Secretary, and discussing Trump's likely energy policy. Are those paragraphs proof of the article's thesis? Does Perry's lack of qualification — if that's what it is — support the thesis that he thought he was going to be controlling energy use policy instead of nuclear security?
Corroboration: Anonymous or obscure sources are not inherently impermissible in search warrants or in journalism. A search warrant may rely in part on an anonymous source if the affiant corroborates that source — that is, offers other facts supporting what the source says. In theory a warrant application should corroborate facts only an insider could know. "My source told me that methamphetamine is being cooked at a green house at 123 Elm. I traveled to 123 Elm and observed that the house is, in fact, green" is not meaningful corroboration. "My source told me that suspect ROBERT is cooking methamphetamine at 123 Elm, that he began cooking in March 2016, and that he had precursor chemicals delivered there beginning in April. Based on my review of the Southern California Edison records described above, I noted that there was a 300% spike in energy usage at 123 Elm beginning in March 2016. My review of the UPS records described in paragraph 17 above showed a series of deliveries from an online chemical supply company beginning in April of 2016" is good corroboration.
I can't critique the New York Times Perry story on source corroboration because it's not clear what parts of it come from sources, anonymous or otherwise. But it's now routine for the media to offer sources — anonymous and named — with no corroboration and very little indication of the source's basis for knowledge (which is also an attribution problem). I recognize that journalists have an interest in protecting their sources, but that protection has a cost, and that cost ought to include a higher level of skepticism with readers. A reliable story based on an anonymous source would corroborate elements of the source's story in a meaningful way for the reader. Otherwise it's just the reporter's appeal to his or her own authority — I trust this person so you should as well — and that's no different than an agent's "trust my skeevy anonymous informer because I'm a cop so you can trust me."
If you're reading this to suggest that I think one "team" or another is more guilty of this or more or less credible, you're reading it wrong. Skepticism and critical reading are good. The fact that we'll certainly fall short is not a reason not to try. And gosh, what if a habit of critical reading of the news could even translate to critical evaluation of law enforcement claims? Nah. One improbable goal at a time.
Edited to add: I missed that Jesse Singal already made the same point about attribution.
- Granted, that's what they're going to do anyway no matter what's in the warrant. But having the paperwork tidy is nice. ▲
Last 5 posts by Ken White
- The Proud Boy And The Sockpuppet - July 21st, 2017
- Ted Rall Is Incensed That Anti-SLAPP Laws Protect Everyone - July 18th, 2017
- The Popehat Signal: Anti-SLAPP Help Needed in California - July 14th, 2017
- Texas Attorney Jason L. Van Dyke: Fraudulent Buffoon, Violence-Threatening Online-Tough-Guy, Vexatious Litigant, Proud Bigot, And All Around Human Dumpster Fire - July 9th, 2017
- CNN, Doxing, And A Few Ways In Which We Are Full of Shit As A Political Culture - July 5th, 2017
I'm honestly not sure if it would be worse if Perry didn't know what the Department of Energy did when he took the job, or that he knew and ran for President with a platform that included eliminating the thing. Both could be true. Or neither. I guess he could have figured it out in the interim.
I've found that Carl Sagan's Baloney Detection Kit has served me pretty well in this same regard.
Perceptional reality is that which we see, hear, etc., with our own senses. Representational reality is that which we hear about on the news. Provisional perceptional reality is that which we've heard about on the news so often and consistently, that we have come to accept it as true. Mark Twain: "It isn't what you don't know that can hurt you, it's what you know for sure that just ain't so." Govern yourselves accordingly.
I wonder what the results would be if someone took the time to apply this to a sample of articles from several different outlets.
I'm thinking your target shouldn't actually be the news outlet consumer but instead the editors and writers. Because I'm sure the result of following these rules as a reader today would be that you couldn't trust a single thing printed in any paper.
It makes me sad (though not surprised) that a prominent national newspaper would be guilty of poor attribution, and at least unclear particularity and corroboration. For the most part, the same qualities that make a good search warrant are also the qualities that are taught as good journalism. If a paper that takes journalism seriously fails to meet these standards in a story on an issue of national importance, what hope is there? Where can one turn for a 'well-reported' story?
As to Ken's point of trying to critically read 'news' articles more frequently, this is certainly a good idea, but I do not think it is enough. One should also make a note of which 'news' source do a good job of attribution, particularity, and corroboration, and avoid those that do not. Continuing to reward poor journalism with more clicks just makes the problem worse. Not to mention that even if one realizes one has just read an article that is poorly attributed, one's thinking has probably been influenced anyway. Was Ken's thinking about Rick Perry changed, or at least re-enforced, by reading that article, even though he realized it was poorly attributed? Probably, although admittedly it was also likely less so than if he did NOT critically analyze the article.
And yes, I realize that the first four sentences of my comment make the suggestion in the second half of my comment impractical. If there are no more sources that consistently maintain proper corroboration, particularity, and attribution standards, what should we do – stop reading the news and become ignorant? That isn't a good solution either, so I don't know what the answer is.
[edit] Yes. What joshuaism said. His comment went up while I was writing mine.
How long until there is an article out calling Ken "Fake News" for daring to question or read skeptically?
Ignorance is strength.
What if somebody made a genius.com-like annotation site that breaks down articles and scores them on a consistent objective standard? One which tracks standards versions as well so that when they're updated older articles still display the standards by which they were measured. What if outlets had their quality tracked over time in this manner?
Ken wrote (emphasis mine):
I have never heard of a "meth cooking" process which requires such extremely high electricity or heating gas use in typical criminal meth cooking operations.
I think observation of increased energy use is irrelevant to any inference about manufacturing methamphetamine (or many other compounds) in quantities less than hundreds of pounds or tons. But such large quantities would have far more readily observable indicia than high utility consumption.
It may be relevant to an illegal cannabis (or any other plant, legal or illegal) indoor growing operation, but not to meth manufacture in typical criminal operation quantities.
IMHO.
And 123 Elm is an empty lot!
You've really hit a new high mark with the quality of this article. I'm definitely keeping it bookmarked!
I think arity has a good idea – a website that keeps a score on various media outlets. A few minutes with Google did not turn up anything like a 'politifact' for journalism, but I did not conduct an exhaustive search. If anyone knows of such a site, please post the link.
I think it would take a huge amount of time to cover enough outlets to be useful, so it would probably have to be a collaborative effort. Maybe some journalism professor who reads Popehat can make it a student project to get it started.
Sadly ketchup it'd be a full time job for MANY people who'd have to be reasonably well trained in order to make it through even a fraction of the news articles from mid-sized local to big national places. And do you include blog posts as well?
Thank you for a well-written article, Ken. I encourage people here to read the comments on Jesse Singal's article to observe the effect that journalistic skepticism has on his readership.
I rent the house at 123 Elm, and I hate all you guys. Search engines still pair up my resume with Freddy Kruger,. SWAT visits the neighborhood often enough that I'm on a first-name basis with all of their commanders. Now I have this detective who is following me around like a creeper making snide comments about substantial sentence enhancement for making meth in a school zone.
Give it a rest. Try 1600 Pennsylvania Ave for the next fake address, OK.
Well done, Ken.
SMDH…
Ken, you don't even have a degree in journalism from Columbia! What could you possibly know?
Other good practice is reading popular journalism on science. My first rule of reading articles about scientific topics is "everything it says is at best maybe possible, usually the actual opposite of what was stated". My other rule about reading news of any kind is that absolutely anything that is sensational is Not Accurate(TM). Probably much closer to Completely Wrong(TM).
I do enjoy challenging my biases–not so much by going to sources that are actively in opposition to them, because that makes me want to dig in my heels, but, at least in part, by reading the sources I already enjoy and then trying to pick apart everything that could be wrong (result: usually like 97%). I find that, at least for me, that reinforces the habit of scrutiny (rather than untrained mistrust) and I begin to simultaneously enjoy and resent my favorite places. Kind of like watching any Twilight Zone episode yet again, where I love the show but somehow roll my eyes at every single episode. Except The Hunt. That one never gets old.
kmc: I second your comment about science as a topic of journalistic malpractice. I further move to amend your comment to include the law as well as any other subject that takes a degree beyond bachelors as a requirement for one's career. It's especially horrendous in local news where they don't know the difference between an affidavit and an indictment. And the only place where the medical field is covered adequately seems to be the sports page.
I *WISH* judges read search warrant applications like this. But I recall a discussion between Radley Balko and Orin Kerr…
In order to meet Times standards, there would ordinarily have to have been at least two sources saying that Perry did not initially know what the DOE was before the claim could make it into print. In this case, my guess would be that three or more people said the same thing independently of one another, because otherwise the Times almost certainly wouldn't have led with something that a source told them when speaking on background and not for attribution.
The story reads to me as if they spoke to multiple sources, about half of whom were pro-Perry, and most of whom were speaking on background, in one form or another. However, there's no way of knowing that. Sometimes the Times doesn't meaningfully adhere to its own standards, a famous example of which is the Judith Miller/WMD story, which rode entirely on two anonymous sources, one of whom was later revealed to have been merely repeating what he'd heard from the other. You're basically playing the odds by assuming that they were behaving the way they purport to.
The above is only the word of a single anonymous source, and I advise you to therefore regard it as being of uncertain reliability.
In all events, you're completely right that they're asking you to take it on trust that they did a reputable, responsible job. I also agree (as I think it's likely that Coral Davenport and David Sanger would do as well) that anonymously sourced articles should be read with a higher degree of skepticism.
to arity's idea
A) genius.com already released an add-on you can use to comment on anything, just to let you know
B) it would get biased pretty quick though, you know? Is what the Times did here fearless reporting, or total bull? And this is a pretty clear-cut case. I'm hearing now that, for example, Monica Crowley might not be such a plagiarist, but what I'm saying is (convincing, mind you) opinion to that effect. How would you rate the makers of those claims in light of this? Although since CNN made those claims, it's not like they could go any lower, but you get the idea :)
Ken, this is an excellent approach and it's very similar to my own. I would also note that one thing that people who put out disinformation like to do is to base something on an anonymous source, then create a web of articles that reference each other and point to the sheer number of articles to get people to assume that there must be a there in there somewhere.
This really makes tracing the chain of evidence, as it were, hard at times, because it's often layered deep. But it also provides a sort of smell that can help you identify likely BS.
I've also adopted the exact same tactic when arguing things. Instead of saying this article says X, I will instead identify the primary sources and cite them directly. If it's a quote, I link you straight to the source.
Apply these criteria to the liberal canards (OK, I'm being judgmental) that Trump is misogynist, homophobic, and against immigration.
Why would it be bad if he knew what it currently does and wanted to close it? It's not as if there's any reason why it currently manages the nuclear arsenal. It only took that function over in the first place because it was looking for something to do.
It's not as if Perry came up with the idea of shutting the department down on his own. It's been a standard conservative policy for decades. and for that entire time that policy has included transferring control of the nuclear weapons stockpile to the Pentagon.
@kmc: Yup, reading journalism on science is a great way to remind you of Gell-Mann-Amnesia. If you notice that articles about topics you know are mostly nonsense, you should assume the same about topics you don't know…
This is an excellent argument. I do wonder, though, how much longer attribution will even be a criterion in journalistic standards…with news article bylines being replace by journalists's blog posts, with so much media being "cut and pasted" from other media, and what news items remain relying so heavily on tweets and other social media verbalizings as primary source material, hasn't the cult of personality already overtaken source vetting and presumptions of authority? If any source and any utterance will do, is there any point to having attribution?
Where does bias fit in with these attributes? Bias (implicit or explicit) would seem to encourage news authors to make the facts fit the conclusions.
I have noticed that alt – right sympathetic news sites, such as Breitbart and Instapundit, are not much better at attribution as they are essentially news aggregators aiming to incite indignation among their readers. However, they are factually correct more often because they are responding to mainstream news articles that fail the giggle test and therefore are easily debunked. Ironically (or not), the NYT is one of the easier targets because of the blatent contradictions routinely imbedded in their deliberately obfuscating long form "journalism".