Seeking Truth in a Time of Misinformation

Online abuse of trust is one of the topics I’ve studied deeply, including significant work toward an unfinished PhD, and over two years working on a spam and abuse team at Google. These days my main work is on 2D graphics, but seeing patterns of misinformation about Covid-19 is bringing back interest and motivation to think about these topics.

As Covid-19 has been unfolding, I’ve spent significant time and energy trying to understand it, as I tend to do with threats in general. In so doing, I’ve developed some insight on how to tune media consumption patterns towards more truthful content and less misinformation.

America is broken

The astonishingly poor national response to Covid-19 is heartbreaking. Given our incredible richness of resources, it is surprising that many of the failure modes resemble those from low-income countries: superstition, cult of personality, suppression of science, and straight up corruption. A common thread is misinformation and disinformation.

Our response to Covid-19 has been very much like driving a car with poorly-maintained brakes. We’ve done the equivalent of neglecting regular maintenance, ignoring the brake light on the dashboard, and choosing to not to pay attention to the squishy feeling of the brake pedal. It’s as if there’s a devil on our shoulder saying, “oh no, don’t get the brakes fixed, it might cost hundreds of dollars.”

I think it’s worth trying to understand why we’re in that position, but it’s beyond the scope of this blog post. Mass behavior of humans is pretty predictable, but individual choices can probably make some difference. I have no answers for those who want to listen to that devil (a big challenge in and of itself), but for those would prefer to listen to the angel instead, I have some tips.

It is a partisan issue

I often see the plea “it is not a partisan issue” attached to factual arguments. I think this represents an aspiration: it should not be partisan. In 2020 America, the reality is that there is a powerful anti-science, anti-truth faction, and that faction makes up a large part of the composition of one of the political parties. It’s not 100% alignment, and I want to give credit where credit is due: for example, the moderate Republican governor of Massachusetts has embraced science and investment in public health, and the results in that state are one of the best in the country.

But in general, any factual argument that has bearing on a political question such as “should we invest in public health” or “should we restructure the economy away from carbon consumption” is now inherently partisan.

Much of the partisan content in media is around issues that are not very consequential, either pure entertainment, or (at least traditionally) between candidates or parties that have comparable ability to govern. I was hopeful that the life or death nature of Covid-19 would motivate the people who decide what appears in media to higher standards. But apparently, it’s just the nature of the scorpion, they either can’t help themselves or just don’t care. Almost certainly, the media organization with the most blood on its hands is Fox News, which actively promotes an anti-health, pro-infection agenda. I don’t understand how they sleep at night.

Truth threatens a lot of existing power structures. If everybody woke up tomorrow and decided they were tired of being lied to, there would be seismic shifts of power. It’s worth thinking about who might be most affected by such a thing, and the kinds of things they do to keep it from happening.

High-end science journalism

I’m going to cover a range of information sources in this post, but there is a tl;dr: read up on actual science journalism by people who know what they’re talking about. Coverage by Science, Nature, and STAT News has been uniformly excellent. Good journalism is always teamwork, but I’ll also raise up the voices of particularly insightful individuals: Jon Cohen and Kai Kupferschmidt of Science, Amy Maxmen of Nature, Helen Branswell of STAT News. These are science journalists who have invested years in honing their craft.

I’ve also been impressed by coverage in The Atlantic. Their science content is quite good (thanks to top-notch contributors such as Ed Yong), but they also put the science in the context of American culture and politics. It is for this reason, I believe, that our President has singled them out for criticism.

It’s interesting that these sources tend to be publications that have been around for a while (STAT News is a brand of the Boston Globe media group). It’s not exactly rocket science how to do this, you just need to find experienced, capable writers and pay them to do journalism, with editorial and institutional support. But in today’s hyper-capitalist society, sustaining such efforts is a struggle. If you can, try to support publications that do good work by subscribing, or donating in the case of non-profits.

For someone with a limited budget of time and attention, good science journalism is absolutely the best bang for the buck. The other recommendations in this post are for people who want to spend a little more effort to gain, hopefully, a little more insight.

By far the most interesting experience I’ve had has been Twitter. What makes Twitter unique is its extremely wide dynamic range of information quality: both the worst of the worst and the best of the best are well represented. (An example of an information source of low dynamic range is an encyclopedia: almost every article is pretty good, garbage and brilliance both being rare)

A well-curated Twitter feed is one of the best information sources available today, and following random people on Twitter is one of the worst.

Even though it has no formal structure such as subreddits, Twitter functions as a set of overlapping communities, often described as “X Twitter” or “Y Twitter” (incidentally, a good explanation of how communities work on Twitter can be had in Jeff Jarvis’ interview of the voice behind Steak-umm). There are many experts who generously share their knowledge and thoughts with us, and I’ve found that with a modest investment of time, it’s possible to follow along.

In one of the Jeff Jarvis interviews, they jokingly referred to “idiot Twitter,” and, though it was a joke, the concept has stuck with me. Twitter, like all social media properties, works by engaging very large numbers of people with content that appeals emotionally. The expert communities are far too small a fraction of Twitter’s traffic to be profitable for them, but they support them as a way to maintain the prestige of the platform. The replies of any popular post are filled with all manner of conspiracy theory, miracle cure, denial, blame, and, I think at least sometimes, active disinformation by those who would do us harm (it’s hard to tell the difference and the outcome is the same). I think of all that as “idiot Twitter” and just do my best to avoid it. I also see it everywhere, of course, not just Twitter, and the quantity of it is a good measure of the value in a social media platform.

Twitter has a delicate balance to maintain, as there are many aspects of the site design that push you toward “idiot Twitter” even when you’re trying to curate your experience to avoid it. One example that consistently frustrates me is the trending panel, as that seems to be about half composed of idiot Twitter. It’s addictive, even so, because it tempts you with the thrill of discovering things early. I’ve also found that it’s easier to avoid junk on the desktop site than the mobile app. I’m worried the experience will degrade; Quora stands as a cautionary tale of what can happen to a relatively respectable site when they pursue clicks at the cost of all else.

Whether it would be viable to develop a social network designed for quality of discussion is another topic beyond this blog. I’m not especially hopeful on this front, as it’s hard to see how it would be profitable. In any case, there’s not much value in wishing for the possibly impossible, as somebody seeking quality information can find it even today.

List curation

One of the interesting features of Twitter is that people will curate and share lists of people they feel are worth following. My covid-19 list is one such, and a lot of thought went into it: I’ve intentionally included people with a wide range of expertise (including, for example, Joseph Allen for his insight into healthy buildings and Dr. Uché Blackstock for expertise on health equity). I’ve also deliberately excluded people from this list who express strong political views or often use an emotional tone. Because of the latter criterion, it’s missing people I personally feel are worth listening to, especially Gregg Gonsalves and Jeremy Konyndyk. As always, your mileage may vary. The goal of this particular list is not to be comprehensive, it’s to select a set of voices that have excellent signal to noise ratio; for a much more comprehensive list, see this one curated by Jeff Jarvis.

One phenomenon I’ve noticed is that people at the top of these lists tend to recommend each other. I wouldn’t be surprised if some automated evaluation (perhaps based on eigenvalues) could yield a good curated list. That said, in my past research, I’ve found that any automatic metric with an incentive will be gamed, and invariably will lift up self-promoters over quiet, thoughtful voices.

Speaking of self-promoters, two names do not belong on lists of top experts, but are often found there: Eric Feigl-Ding & Laurie Garrett. Both are good communicators, but are also sloppy enough with their science to get called out regularly. Inclusion of these names is a sign that a list of experts is not especially carefully curated.

Another important warning sign is lack of inclusion of women and people of color; mainstream media is often guilty of this. Health equity is an especially important part of the Covid-19 story, and not listening to expertise in this area is as irresponsible as it is predictable. Dr. Angela Rasmussen is a good critic of media representation problems, on top of being a brilliant virologist (she is co-author on that D614G paper mentioned below) and master Twitter communicator.

Medium quality sources

This section is easily skippable, as these medium quality sources can be safely ignored. But if they make up a significant fraction of your information diet, it might be useful to examine their flaws.

It’s been fascinating watching experts and more mainstream sources side by side, as coverage of stories so often sounds completely different. There are particular stories that seem to grab the attention of the media B ark:

• Anything indicating that Covid-19 is less serious than it really is. For example, the Stanford seroprevalence studies were hyped massively in the press, even though they were based on shoddy science.

• Alarmist narratives as well (“if it bleeds, it leads”). An exceptionally good marker for this is the D614G mutation. In the mainstream press, including reporters I would ordinarily trust, it’s a widely believed fact that this mutation represents a deadlier strain of the virus. Even before attention on this specific mutation, the excellent Nextstrain work was often misinterpreted as varying degrees of transmissibility or infectiousness (as is the case for influenza). It is possible that there will be a “there there” on D614G, but there’s no evidence yet for practical consequences.

• And, combining the two themes above, the idea that the virus is “weakening,” seemingly based on theories of evolutionary pressure.

I spend a fair amount of time on Hacker News as a source of tech information and discussion. It tends to run about one Covid-19 story a day, and the quality of discourse there is, regrettably, pretty bad. It is a reasonable source of insight into which particular strains of misinformation are widely believed by “tech bros,” who tend not to be shy at all about confidently trumpeting their armchair takes.

I reserve special condemnation, though, for the NY Times opinion page. They frequently run pieces that are simply indefensible to anyone who cares about truth, and wouldn’t pass the laugh test if reviewed by experts. A fairly extreme but sadly not completely atypical example is the Brett Stephens column America Shouldn’t Have to Play by New York Rules. It likely is the case that the NY Times opinion page accurately records the dominant narrative of those in power, and as such is a good window into that aspect of America’s brokenness. Whenever they run a particular bad howler, Gregg Gonsalves can usually be relied on to provide an expert critique.

I have not found video sources to be particularly helpful compared with the best of the other sources cited above (partly it’s my bias toward text for complex thoughts), but of the cable news hosts, Chris Hayes has been consistently above average, treating the topic seriously and avoiding oversimplification.

I’m not going to talk about actual low-quality sources here, though it’s an extremely important part of understanding the climate of disinformation. I will note, though, that I have actually deleted (not just paused) my accounts on Facebook properties.

Authority

In a functioning government, we’d be able to rely on government authority as a good source of information. Indeed, that could be seen as one of the primary functions of government. The CDC was once widely acclaimed as the best public health organization on the planet, and an acknowledged leader on the world stage. Today it is muzzled and ineffective. A particular failure is guidance around masks. Through conflicting messages and weak communication, they have squandered a great deal of trust.

Another highly troubling story is their case fatality rate estimate, which is much lower than expert consensus. They’ve been repeatedly asked to provide data and scientific reasoning, but don’t even pretend to be able to defend their work.

As a result, you’re in a situation where, as a computer scientist with a passing interest in Covid-19, it is extremely likely that my estimate (0.5% - 1.0%) for infection fatality rate is more accurate than that of the disease authority of the land, and by linking to Carl Bergstrom’s thread I have just provided better scientific evidence for that assertion than the CDC is capable of managing. [If, in a couple months, it turns out I am wrong and the CDC is right, by all means please use this post to discredit anything else I say that’s out of my lane.]

This state of affairs is a bit of a problem for organizations like search engines and social media, who generally need clear criteria for what’s good information and what’s disinformation, so naturally want to turn to authority. Since this blog post disagrees with the CDC, it might well run afoul of these kinds of guidelines.

By contrast, I was originally fairly critical of people like Tomás Pueyo for self-promotion over proper expertise. But from his work as a “vice president of growth” at a Silicon Valley startup, he is clearly skilled in understanding communication and human behavior, and his “flattening the curve” piece arguably saved many more lives than the CDC’s halfhearted comms efforts. Even more impressively to me, his The Hammer and the Dance piece set out a roadmap in March with advanced concepts such as budget for staying below R = 1. By contrast, our actual public health “leadership” appears to have been eating paste, blowing that budget on stupid shit like bars, while leaving us in an impossible situation where we have to reopen schools in the fall without a viable plan to keep people safe. As I write this, the CDC is facing a firestorm of criticism for allowing political and non-science-based considerations to influence their guidance for schools in the fall, and, in trying to appease both sides, have lost their credibility with both as well. Leadership can come from many places, and we have to find it where we can get it.

That said, while the leadership of the CDC is broken in the same way as so many institutions in America, it still has lots of excellent people working for it. And don’t confuse my criticism as being anti-expert, quite the contrary. Armchair epidemiologists often cite failures (real or perceived) of authority as justification for why their rando opinions should be taken seriously, but that’s just a logical fallacy.

Peer-reviewed papers

I did a minor in molecular and cell biology as part of my masters degree at Berkeley, so I have more than a passing intellectual curiosity in such things, but overall I am not a huge fan of lay consumption of scientific papers. A huge risk is cherry-picking. It’s possible to find support for just about any viewpoint in the peer-reviewed literature, and that’s actually working as it should be — it’s how scientific conversations happen.

The hydroxychloroquine story dramatically illustrates potential failures of peer review. There are two layers to the HCQ story. Just looking at pure science, it’s quite routine for pharmaceuticals to show early promise, then on closer examination, no real benefit. Hydroxychloroquine is no different than dozens of others in this respect. But, of course, the other layer is that it was adopted as a miracle cure, especially by leaders of certain political parties around the world. In the early days, this narrative was fueled by a peer-reviewed paper which was later retracted, with the integrity of the journal questioned (a blog post by Derek Lowe also makes good reading).

But the story gets weirder. While experts were converging around a consensus that HCQ had limited if any effectiveness, a paper in the highly regarded Lancet journal seemed to definitely answer the question in the negative, to the contrary arguing that it was actively harmful. However, on closer examination, that paper turned out to be extremely suspect and was also retracted. Given the rather obvious flaws, how did it get published in the first place? My personal favorite narrative is from the Respectful Insolence blog, as it’s both factual and colorful. Adding to the strangeness, one of the best criticisms of the Lancet paper (cited by that linked blog) is from James Todaro, who is best known for publishing a “paper” in favor of HCQ that overstated the credentials and affiliations of the authors in a way that verges on fraudulent, if not crossing the line. Strange days.

The moral of this story is that appearing in a peer reviewed publication is nowhere near a guaranteed stamp of truth. As always, critical thinking wins. In monitoring #EpiTwitter, I’ve seen the pattern quite frequently of some controversial paper appearing (very often “science by press release” before a proper preprint is even available), attracting skepticism and criticism early on. Frequently Carl Bergstrom, an expert on bullshit, is a strong voice of such criticism.

The equation changes of course, when one is willing to invest the time and energy to really study a topic, to develop the background of knowledge to understand and appreciate the work.

There’s a major discussion to be had regarding the flaws of peer review, including the role of preprints, the gatekeeping role of publishers, and fraudulent paper mills (see Elisabeth Bik for a steady stream of offenders on this front), but that’s beyond this blog.

You

I’ve argued that you can’t trust authority like the CDC, and that mainstream sources fall far short of what we need. As a reader of my blog, you likely have above average ability to discern truthful information from lies, and you also probably can communicate reasonably effectively to other people. Because our systems and institutions are failing us so badly, the responsibility then falls on you, my dear reader, to do your part to improve science communication in general, and in particular around Covid-19.

Cut out disinformation. Seek out actual experts. Do your own critical thinking, and don’t follow just because you like the source or feel alignment with their politics. Amplify voices worth amplifying, and don’t give lies more oxygen. I hope these notes are useful in some way.

Thank you for these efforts, and please, take care of ourselves and each other.

from Hacker News https://ift.tt/2BHwyvL