Welcome to The Plague Pit – issue number 30.

The COVID-19 death rate is falling in the UK and the US, and rising in many other countries. As the pandemic plays out globally, we are learning more and more about the disease.

According to one major database of academic literature, 3,406 relevant new papers were published last week alone. That makes more than 23,000 this year. A few of these may go on to influence the way we manage the condition in individual patients or at a policy level

  

We’ve heard a lot about ‘scientific papers’ in the media recently. Many of these contain complex clinical and experimental detail incomprehensible to the lay reader. Even doctors rarely understand the statistical bits in full, unless they are involved in research themselves. And the academic medical literature en masse can be frankly terrifying.

A doctor who encounters a patient with an unfamiliar medical problem will want to find out how other doctors have dealt with it. Similarly, a researcher looking at a potential new treatment for a disease like COVID-19 will want to know if anyone has thought of it before. They will often do this by tackling that huge body of medical literature through one or more online databases. The people who often know a good deal about these are librarians.

I like libraries. When I was a medical undergraduate in the 1980s, the internet had not been invented. Digital resources were almost non-existent. If I was stuck for ideas for an essay, I would make my way to the University Library (UL) for tea and a scone – and a look at the Cumulated Index Medicus. The US National Library of Medicine used to produce this set of books annually – a summary on paper of all the reputable medical research articles published in the last year

To give an idea of scale, Volume 26 of the Index (the one for 1985) comprises fourteen books – each like a telephone directory. Seven of them list medical subject headings alphabetically, with details of relevant publications under each subject heading. Book 10 – for example – covers subjects starting ‘Fr-Ins’ and is 1100 pages long. In five more books, the individual papers are listed again under first author surname, arranged alphabetically.

The Volumes for two decades – hard-bound, hundreds of them – filled a small room near the library entrance. They were arranged on two deep shelves running right round a central island, the length and breadth of a small lorry. It had a flat top at waist height so you could haul out a book, set it down immediately and flip through the pages, thin as cigarette papers. In tiny print, the 1985 entries for ‘Fracture Fixation’ fill five of these.

Thank heavens for digitization?

That’s all gone now. As in other walks of life, digitization has made things easier but left us with too much choice. If I want to find out whether social distancing prevents COVID-19 transmission, which electronic database of global medical literature should I search? And once that has given me a huge list of articles, which of these should I read?

My usual approach is to start small. My ‘go-to’ electronic database is PubMed, run by the US National Library of Medicine (https://pubmed.ncbi.nlm.nih.gov/).

In the search bar, I type the following:

Search (All Fields): (covid-19 OR sars-cov-2 OR pandemic OR coronavirus)

Today, that gets me 62,788 publications. Rather too many to read before lunch. I can date-limit the search to this year’s publications alone, like this:

Search: ((covid-19 OR sars-cov-2 OR pandemic OR coronavirus)) AND ((“2020/01/01″[Date – Publication] : “2020/06/10″[Date – Publication]))

 That returns 23,077 titles. Still huge.

These two numbers, incidentally, strongly suggest that my search term is not optimal – it is unlikely that most articles on COVID-19 were published before 2020. It might be worth dropping ‘OR coronavirus’ from the search term but I’ll carry on for the moment

There are thousands of medical journals and most have been publishing papers on COVID-19. Some journals are better than others, though. If I want to avoid reading 23,000 articles, it makes sense to limit myself to those in the best journals. PubMed has already helped a little in this respect – it only includes articles from journals that are peer-reviewed.

In the peer-review system, authors submit manuscripts describing their research to journal editors in the hope of publication. The journal editor distributes these manuscripts to volunteer reviewers, usually senior clinicians (‘peers’) in the specialty covered by the journal. Reviewers make comments, suggest amendments and recommend whether the manuscript should be accepted for publication or not. If the subject matter is very specialist, the editor will consult an ‘expert reviewer’. I sometimes get asked to review manuscripts on pharmacogenomics, the subject of my doctoral research.

Editors will make a decision to publish based on the reviews and on his own assessment. Is the research original, interesting, ethical, well-conducted, likely to influence future clinical or research practice? The peer-review system has come under fire in recent years. There’s concern that statistical methods have become so complex that most ‘peers’, and many experts, don’t understand them well enough to spot important errors. There’s also the matter of ‘publication bias’.

This bias exists because well-conducted studies with positive findings (Drug A works better than Drug B) are more likely to get published than equally well-conducted studies with negative findings (no significant difference in effect between Drug A and Drug B). Readers may be more interested in positive results and this perception is thought to skew decisions on publication by reviewers and editors alike.  

Many clinicians who are short on time choose to read the journal in their field with the highest impact factor (IF). IF is a complicated numerical rating, updated annually, that is said to reflect the ‘quality’ of a journal. It’s based on how often authors everywhere cite work published in it. Impact factors are not without controversy.

https://en.wikipedia.org/wiki/Impact_factor

For general clinical research, I usually try the Lancet (IF 59.1), BMJ (27.6) and New England Journal of Medicine (70.7). For research specifically about anaesthesia, the journal with the highest IF is currently the British Journal of Anaesthesia (6.2). All data here come from 2018.

So, let’s say I’d like to limit my PubMed hunt to COVID-19 articles in the Lancet. I enter:

Search: (((covid-19 OR sars-cov-2 OR pandemic OR coronavirus)) AND ((“2020/01/01″[Date – Publication] : “2020/06/10″[Date – Publication]))) AND (“Lancet”[Journal])

252 articles. Getting better

For general information on a mainstream medical topic, a review article can be a good place to start. Academic medical journals publish these, alongside original medical research papers.

A review article is an overview of the medical literature on a particular topic, often written by researchers in the same field. Review articles are really useful for readers wanting to get up to speed on a subject quickly. Medical students with essays to write who have spent too long in the library tea room, for example.

Reviews take two forms – narrative and systematic. Narrative reviews were once a good opportunity for anyone with a bit of specialist knowledge or experience to get their name in print. Typically, the author of an old-style narrative review would not indicate how he/she chose the published papers to which they refer. Author bias was therefore a risk from the start. Narrative reviews often tackle a broad subject, without reference to a specific clinical question. Here is a particularly egregious example.

For some years now, editors of major journals have rejected narrative reviews and insisted on systematic ones. Narrative reviews still appear, even on COVID-19, but mostly in less prestigious publications (1). In a systematic review, the authors start by describing exactly what they have done to identify and consider all potentially relevant publications.

In a recent systematic review of the literature on physical distancing, face masks and COVID-19 transmission, (see below) the authors searched PubMed and more than ten other major electronic databases. Of the 20,000 COVID-19 publications they identified and screened, only 172 met the criteria for inclusion in the review.  A ratio of this order is not unusual for a well-conducted systematic review.

As is often the case, the review was followed by meta-analysis – a statistical technique which combines data from multiple studies to increase statistical power.

Meta-analytical techniques are mind-bogglingly complicated. Most researchers and institutions believe that they provide the best possible answer to clinical questions – the highest level of evidence. Many though, including me, remain suspicious of the subjective nature of the processes – particularly with respect to bias in the selection of articles for inclusion in the statistical analysis (https://en.wikipedia.org/wiki/Meta-analysis)

Data are also a concern. Authors of meta-analyses have not personally conducted the original studies themselves, so they must take great pains to avoid the inclusion of unreliable data. Nevertheless, a few major meta-analyses have been retracted, after publication, because studies included by the authors in good faith turned out to be fraudulent in some way.

Sometimes unscrupulous researchers, for example, will use the same patient data in two research studies. If the authors of a meta-analysis don’t spot this and assume the two original studies relate to different patient groups, the whole meta-analysis may be invalid.

And finally….

Lets’s get back to my 252 articles. I’m an hour away  from publishing this issue of The Plague Pit – and I just don’t think I can read them all.

 If I look on the left hand side of the PubMed screen, though, there are some tick boxes under the heading ‘Article Type’.

One of the Article Types (out of shot in the image above) is ‘Systematic Reviews’. If I tick this, I get only one article:

Chu DK et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. Lancet. 2020 Jun 1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7263814/pdf/main.pdf

Phew – just in time. It’s an important article, I think. It looks like social distancing does reduce COVID-19 transmission, and that masks probably work, too. The statistics are mostly gobbledygook to me, though. I’m really sorry…..

I’ll get onto some of the simpler research methods that have cropped up in pandemic related publications next week. And another dodgy paper that’s been in the news, too.

The Good Soldier

.

(1) Di Gennaro F et al. Coronavirus Diseases (COVID-19) Current Status and Future Perspectives: A Narrative Review. Int J Environ Res Public Health. 2020 Apr; 17(8): 2690. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7215977/