I have written previously about the need for transparent and detailed reporting of scientific research. There should be enough detail in a study report to allow an interested reader to replicate the study should they wish. Replication is a fundamental scientific tenet. Yet, like the general science field, the evidence that the healthcare field is living up to this ideal is not very encouraging.
According to an article in the Wall Street Journal, Amgen more often than not are unable to reproduce findings published in scientific journals. In addition, Bayer has halted nearly two thirds of its early phase projects for the same reason. Unfortunately, the situation within health economics and outcomes research is, in my experience, even worse.
It is a constant battle between editors wanting to improve the transparent and detailed reporting of papers and authors (or more likely study sponsors) wanting to hide behind concerns of intellectual property. In the past few weeks I have had three such examples. In two of these examples, the papers were reports of complex modelling studies where it was very unclear in the original submitted paper how the input paramater values were derived. After several weeks of correspondence (akin to having a wisdom tooth extracted) we managed to obtain a supplementary appendix which went a little way to meeting our request. In the third example, we had a battle with the authors of a survey-based paper who even had concerns about sharing the survey with the reviewers, yet alone the readers!
How we assuage the need for transparency yet overcome the concerns regarding intellectual property I do not know. Perhaps in the case of modelling a move to reference case models might be a move in the right direction? However, how authors of survey-based research expect readers to critically evaluate such research without recourse to the survey instrument is beyond my comprehension. Perhaps this is one more indication of how interests in financial returns are stymying the pursuit of science and hindering advancements in patients’ healthcare.
One final comment. I do believe that those of us who elect to become members of the leadership of various guideline/good practice groups, such as the ISPOR Task Forces, do need to live by what we preach. Editors need to ensure that published papers are in accordance with accepted reporting guidelines. As authors, if we preach transparency then every paper we put our name to should live up to that ideal. In my experience this has not always been the case.
Whenever I give presentations on the topic of transparent reporting of health research I always encourage the use of reporting checklists of which the EQUATOR network is a fantastic repository.
I have given the topic of reporting checklists some considerable thought in the last few months. As a member of the leadership team of the ISPOR Task Force on Health Economic Reporting Guidelines I have spent some time thinking about what are the essential elements to a good report of an economic evaluation. In addition, in recent months I have read a considerable number of published systematic reviews and evaluated them against the PRISMA statement and checklist.
Undoubtedly reporting checklists are a useful aide-memoire for authors and are also help editors and reviewers in assessing whether the essential elements of a piece of research have been reported. However, one thing I had not considered (until a colleague asked me this week) was whether journals should publish (most likely as online supplementary material) the checklists as completed by authors. This would undoubtedly be in the spirit of transparent reporting but would it lead to the checklists being used as a quality measure, which is not their intended purpose? There is a tendency to give equal weighting to the items in a checklist. This may not be appropriate and this is something editors and reviewers have to consider when making a decision. It is very hard if not impossible to find a ‘perfectly’ reported systematic review, as defined by the PRISMA statement and checklist. What are your thoughts? Please answer the poll below. It would be great to know what your thoughts are.
Last week I had the pleasure of attending the International Publication Planning Association Annual Meeting as an invited speaker. It was an interesting insight into the many issues that face the medical publication planning industry. The issues that appeared to be at the forefront of many at the conference included authorship, The Sunshine Act, journal selection and communication with editors, and Corporate Integrity Agreements. Although the topics discussed were diverse there was one word that I kept hearing again and again from speaker after speaker (including myself). What was that word? It was the T word – Transparency.
Journal editors are constantly reminded of the need to ensure that original research is reported in a transparent fashion, authors are being urged to be transparent in reporting potential conflicts of interest and how they meet authorship criteria, the pharmaceutical industry is becoming more transparent ( we now have registration of clinical trials), payers are being more transparent about reimbursement criteria, the World Bank recently published a report on transparency in Industry, in New Zealand hospitals are being urged to be more transparent about medical misadventure claims, and so on. We do truly live in a world where transparency is an expectation and all our actions come under intense scrutiny. If something is not transparent we immediately become suspicious and suspect something is awry.
Economists have been somewhat slow to appreciate the demand and need for transparency. Only in very recent times has the American Economic Association started to request economists declare where they act as both advisors to government and private industry. Health economists still in many cases fail to see the potential conflicts in acting as advisors to health technology assessment agencies and consulting for the pharmaceutical industry. Many published descriptions of economic models fall well below what could be described as transparent reporting. This is often cited as one of the reasons as to why economic evaluations fail to influence decision making.
We would love to hear your opinion on why economic models are often reported in an unclear fashion. Should journals routinely request a copy of the model? Are we making models unnecessarily complex just because we can? How can we improve transparent reporting of economic models?