Expert panel diagnosis for diagnostic test poorly described, experts not blinded to test under study

Public release date: 15-Oct-2013
[


| E-mail



| Share

]

Contact: Fiona Godwin
medicinepress@plos.org
01-223-442-834
Public Library of Science

Evaluation of diagnostic studies is often a challenge in diseases that are not defined by a specific test. Assessment of the accuracy of diagnostic tests is essential because they may be used to define who is considered to have a disease and receive treatment for it. However, measuring the accuracy of a diagnostic test requires an accurate gold standard, which defines which patients truly have and do not have the disease. Studies of diseases not defined by a specific test often rely on expert panels to establish the gold standard. In a systematic review and analysis of the diagnostic literature using expert panels to define the gold standard for a given disease, Loes Bertens and colleagues from University Medical Center Utrecht determined how expert panels were used in such studies and how well their process was described and reliability assessed.

The authors evaluated 81 diagnostic studies published up to May 31, 2012, including studies of diagnostic tests for psychiatric disorders (30 of 81 papers, 37%), half of which pertained to
dementia, cardiovascular diseases (17 papers, 21%), and respiratory disorders (10 papers,
12%). They found that reporting was often incomplete, with 83% of studies missing at least some important information about the expert panel. In 75% of studies the panel consisted of three or fewer members, and panel members were blinded to the results of the test results being evaluated in only 31% of studies. Blinding is important because knowledge of the index text results could influence the panelists' decision as to whether the patient had the disease. Reproducibility of the decision process was assessed in only 21% of studies.

The authors state, "Complete and accurate reporting is a prerequisite for judging potential bias in a study and for allowing readers to apply the same study methods. In total, only 14 (17%) papers reported complete data on key issues such as the panel constitution, the information presented to the panel and the exact decision process to determine the final diagnosis." They also found that despite publication of reporting guidelines, the completeness of reporting did not improve over time, perhaps because the reporting guidelines do not include specific criteria for expert panel diagnoses. The authors make a number of recommendations to improve reporting of expert panel diagnosis. They conclude, "Our review revealed a large variation in applied methods as well as major deficiencies in the reporting of key features of the panel diagnosis processThe results of our review may serve as a starting point in the development of formal guidelines on methodology and reporting of panel diagnosis."

###

Funding: The study was conducted as part of the Dutch National Care for the Elderly Program (ZonMw-NPO). Research grant from the ''Netherlands Organization for Health Research and Development'' (ZonMw grant 311040302). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing Interests: The authors have declared that no competing interests exist.

Citation: Bertens LCM, Broekhuizen BDL, Naaktgeboren CA, Rutten FH, Hoes AW, et al. (2013) Use of Expert Panels to Define the Reference Standard in Diagnostic Research: A Systematic Review of Published Methods and Reporting. PLoS Med 10(10): e1001531. doi:10.1371/journal.pmed.1001531

IN YOUR COVERAGE PLEASE USE THIS URL TO PROVIDE ACCESS TO THE FREELY AVAILABLE PAPER:

http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1001531

Contact:

Loes Bertens

University Medical Center Utrecht

NETHERLANDS

mobile: +31 6 142 86 663

l.c.m.bertens-3@umcutrecht.nl

---

[


| E-mail



| Share

]

 

AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert! system.

Public release date: 15-Oct-2013
[


| E-mail



| Share

]

Contact: Fiona Godwin
medicinepress@plos.org
01-223-442-834
Public Library of Science

Evaluation of diagnostic studies is often a challenge in diseases that are not defined by a specific test. Assessment of the accuracy of diagnostic tests is essential because they may be used to define who is considered to have a disease and receive treatment for it. However, measuring the accuracy of a diagnostic test requires an accurate gold standard, which defines which patients truly have and do not have the disease. Studies of diseases not defined by a specific test often rely on expert panels to establish the gold standard. In a systematic review and analysis of the diagnostic literature using expert panels to define the gold standard for a given disease, Loes Bertens and colleagues from University Medical Center Utrecht determined how expert panels were used in such studies and how well their process was described and reliability assessed.

The authors evaluated 81 diagnostic studies published up to May 31, 2012, including studies of diagnostic tests for psychiatric disorders (30 of 81 papers, 37%), half of which pertained to
dementia, cardiovascular diseases (17 papers, 21%), and respiratory disorders (10 papers,
12%). They found that reporting was often incomplete, with 83% of studies missing at least some important information about the expert panel. In 75% of studies the panel consisted of three or fewer members, and panel members were blinded to the results of the test results being evaluated in only 31% of studies. Blinding is important because knowledge of the index text results could influence the panelists' decision as to whether the patient had the disease. Reproducibility of the decision process was assessed in only 21% of studies.

The authors state, "Complete and accurate reporting is a prerequisite for judging potential bias in a study and for allowing readers to apply the same study methods. In total, only 14 (17%) papers reported complete data on key issues such as the panel constitution, the information presented to the panel and the exact decision process to determine the final diagnosis." They also found that despite publication of reporting guidelines, the completeness of reporting did not improve over time, perhaps because the reporting guidelines do not include specific criteria for expert panel diagnoses. The authors make a number of recommendations to improve reporting of expert panel diagnosis. They conclude, "Our review revealed a large variation in applied methods as well as major deficiencies in the reporting of key features of the panel diagnosis processThe results of our review may serve as a starting point in the development of formal guidelines on methodology and reporting of panel diagnosis."

###

Funding: The study was conducted as part of the Dutch National Care for the Elderly Program (ZonMw-NPO). Research grant from the ''Netherlands Organization for Health Research and Development'' (ZonMw grant 311040302). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing Interests: The authors have declared that no competing interests exist.

Citation: Bertens LCM, Broekhuizen BDL, Naaktgeboren CA, Rutten FH, Hoes AW, et al. (2013) Use of Expert Panels to Define the Reference Standard in Diagnostic Research: A Systematic Review of Published Methods and Reporting. PLoS Med 10(10): e1001531. doi:10.1371/journal.pmed.1001531

IN YOUR COVERAGE PLEASE USE THIS URL TO PROVIDE ACCESS TO THE FREELY AVAILABLE PAPER:

http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1001531

Contact:

Loes Bertens

University Medical Center Utrecht

NETHERLANDS

mobile: +31 6 142 86 663

l.c.m.bertens-3@umcutrecht.nl

---

[


| E-mail



| Share

]

 

AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert! system.

Source: http://www.eurekalert.org/pub_releases/2013-10/plos-epd100913.php
Tags: Washington Navy Yard   Boulder Flooding   9/11 Pictures   Perez Hilton   vanessa hudgens