Evidence Based Medicine

PLoS One. 2025 Apr 4;20(4):e0321211. doi: 10.1371/journal.pone.0321211. eCollection 2025.

ABSTRACT

OBJECTIVE: To explore teaching- and assessment-related factors that predict medical students' perceived attainment of sufficient skills to incorporate scientific evidence into patient care.

METHODS: An anonymous questionnaire was distributed to final-semester students in six medical programs in Sweden. The students were asked to rate statements concerning the extent to which 11 national degree outcomes related to the scientific basis of medicine (scholarly degree outcomes) had been adequately assessed during the program; their perceived preparedness for evidence-based patient care; and training during the program regarding the components of a systematic review/health technology assessment (HTA).

RESULTS: In total, 433 students (median age: 25 years [interquartile range: 24‒28], 59% female) participated in the study (response rate: 68%). A multivariate analysis indicated that experienced adequate assessment on a single scholarly degree outcome (i.e., "Demonstrate knowledge of the scientific foundation of medicine and insight into current research as well as knowledge of the link between science and proven experience") predicted the students' perception of having developed sufficient skills in incorporating scientific evidence into patient care (odds ratio: 6.17 [95% confidence interval: 3.10; 12.3]). The educational content predictors of this perception included the teaching of HTA (11.3 [1.44; 89.5]) and training regarding two components of a systematic review/HTA: appraising scientific articles using checklists (2.46 [1.23; 4.90]) and assessing organizational aspects related to the introduction/withdrawal of a health technology (2.65 [1.05; 6.67]). The presence of hands-on, credit-bearing, evidence-based medicine (EBM)-related learning activities during clinical courses was also predictive (4.68 [1.69; 13.0]).

CONCLUSIONS: This study highlights important educational activities that prepare medical students to incorporate scientific evidence into patient care: (i) adequate assessment of key content regarding scholarly outcomes, including the scientific foundation of medicine; (ii) learning activities about HTA and the systematic review process; and (iii) hands-on application of EBM-related learning activities integrated into clinical courses.

PMID:40184390 | PMC:PMC11970701 | DOI:10.1371/journal.pone.0321211

BMC Med Res Methodol. 2025 Apr 2;25(1):87. doi: 10.1186/s12874-025-02534-0.

ABSTRACT

BACKGROUND: The use of Expert Opinion (EO) in clinical guidelines is highly variable and lacks standardization, leading to ongoing controversy. A clear and universally accepted definition of EO is also lacking. To date, no research has systematically assessed how guideline-developing societies conceptualize and apply EO. This study aims to map methodological manuals, evaluate their rationale for EO use, examine its foundations, and synthesize a comprehensive definition.

METHODS: Systematic searches for clinical guidelines were conducted in PubMed to identify guideline-developing societies, supplemented by additional searches. Systematic searches were then conducted to identify methodological manuals from these societies. Screening was performed independently by two reviewers, and data extraction was conducted using piloted forms. Findings were summarized through narrative evidence synthesis using descriptive statistics.

RESULTS: A total of 473 national and international societies were identified, and methodological manuals from 98 societies were mapped and analysed. These manuals included 61 handbooks, 29 journal articles, and 8 websites. EO is mentioned in 65 (66%) manuals, with substantial variation in its utilization and terminology. EO is primarily used in two contexts: (1) filling evidence gaps (72%), and (2) interpreting existing evidence (8%). In the remaining 20%, EO use is unclear. Five main foundations could be identified as a potential basis for EO (clinical experience, indirect evidence, low-quality evidence, mechanism-based reasoning, and expert evidence/witnesses). Based on these findings, a novel comprehensive definition of EO was synthesized.

CONCLUSIONS: EO is widely used to address evidence gaps and interpret ambiguous evidence, underscoring its importance in guideline development. However, the variability in its application and conceptualization across societies highlights the need for standardization. We propose a comprehensive EO definition as a first step towards standardization to improve consistency, transparency, and clinical decision-making.

PMID:40175922 | PMC:PMC11963610 | DOI:10.1186/s12874-025-02534-0

J Eval Clin Pract. 2025 Apr;31(3):e70051. doi: 10.1111/jep.70051.

ABSTRACT

RATIONALE, AIMS AND OBJECTIVES: Despite using state-of-the-art methodologies like Grades of Recommendation, Assessment, Development and Evaluation (GRADE), current guideline development frameworks still rely heavily on panellists' intuitive integration of evidence related to the benefits and harms/burdens of health interventions. This leads to the 'black-box' and 'integration' problems, highlighting the lack of transparency in guideline decision-making. Combined with humans' limited capacity to process the large volumes of information presented in Summary of Findings (SoF) tables-the primary output of systematic reviews that underpin guideline recommendations-this reliance on non-explicit processes raises concerns about the trustworthiness of clinical practice guidelines.

METHODS: SoF tables provide the best available evidence, derived from frequentist or Bayesian estimation frameworks. Decision analysis, which integrates both types of estimates but considers intervention consequences, is the only analytical approach that combines multiple outcomes (benefits, harms and costs) into a single metric to support decision-making. Such analysis seeks to identify the optimal decision by balancing harms, benefits and uncertainties. This paper leverages the PICO format (Population, Intervention, Comparison(s), Outcome) as a conceptual basis for deriving SoF tables. Subsequently, we propose a solution to GRADE's "black-box" and "integration" problems by matching PICO-based SoF with decision models.

RESULTS: We succeeded in connecting the PICO framework to simple decision-analytical models, restricted to time frames supported by empirically verifiable evidence, to calculate which competing intervention offers the greatest benefit (net differences in expected utility; ΔEU). The single metric [ΔEU] enabled a simple, transparent and easy-to-understand assessment of the superiority of competing management strategies across multiple outcomes (considering both benefits and harms), addressing the 'black-box' and 'integration' problems. Completing a SoF-based decision model takes about 10 min. Not surprisingly, the recommendations based on ΔEU may differ from the intuitive recommendations of panels.

CONCLUSION: We propose that incorporating the straightforward and transparent modelling into guideline panels' decision-making processes will enhance their intuitive judgements, resulting in more trustworthy recommendations. Given the simplicity of calculating ΔEU, we advocate for its immediate inclusion in systematic reviews and SoF tables.

PMID:40165549 | PMC:PMC11959216 | DOI:10.1111/jep.70051

Ren Fail. 2025 Dec;47(1):2476736. doi: 10.1080/0886022X.2025.2476736. Epub 2025 Mar 26.

ABSTRACT

BACKGROUND: Systematic reviews and meta-analyses play a pivotal role in evidence-based medicine, including nephrology, by consolidating findings from multiple studies. To maximize their utility, rigorous quality assessment during peer review is essential. Challenges such as heterogeneity, bias, and methodological flaws often undermine these studies, necessitating a structured appraisal process.

METHODS: This guide outlines a framework for nephrologists on appraising systematic reviews and meta-analyses. Key areas include heterogeneity assessment using the I² statistic, interpretation of forest plots for pooled effect estimates, and the use of funnel plots with Egger's test to identify potential publication bias. Risk of bias is evaluated using RoB 2 for randomized controlled trials and ROBINS-I for non-randomized studies. Subgroup and sensitivity analyses, along with meta-regression, address heterogeneity and examine the robustness of findings.

RESULTS: The I² statistic quantifies heterogeneity by estimating the proportion of variability in a meta-analysis. Funnel plots and Egger's test help detect publication bias. Major biases, such as selection, performance, detection, and publication bias, are identified using structured tools like AMSTAR 2, Cochrane RoB 2, and ROBINS-I. The GRADE framework further assesses the overall certainty of the evidence. Emphasis is placed on PRISMA compliance, protocol pre-registration, and transparent reporting of statistical analyses, subgroup, and sensitivity assessments. The inclusion of grey literature remains optional.

CONCLUSION: By focusing on key areas such as heterogeneity, risk of bias, and robust statistical methods, this guide enables nephrologists to critically appraise systematic reviews and meta-analyses, fostering better clinical decision-making and improved patient care in nephrology.

PMID:40143401 | PMC:PMC11951313 | DOI:10.1080/0886022X.2025.2476736

BMC Med Res Methodol. 2025 Mar 27;25(1):80. doi: 10.1186/s12874-025-02535-z.

ABSTRACT

BACKGROUND: The goal of evidence-based medicine is to make clinical decisions based on the best available, relevant evidence. For this to be possible, studies such as randomized controlled trials (RCTs), which are widely considered to provide the best evidence of all forms of primary research, must be visible and have an impact on clinical practice guidelines. We further investigated the impact of publicly and commercially sponsored RCTs on clinical practice guidelines by measuring direct and indirect impactful citations and the time to guideline impact.

METHODS: We considered the sample from the IMPACT study, where a total of 691 RCTs (120 German investigator-initiated trials (IITs), 200 international IITs, 171 German industry-sponsored trials (ISTs) and 200 international ISTs) was sampled from registries (DFG-/BMBF-Websites, the German Clinical Trials Register, and from ClinicalTrials.gov) and followed prospectively. First, all eligible IITs were sampled. Then, ISTs were randomly selected while ensuring balance across certain trial characteristics. Next, the corresponding publications in the form of original research articles were identified. A search was then conducted for (1) systematic reviews (SRs) citing these articles and (2) clinical practice guidelines (CPGs) that cited either the original articles or the SRs. The methods and results of this effort were already published. In this investigation we aimed to better characterize the impact of RCTs in CPGs. Therefore, we identified all citations of the original articles and SRs in the citing CPGs and classified them into impactful and non-impactful. This allowed us to calculate an estimate for the guideline impact of a trial. In addition, we estimated the time-to-guideline-impact, defined as the time to a direct and indirect impactful citation in a CPG. Direct means that the publication of a trial was cited in the main text of a CPG. Indirect means that the publication was cited and included in the findings of a SR and the SR was cited in the main text of a CPG. We also investigated to what extent pre-defined study characteristics influenced the guideline impact using multivariable negative binomial regression as well as the time-to-guideline impact using multivariable Cox proportional hazards regression.

RESULTS: Overall, 22% of RCTs impacted a CPG. For international ISTs, only 15% of trials had an impact in CPGs. Overall, of the 405 associated guidelines, 331 were impacted. Larger trials were associated with more impactful main text citations in CPGs and earlier time-to-guideline impact, while international industry-sponsored trials were associated with smaller impact on CPGs and longer time-to-guideline impact. IITs funded by governmental bodies in Germany reached an impact on CPGs that is on par with German ISTs or international IITs and ISTs.

CONCLUSION: This study demonstrated that a considerable number of trials previously identified as being linked to CPGs have had impact in those CPGs (85%). International ISTs seem to have a lower impact on CPGs, and fewer of them influence CPGs at all.

PMID:40140983 | PMC:PMC11948659 | DOI:10.1186/s12874-025-02535-z