How do we know if a newly hired doctor has what it takes to prevent patient deaths?
After completing residency training, graduating physicians typically take a medical certification exam upon entering practice. But surprisingly little is known about the ability of these standard tests to predict what matters most in a physician’s ability, such as a patient’s chance of survival or avoidance of a return visit to the hospital.
New study published on May 6th Japan Automobile Manufacturers Associationan internal medicine patient was found to have received top scores on the certification exam for newly trained physicians – a comprehensive examination typically taken after a doctor completes residency training – seven days after admission The risk of dying or being readmitted to the hospital was low.
The analysis was led by researchers at Harvard Medical School and the American Board of Internal Medicine (ABIM), which develops and regularly updates the exams that qualify physicians as internal medicine physicians. Some of the study authors, including lead author Bradley Gray, are employed by his ABIM.
The research team says the findings provide reassurance that the internal medicine specialty exam reflects future physicians’ performance on important indicators of patient care and outcomes.
These results confirm that certification exams measure knowledge that directly translates into improved patient outcomes. ”
Bruce Landon, lead study author, HMS Health Policy Professor, and internist at Beth Israel Deaconess Medical Center
Landon et al. said this is one of the first known attempts to measure the reliability of test scores in predicting patient outcomes.
The researchers also compared patient outcomes to a “medical milestone” assessment developed by the Accreditation Council for Graduate Medical Education (ACGME). These evaluations are based on regular reviews of trainees’ knowledge and skills during their training. This analysis found no association between patient outcomes and physician scores on milestone assessments.
Examination of doctors in training
Anyone who has watched a television medical drama knows that graduating from medical school is only the first step in a long journey of medical training and education. After graduating from medical school, students undergo training in their chosen specialty and then take a board certification exam to become fully qualified independent physicians in their specific medical field.
In recent decades, new ways to test the knowledge of these budding physicians have emerged.
In the field of internal medicine, resident competency is assessed in two ways. ACGME milestone assessments are conducted at various points during training. The idea is that regular milestone tests can provide feedback to doctors-in-training about how well they are doing.
At the end of their training, nearly all internal medicine residents take the ABIM certification exam, known as the Medical Board, although the length of time varies depending on specialty. This final comprehensive test assesses whether you qualify to join the ranks of independent practitioners who can care for patients without supervision.
A scientific approach to improving medical training
Most internal medicine specialists begin their careers as hospitalists, providing care to inpatients in hospitals. For this study, the researchers counted approximately 70,000 newly trained hospital physicians who treated Medicare beneficiaries out of 455,000 hospitalizations from 2017 to 2019. Outcomes were analyzed. The researchers compared outcomes for patients treated by doctors with different test scores within the same hospital. This allowed the researchers to eliminate, or at least minimize, the effects of patient population, hospital resources, and other variations that could affect a patient’s probability of death or readmission, independent of physician performance. I was able to keep it to a minimum.
Board exam performance was strongly associated with a patient’s risk of death or readmission. For example, patients whose doctors scored in the top 25% of exams were 8% less likely to die within seven days of admission compared to patients whose doctors scored in the bottom 25% of exams. , it was still a passing score.
Although physician milestone assessments do not appear to be predictive of patient outcomes, using them as a periodic assessment to determine where physician trainees are still valuable to those implementing training programs. The researchers said it could be a useful tool.
“This type of evidence-based evaluation of our unique testing tools provides valuable insight into what types of tests work for what purpose and educates future healthcare professionals and healthcare leaders. “We will inform how testing should be implemented at that time,” Landon said.
Authors, funding, and disclosures
Other authors include Jennifer Stevens of HMS, Rebecca Lipner, Furman McDonald, and Jonathan Vandergrift of the American College of Internal Medicine.
Gray, Lipner, McDonald, and Vandergrift reported being employees of ABIM. Mr. Landon reported that he received consulting fees from ABIM for his ongoing work during the conduct of the investigation. No other disclosures were reported.
sauce:
Reference magazines:
Gray, B.M.; other. (2024). Association of Internal Medicine Training Milestone Assessments and Certification Exam Scores with Patient Outcomes. Japan Automobile Manufacturers Association. doi.org/10.1001/jama.2024.5268.
