Medical Knowledge

Medical Knowledge deficiencies requiring remediation are common in Emergency Medicine training programs. This page lists resources for Medical Knowledge remediation, including best practices, words of wisdom, and Medical Knowledge remediation milestones and worksheets.

Words of Wisdom

From the CORD Remediation Task Force:

1. Medical Knowledge Deficiencies (MKD) are rarely a knowledge issue alone.
2. MKD are frequently multi-factorial. Evidence of a MKD requires the program to carefully and meticulously evaluate the deficiency to evaluate the true etiology of the issues (processing, psychiatric and time management issues, )
3. The ABEM administered in-training exam (ITE) is considered the gold standard for evaluating medical knowledge. Deficiencies on the ITE may be misleading if the above issues have not been addressed. The MKD task forces cautions programs from using this as a single source of information.
4. The task force considered a score of 30 percentile or lower “significant” for a knowledge deficiency. This score was felt to be a “trigger” to alert the program for a knowledge component deficit alone or in combination with other issues noted above.
5. Medical knowledge remediation is obtainable, if the resident is willing and participant in a rigorous course of evaluation and self-study.
6. Programs should use the GME offices for those residents who have MKD. Often times, groups of students can be taught general principles related to knowledge acquisition skills specific to residency regardless of the specialty that they represent. In addition, the GME office may have resources for additional psychometric testing as needed.
7. Reassessment of the deficit is critical. Dynamic study plans are instrumental for overcoming any deficiencies.
8. Poor clinical evaluations by faculty or peers were also considered excellent tools to alert the Program for an evaluation of the resident MKD.

Medical Knowledge FAQ:

What is the definition of Medical Knowledge?

Medical knowledge as it applies to Emergency Medicine (EM) has previously been defined as:

1. The specialized immediate recall of information for care of critical patients.
2. The understanding of the use of medial resources for the immediate care of the patient.
3. The ability to apply information to the undifferentiated patient presentations with an emphasis on data synthesis, the identification of life-threatening conditions, identification of the most likely diagnosis, and sequencing of critical actions in patient care.

back to top

What score is considered to be “deficient” by the academic EM community?

1. 30th percentile or lower on the ITE
2. The ITE is used because it is the only standardized tool that programs can use for assessment

back to top

What other evaluation tools do I have to assess for MKD

ABEM’s policy and the ACGME’s recommendations regarding assessment, programs should utilize multiple instruments, multiple observations, and multiple observers to assess MK
back to top

What are the principles to successfully remediate a resident with MKD?

Hauer and colleagues proposed the following four core elements of a powerful remediation program:20

1. An initial screening which uses multiple assessment tools to identify deficiencies (competence assessment)
2. Diagnosis of problems and development of an individualized educational plan
3. Provision of instruction which includes deliberate practice, feedback and reflection
4. Reassessment and certification of competence

back to top

What resources are available to remediate a resident with a MKD?

While the PD is often an instrumental factor in remediating residents with a knowledge deficit, consider the following resources for more generalized help

1. GME department
2. Psychometric testing
3. Board review classes
4. Simulation
5. CORD created question banks
6. Review books

back to top

What are important elements to remediate a resident with a MKD?

1. Evaluation
2. Multiple settings, multiple opinions
3. Diagnosis
4. A meticulous “comb” through of the resident’s study habits, social engagments, mental well being. If there is a psychological component, appropriate referral, if there is a learning disability, contact your GME, if there is a commitment to self study, a structured reading and testing plan that ownership by the resident and faculty member.
5. A multi-faceted, approachable plan that is reasonable and has time scheduled to maintain personal time.
6. Reassessment
7. A dynamic and rigorous reassessment of the resident’s performance

back to top

Remediation Plans by Milestone

Level 1: Passes initial national licensing examinations (e.g. USMLE Step 1 and Step 2 or COMLEX Level 1 and Level 2)

Remediation Task Force Recommendations:
Most medical schools will require this for graduation from medical school. However, PDs may use either a previous failure of these standardized exams or a sub-par performance to engage the resident early in remediation efforts prior to the first ITE.

1. Bring the resident in to discuss performance on the USMLE. Discuss the annual ITE and how often these scores are predictive of performance on theITE.
2. Construct a plan for study.
   1. This may include a specific reading curriculum.
   2. PD may elect to begin short questions from the various resources available paralleling your reading schedule for the overall residency or your conference curriculum.
3. Consider acting early with additional testing of the resident a.Most medical schools have resources available for psychomotor testing. Consult with your Dean’s office for these resources. If program is not affiliated with a medical school, reach out to your DIO. Additionally, commercial testing companies sometimes have services available (e.g. Kaplan, Princeton Review, Sylvan Learning Center). Keep in mind that poor test performance may not just be an indication of medical knowledge deficiencies but may involve other factors.

Level 2: Resident develops and completes a self-assessment plan based on the in-training examination results. Completes objective residency training program examinations and/or assessments at an acceptable score for specific rotations.

Remediation Task Force Recommendations:
Residents should complete a self-assessment plan based on their ITE results. These can incorporate satisfaction with their score, plans for study, goals for next year. This tool will be helpful for the sit down with the PD to see how much insight the resident has into his/her performance. An example of a self-assessment tool (courtesy Flavia Nobay) is included.

PDs will need to determine what an acceptable score is on the residency training program examination as stated in the milestone. This would be the ITE for most people.

1. Previous Medical Knowledge Task Force recommendation has been the 30th percentile nationally.
2. The ITE now gives PDs useful information in the predictive value of the resident’s score for passing the examination. The Task Force recommends that using a 90% prediction of passing the certification examination is an acceptable cut off.

Remediation recommendations for those that are not at this percentile:

1. Begin a clear educational plan for the resident over the next year leading up to the following inservice examination
2. Consider creating an elective rotation supervised by the PD or designated faculty member that will focus on medical knowledge (e.g. reading elective with tests)
3. Consider mandating attendance at a national board review course
4. Avail yourself of the resources in the Task Force folder and decide which will best suit your resident stylistically and your program financially
5. If performance on the ITE is not consistent with previous test taking scores on the USMLE (hopefully alerted on Level 1 and entry into residency), evaluate for other distractors
6. This may be the first opportunity to do a comprehensive evaluation of a resident and address learning from all aspects:

Level 3: Demonstrates improvement of the percentage correct on the in-training examination or maintain an acceptable percentile ranking.

Remediation Task Force Recommendations:
In order to get to this level, a resident will have had to complete level 2 which marks those at risk for not passing the inservice examination. The Task Force recommends that on this level, PDs play close attention to those residents who are just at a plateau on their ITE score or have in fact declined. In each year of training, we expect that the percentage correct on the examination should increase. A PD can determine what amount is acceptable, but it should be in accordance with acceptable percentile rank of peers.

If an increase does not occur, but there is a satisfactory percentile, focused remediation may be considered:

1. Evaluate specific topic areas with lower scores on the ITE
2. Consider focused remediation for these areas with reading and testing on specific topics
3. Discuss resident’s reading habits and educational plan over the year outside of residency conference.

Level 4: Obtains a score on the annual in-training examination that indicates a high likelihood of passing the national qualifying examinations. Successfully completes all objective residency training program examinations and/or assessments. Passes the final national licensing examination (step 3 or Level 3)

Remediation Task Force Recommendations:
The achievement of Level 4 on this milestone can theoretically happen midway through residency or during the final year. There are two scenarios for this milestone:

1. A resident who is in the final year of training and has not met this or
2. A resident who previously was at Level 4 and has now not met it based on intraining examination.

Scenario 1 (resident who is approaching graduation and has not yet met this milestone)

1. PD should develop an intensive study plan for the final months of training. Given the proximity to graduation when scores are received, there is little time for continued remediation and this resident has a high likelihood of not passing the certification examination.
2. Map out a specific plan for the last several months of residency.

Scenario 2 (resident who previously was at this level and has now fallen off)

1. This should alert the PD to investigate other causes
2. If there are no identifiable outlying causes, consider administering another mock examination to the resident drawn from available resources to sense knowledge deficits
3. If resident performance remains poor, no outside issues, begin intensive remediation as noted above for scenario one.

back to top

Medical Knowledge Remediation Tools

This is a list of useful tools to help residents with specific medical knowledge deficiencies.

Test Taking Strategies
A short list of test taking strategies for standardized tests

OACEP Board Review
Ohio ACEP Board Review Course: Several programs have used this for residents who are struggling and paid for the resident to attend with departmental funds.

Peer VIII
For apple products, also available through the itunes store: Peer VIII for IPad

Rosh Review
Question bank currently with 1,500 questions and comprehensive explanations that refresh every July 1, so residents never see the same question twice. A Program Director's Dashboard (PD Dash) which provides valuable insights into resident progress, offers over 500 novel questions made into "minitests" to help with remediation and individualized interactive instruction, and a prediction model to identify "at risk" residents for failing the initial certification exam. The PD Dash also contains free access to quizzes for EM Foundations. Attending physicians are on call to help with all content questions. Fees do apply. 

MD Challenger: Fees do apply.

HippoEM
Board Review and CME made by the educators of EMRAP. Includes 60 hours of video, hundreds of board questions and ability to send questions to the faculty.

Commercial Online In-Service Review

EMboards.com
Through the Center for Emergency Medical Education. Annual Board Review Course and also option to purchase a streamed video version, DVD, MP3.

EMedHome.Com Access to 40 hours of online lectures.

ITunes Board Review
Board review specifically for iphone, ipad or Android: Features 5000 board review questions for $8.99

Board Review Resources
Great central listing of many of the resources with some reviews

Academic Life in Emergency Medicine AIR Series
A core independent study resource, modules with questions and answers, very high quality reviews of important emergency medicine topics

back to top

RTF Consults:

Medical Knowledge Consult 1