v. 26 May 05
Understanding ED-2: “Q&A”
These are answers and comments from Drs. Carol Aschenbrener and Robert Eaglen of the LCME to clarify questions pose by Louis Pangaro, President of the Alliance for Clinical Education (ACE), on behalf of their constituent clerkship director groups.
General Comment: Doctors Aschenbrener and Eaglen would like to emphasize that the recent clarifications (June 2004) are meant to ensure that attention is being paid to the experiences that students have in clinical clerkships, particularly that the experiences are sufficient and consistent, and that all students meet the objectives decided by the faculty. The comments below repeatedly emphasize the collective judgment of faculty (under the leadership of the school and departments) in determining what core expectations are. They should not be left to individual teachers, to site directors or ward attending physicians.
Underlying this is the premise that the overall goals and objectives of the school have been determined, and that departmental faculty have determined clerkship objectives that support those goals. The LCME anticipates that by specifying this as a priority task for the clerkship, the clerkship directors will be given the time and resources they need to meet the expectations.
The list of core items is the prerogative of the faculty of the individual medical school and should support the objectives of the institution; but it is expected that such a list is defensible and not appear arbitrary to others. Overall, the expected planning starts with the particular school’s deciding what a successful graduate looks like, then determining an overall plan for achieving that success for each student. Once the overall plan is set, resources (patients and reasonable alternatives to patients) should be in place within the clerkships to make sure that the objectives are achieved, and the goals are met; in other words, there should be suitable process and outcome measures.
The LCME will try to clarify some of these problems in the FAQ section on their web site, and work with LCME site visitors to be sure that ED-2 is applied as intended. In the meantime, this summary of points from Dr. Pangaro's discussion with them is meant to help clerkship directors understand what is expected.
· Symptoms, syndromes and other common clinical presentations do meet this expectation. In other words, departmental faculty can determine that "abdominal pain" is a core problem, and that each students does not need to see a patient with peptic ulcer disease or cholelithiasis.
· This ED-2 expectation could be documented either by having the student’s check list specify "abdominal pain", or by having the student check "peptic ulcer disease" or "cholelithiasis" and the program’s software give credit in a master list for "abdominal pain"
· There is no minimal, or maximal, number of core problems that must be specified, but the specified list of core items should be sufficient to meet the core objectives of the clerkship; again, the list should be defensible.
· Items on a core list are those "essential" to meet the department's and school’s objectives. The faculty must determine what these items are. For instance, the "essential" item could be "common malignancies" and breast cancer, colon cancer or lung cancer could be judged by the faculty to satisfy the student’s having encountered this problem
· Other core problems may be considered "desirable", for instance hyperthyroidism; and this could be satisfied by seeing Graves' disease, a toxic nodular goiter or an excessive dose of exogenous thyroxine. "Desirable" items are, by definition, not part of the "core" list and it is up to the clerkship to determine whether they are to be tracked.
· The core list may include both disease conditions (symptoms/syndromes/diseases) and clinical skills (geriatric patient evaluation, an evidence based medicine exercise, etc.)
· The list should be related to the explicit objectives for the clerkship. For instance, the objective of becoming proficient in geriatric evaluation could be satisfied with an elderly patient who had heart disease, rheumatoid arthritis, or any number of other conditions, as long as the specific tasks related to the geriatrics objective (e.g., mobility, mentation, ADLs) were met by each student
· The overall balance of conditions and skills should be defensible. In other words, a core list in internal medicine would probably include common/serious conditions (for instance, heart failure) for which the inherent content is very important. This kind of problem could probably not be totally displaced by a list of core skills.
· If the specific medical condition is considered "core" by the faculty (such as upper respiratory infection on pediatrics), then the clerkship should anticipate this and make provision for providing an alternate exposure to the problem (for example with a video case simulation) irrespective of the time of year
· On the other hand, if the faculty considers the core problem to be "acute respiratory compromise", then a number of specific medical conditions could satisfy this (pneumonia, severe asthma, aspiration). The important point is that if the faculty considers the problem to be part of the "core”, then acceptable ways of having a student encounter the problem should be anticipated and provided.
· It is a judgment of faculty whether more than one encounter with a clinical condition or clinical skill is necessary. For instance, post-operative wound care may require at least four encounters in a surgical clerkship, although participation in only one laparotomy may be expected.
· The faculty should also determine whether there are alternatives experiences for achieving the required number of exposures to a problem; whether for instance, observation and subsequent discussion of a videotaped laparoscopic procedure would allow the student to meet the specified clerkship objectives.
· It is not expected that experience in the required number of patients on a typical clerkship will exceed reasonable work hours for student. A single patient may satisfy three separate items on a core list. For instance, a patient with an emergency laparotomy for an acute abdomen may satisfy three items from different lists: acute abdominal pain, participation in surgical procedure, and postoperative care. Again, it is the faculty, who must determine what is essential, and what is desirable.
· The faculty should have already made this determination: if a problem is in the "core" AND the faculty feel that a simulation is not an appropriate alternative to encountering the problem, then each student should see a real patient (either participating in the patient's care, or interviewing/ examining another student’s "real" patient on rounds). It is likely that core problems and skills for each discipline will be sufficiently common in the clinical setting for this to be a reasonable expectation.
· For clinical problems or skills that the faculty considers to be essential there must be some mid-clerkship attention paid to what the student has already seen, and what he/see is likely to see in the remaining weeks. This could be done by, for instance, by the ward team resident, principal attending or the clerkship director, and alternative exposures to the problem (seeing a patient worked up by another student, standardized patient, video simulation, etc.) should be in place.
· Progress toward implementing adequate exposure to core problems should be clear. Even if you have the money, SPs and video simulations take a while to get up and running. Some interim alternative, but not necessarily an expensive or complex one, should be available if the experience is essential – a paper case, supplementary lecture, or review session with the preceptor or clerkship director might suffice. While interviewing another student’s patient might be adequate, depending on the level of encounter expected by faculty, a casual encounter such as occurs during rounds would not seem sufficient.
· As faculty define the types of encounters expected of students, they should consider the reason for putting a disease state or clinical condition on the “required” list: should the student just “see” the disease or condition; do you want the student to work through differential diagnosis or management issues; or do you want the student to exercise some skill related to that condition? The level of exposure should help guide decisions about when simulations or standardized patients are appropriate and when they are not.
· Yes, as long as the faculty feel that the alternatives are appropriate. For instance, a simulation or even a "paper case" discussion may be judged by faculty to be an acceptable alternative for encountering a laboratory condition such as an acid-base disorder; on the other hand, a paper case would not be a reasonable substitute for a stroke patient with hemiplegia. For stroke, the neurology faculty might consider that a well-trained standardized patient might be an acceptable alternative.
· Mid-course attention to each student’s clinical experience may happen at the same session as mid-way feedback, but it is in addition to, not in place of, midcourse feedback to students on their performance.
· The faculty are essentially making this determination when they decided what is the list of core problems to be seen, and what are the acceptable ways that a student can encounter them.
· It would not seem reasonable that the faculty would decide that all problems could be encountered in simulations, since exposure to real patients is essential to the clerkship experience.
· It seems reasonable at this point that clerkship directors and faculty would expect that the majority of problems seen on a clerkship would be in circumstances sufficiently "authentic" to have face validity. For instance, standardized patients may be a reasonable proxy for encountering an angry, drug dependent patient on a psychiatry clerkship; but, a standardized patient would not be an authentic proxy for taking care of a patient with postoperative fever.
· The core of how the clerkship prepare graduates for their future professional responsibilities is meaningful participation in relevant settings in the care of actual patients who have the types of conditions that all graduates should be prepared to recognize.
· If the faculty determine that a specific condition he is "essential" or core" to their discipline, then the LCME feels that they and the clerkship director have the responsibility for making this sure the student has seen the problem, in the setting and in the depth required to meet the clerkships objectives.
· If for instance a student on a Family Medicine clerkship had already encountered and documented several patients with hyperlipidemia while on Medicine several months earlier, then the Family Medicine clerkship director could consider that core problem to have been encountered. On the other hand, if the Family Medicine faculty specified that discussion of the dietary and lifestyle issues in the home setting were necessary to meet their core objectives, then seeing the problem on Medicine may not have satisfied this objective.
· It is important to reemphasize that the purpose of ED-2 is to make sure that attention is being paid to what the student experiences, and that departmental objectives are being met. It is, for instance, up to the Psychiatry faculty to determine whether and how having seen a depressed patient on the Family Medicine Clerkship meets their expectations.
· Educational research is allowed and even encouraged, with the proviso that any variation between students and their experiences still meets the objectives set by the faculty in the department
· In other words, if the judgment of faculty is that a certain clinical condition is "essential" for all students to see, then having some students not see that problem would not be acceptable.
· If the faculty are not certain whether exposure to a particular problem should be in their discipline’s “core”, then it would be fine to do the research to answer that question, even if all students did not see the problem. In other words, if a problem is not judged to be in the "core", there is clearly flexibility.
· This would be true for both the clinical condition itself and the nature of the student’s experience. For instance, if in fact, the faculty do not know whether seeing a "real" patient with chest pain is better than seeing a standardized patient with the problem, it would not be necessary to specify that this problem be encountered in a real patient, at least until the research answers the question.
· This also applies to the number of patients seen: for instance, if the faculty do not know that doing four exercises in evidence based medicine is superior to doing two, then the threshold could be set at two, and it would be worthwhile to study the question of whether four produced a superior outcome in meeting the clerkship’s objectives.
6. Process versus product measurements: The goal “to achieve the objectives of the learning experience” seems to emphasize documenting the short term process measurement, rather than the longer term outcomes. Is that correct? Do you envision a next phase (like the ACGME for 2006-2011) in which process and product must be related?
· Outcome or "product" measurements are part of the educational process. There should be an overall plan at the school level to ensure that the school’s objectives are met. Some objectives and measurements might be achieved in any of several clerkships, and it is the school’s responsibility to assure that the objectives are met independently of any clerkship-specific requirements.
· It is the responsibility of faculty to determine which conditions and skills are the essential core, and what assessments (whether process or product) should be achieved by each student.
· To the extent that clerkship objectives are discipline specific, there should be clerkship-level attention to whether the objectives are being met. This could be a "process measurement" such as documenting the number of times a student has practiced informed consent, or an interim outcome measure such as direct observation by a member of the faculty.
· The standards require that the clerkship objectives be linked to the institutional goals and expected competencies. Some assessments performed during or at end of a clerkship may relate only to clerkship objectives; other assessments may contribute to the school’s overall determination of how its students/graduates achieve the institutional objectives (outcomes). Thus, it’s important that outcome measures not be replaced by process measures; both are needed. With respect to skill in soliciting informed consent, for example, we would expect the skill to be observed if it were on the target list for the clerkship.
· The LCME recognizes that actually tracking what a student does (process measurements) and measuring clerkship specific outcomes (for instance, use of the subject examination series of the NBME, or direct observation of skills by faculty) takes time and resources. The point is that if the faculty consider something "essential", then the school should have committed those resources.
7. Limitations of currently used patient logging methods to document students’ experience: The requirement to “verify, by appropriate means, the number and variety of patient encounters in which students participate” has caused anxiety, because getting enough accuracy in logbooks for high stakes evaluation has been elusive.
· At this point, the LCME is looking for reasonable methods of documentation for what each student sees. The important issue now is that someone is paying attention to what each student is doing, so that if the student has not logged in seeing a core problem by midway, someone works with a student to achieve the objective. If the student says that he/she has, in fact, seen the problem, then the log can be corrected. If the student hasn't, then the student is given direction on how to satisfy the core item
· The key point is that if the faculty has determined that there is a list of 10 core clinical conditions and five clinical skills for their discipline, then it ought to be feasible for each student to keep an accurate log for this number of problems.
· Since some mid-way attention is expected for each student, the tracking method should be feasible for the setting, and this could be achieved with either paper methods, PDAs or web based methods.
· LCME survey teams may or may not wish to see samples of clerkship logs, but are very likely to ask students if they are keeping logs and whether they are reviewing them with someone who can help them complete all the core experiences (for example, clinic supervisor, team resident, preceptors or clerkship directors).
· Clerkship directors have to make some judgment about whether they wish to capture a longer list of "desirable" clinical problems and skills. In other words, it may be easier to get accurate data from students if the lists are short, and the tracking logs focus on what is essential. However, this may be a function of the method used to track student experiences (PDA, PC, optically scanned paper forms, paper lists) and no explicit recommendations can be made by LCME.
8. As we move from documentation that a teaching site can provide adequate clinical experience for group of students, to a stricter requirement for each student to have documented a core list of problems, the cooperation of students has to increase. This is going to translate into sanctions on a student who does not complete and submit documentation of their patient contacts, both midway and at the end of the clerkship. Does the LCME see this as part of the process? What about requiring students to purchase palms/handheld devices?
Documentation of patient encounters is an essential part of medical practice. Thus, a school’s specific requirements for students to document their clinical experience with patients is preparation for medical practice, and accurate documentation may be regarded as a component of professional behavior. We would expect that completion of required documentation would be dealt with similarly to completion of papers, projects and other course and clerkship requirements. Both paper logbooks and computer logs (with handheld devices) are in use in schools that have implemented quantified patient criteria. Decisions about purchase of palms/handheld devices are the province of the individual schools, similar to decisions about learning materials and instruments.
9. What are the trade-offs in having “core” lists versus a longer list of desirable problems for programmatic purposes? Longer lists and menus may decrease accuracy. And do we want to encourage clerkships to have short lists just to make it easier to track?
· Once again, the length of the lists depends on the clinical learning objectives and the ability of the program to monitor the experiences.
· Ultimately, the objectives and “core list” of items to be tracked, while determined by the faculty, should reflect the standards of the profession and concerns of the public.