Raising Questions About The Use Of Minimum Activity Volume With CPME 320

MarOnlineEx2010.png [3]

Chart1marOE.png [4]

Chart2marOE.png [5]

The Council on Podiatric Medical Education (CPME) has released proposed revisions to CPME 320, which have sparked quite a bit of discussion on residency training. This author raises concerns about the use of Minimum Activity Volume (MAV) to help establish competency and suggests some alternative options.

   The Council on Podiatric Medical Education (CPME) recently released proposed revisions to the CPME publication 320, Standards and Requirements for Approval of Residencies in Podiatric Medicine and Surgery.¹ Since the release of these proposed revisions, there has been a flurry of comments and recommendations, including a variety of opinions regarding the requirement for residents to meet Minimum Activity Volume (MAV) for podiatric cases and procedures. My recommendations on this topic assume that the reader has some familiarity with residency training and specifically, the current and proposed CPME publications 320.^1,2

   Currently, Standard 6.10, Section A of CPME 320 dictates MAVs for surgical procedures and cases for PM&S-24 and PM&S-36 programs (see the table “A Closer Look At Current CPME Requirements For Surgery” below). As the standard says, “MAVs are patient care activity requirements that assure that the resident has been exposed to adequate diversity and volume of patient care. MAVs are not minimum repetitions to achieve competence. For some residents, the minimum repetitions may be higher or lower than the MAVs. It is incumbent upon the director of podiatric medical education and the faculty to assure that the resident has achieved a competency, regardless of the number of repetitions that it takes for the given resident.”² [6]

   The CPME’s purpose of including minimum activity volume as a requirement for residents is well intentioned. Its definition of MAVs, specifically the statement that MAVs are not minimum repetitions to achieve competence, is universally accepted. However, this statement does not correlate with the requirement for residents to perform a designated volume of procedures in order to achieve competency. Unfortunately, the CPME has once again included MAVs in its proposed revisions to the CPME publication 320.¹

   The current values for the MAVs as published in CPME 320 are arbitrary choices with no basis of evidence to justify their inclusion. There is no valid data to support the practice of using a set number of procedures to determine surgical competency that would apply to all residents equally. For example, it is inaccurate to state that a resident is incompetent in digital surgery if he or she has only obtained 99 of the 100 minimum number of digital procedures. [7]

   What is more troubling with the MAVs is that numbers for a PM&S-24 program are different than those for a PM&S-36 (see the table “A Closer Look At Current CPME Requirements For Surgery” above). A PM&S-24 resident meets the digital surgery minimum volume at 80 procedures while a PM&S-36 resident needs 100 procedures to reach the minimum. Do PM&S-24 residents learn faster? One must consider this question when reviewing the proposed revisions to the CPME publication 320. The CPME has proposed eliminating the PM&S-24 program. The new, three-year residency model utilizes the same MAVs as the existing PM&S-36. Why didn’t the council use the MAVs for a PM&S-24, adding in the Reconstructive Rearfoot and Ankle (RRA) component? It appears that the CPME has determined that the present PM&S-24 MAVs are too low, which suggests that PM&S-24 graduates are incompetent in surgery. What data did CPME base this decision on? Should we rescind certificates from PM&S-24 graduates?

VSCDO: An Alternative Proposal To Minimum Activity Volume

   I propose that we modify the method by which we determine surgical competency by employing Verification of Surgical Competency through Direct Observation (VSCDO). Here are some key components to implementing the VSCDO.

   • Remove MAV “Procedure Activities” from CPME 320.
   • Remove the “overrepresentation” requirement since MAVs are part of the formula to determine overrepresentation.
   • Keep the diversity requirement. This states that within each procedure category and subcategory, at least 33 percent of the procedure codes within each category and subcategory must be represented with C-level experience procedures.²

   Accordingly, the VSCDO would read as follows: To ensure competency, the resident must demonstrate and submit written verification of surgical competency in at least 33 percent of the procedure codes within each category and subcategory. For example, in the category of Other Osseous Foot Surgery, the resident must have documented evidence of competency in at least six of the 18 different procedure codes.

   Form VSCDO 1 (a standardized form to be developed by the CPME), for example, must be in use for each procedure to verify surgical competency. Form VSCDO 1 should include: the resident’s name, the name of the supervising attending who is verifying competency, the surgery date and the facility. The form would also include the patient identification, the patient age and the procedure being tested for surgical competency.

   This form would also include a statement summarizing the purpose of the surgical competency verification process and a definition of competency. Immediately below this statement shall be two signature lines: one for the attending and a second for the scrub nurse on the case. Both parties must sign prior to the start of the procedure.

   A legal statement would appear near the bottom of the form. This statement needs the signatures of the attending and the scrub nurse verifying that the resident performed the procedure being evaluated, correctly and independently in a reasonable amount of time. The statement shall also note that the resident:

•    successfully evaluated preadmission testing (PAT) preoperatively including review of all patient results and X-rays, and discussed findings with the attending;
•    discussed the rationale for the treatment as well as available alternate procedures; and
•    participated in anesthesia selection and when indicated, administration.

   By including the scrub nurse as a witness to the verification process, there is less chance of the procedure being verified inappropriately. The scrub nurse will verify that the resident performed the procedure independently but will not verify correctness or procedure length. There will be a final signature line for the program director.

   Only one procedure per case is eligible for competency verification. A single attending may complete no more than one-third of the total VSCDO forms required for a resident. Therefore, an attending may only complete eight of the 26 competency verifications required per resident, not including RRA surgery. Including RRA surgery, the attending may only complete 13 of the 39 competency verifications needed.

   A single attending shall not be the sole evaluator in satisfying a resident’s requirement in a single category or subcategory. For example, in the Other Osseous Foot Surgery category, the resident must have documented evidence of competency in at least six of the 18 different procedure codes. All six competency verifications cannot be from a single attending.

How The New Verification Process Would Work

   When residents feel confident that they can perform a surgical procedure correctly and independently, they first would obtain approval from the program director. Then the resident asks any authorized attending to verify surgical competency in a single procedure for an upcoming case. If the attending agrees to verify competency, then prior to the procedure, the resident must complete and present form VSCDO 1 to the attending and scrub nurse for their review and signatures.

   At the conclusion of the case, if the resident performed the procedure correctly, independently and in a reasonable amount of time, the attending and scrub nurse would sign off on the procedure at the bottom of form VSCDO 1, and return the form to the resident. The resident then submits the form to the residency director and logs the procedure, including a comment that surgical competency through direct observation of the procedure was verified.

   In addition, the program director would make available to all residents a current list of podiatric surgeons authorized to perform competency evaluations and in which categories. All competency evaluators must be board qualified or certified in foot surgery by the American Board of Podiatric Surgery (ABPS). For RRA procedures, the evaluator must be ABPS board qualified or certified in Reconstructive Rearfoot/Ankle surgery.

   The program director may disallow a competency verification if he or she believes that the resident is not truly competent in the procedure. If the resident fails to demonstrate competency during the procedure undergoing testing, the VSCDO 1 form is destroyed with no penalty to the resident. The resident may not make another attempt to achieve competency on anymore procedures until obtaining authorization from the program director.

Recognizing The Potential Benefits Of Verifying Surgical Competency Through Direct Observation

   The VSCDO is the final step in assessing competence. The VSCDO evaluates for intraoperative success for a given procedure. Performance in all other segments of surgery (i.e. perioperative management) would still need to be evaluated by residency programs, but not as a component of VSCDO. Program directors may find that none of their existing competencies and corresponding competency evaluation forms will need revision.

   Podiatry Residency Resource, Inc. (PRR) would need to create a checkbox field for each procedure entry in its Web-based logging tool. The resident could click this to indicate a procedure that attained verification for competency. The creation of two new program report categories is also necessary so residency directors (and the CPME) can run reports. With these reports, one can view all cases in which a procedure attained verification (VSCDO Report by Resident) and a summary report displaying which procedures have and have not been verified (VSCDO Summary by Resident).

   Since residents will be required to document in their logs when a given procedure attains verification for surgical competency, program directors and the CPME will be able to collect and analyze data that has never been available before. The CPME will then be able to provide valuable statistical reports to the profession such as the average number of procedures per category that it takes a resident to achieve surgical competency.

   Residency directors currently feel pressured to have as many patients as possible in order to ensure that their residents will meet the MAVs. They feel no pressure from the CPME to ensure that residents become truly competent in surgery. Today, a resident who meets the MAV in all categories but has questionable competency will likely graduate. A surgically competent resident who does not meet the MAVs will not receive a residency certificate.

   Residents focus on obtaining surgical cases, not surgical expertise, since they become “successful” once they meet their numbers. Now the motivation will change from obtaining numbers to learning and perfecting skills. If they want to receive a residency certificate, residents must become competent surgeons. Therefore, residents will study more, be more inquisitive and hone their skills.

   Residents will seek out cases with attendings who provide them with the best possible training. As a result, attendings who excel at instructing are the most likely to have residents scrub in on their cases. This is an incentive for them to enhance their teaching skills.

   As residents achieve competency in various procedures, program directors can tailor training by directing cases/procedures to the residents who need them. For example, a resident who has obtained VSCDO in digital procedures after 40 cases will not need to focus on additional arthroplasties (although they will continue to perfect their skills in this category throughout their training). The program director can direct this resident to scrub in on other types of procedures in which the resident needs more training. Digital procedures can go to the residents who still need verification of competency in the digital category. This will permit programs to be more efficient in their use of resources (cases).

   Residents will continue to participate in surgical procedures in categories in which they have achieved competency verification. The program director and resident should communicate frequently regarding the resident’s training needs and desires. All residents in a program do not need to participate in the identical number of procedures per category or subcategory. However, the CPME should retain the requirement that each resident receives equitable training experiences. Therefore, residents do not have to be concerned about being excluded from future surgeries after they have achieved their competency verifications.

   By replacing MAVs with VSCDO, there is minimal change in workload for attendings and program directors. The VSCDO requires only several signatures by the attending at the time of surgery. Program directors only need to sign and file the VSCDO 1 forms.

   Since residents will want and need ample training prior to requesting competency verification, programs will still need to provide an adequate volume of surgical cases for residents. In order to meet surgical competency in at least 33 percent of the procedure codes within each category and subcategory, residents will need competency verification in 26 procedures, not including Reconstructive Rearfoot and Ankle Surgery (RRA surgery). When one includes RRA surgery, the total number of procedures is 39 (see the table “What Are The VSCDO Minimums Per Category?”).

   A single attending may complete no more than one-third of the total VSCDO forms required for a resident. I [8]agree with Cherie H. Johnson, DPM, who stated: “Similar to the concept of ‘it takes a village to raise a child,’ it takes multiple individuals to train a well-rounded resident. Therefore, the assessment of the resident’s competency must not be done by one individual.”³

Other Key Considerations In Verifying Surgical Competence

   A new challenge that program directors will face is to ensure that each resident is competent in all areas of podiatric surgery and is not simply meeting some arbitrary number. This new focus can only enhance residency training. Although program directors are currently “required” to ensure competency, there is no specified or compulsory method for doing so. Most programs simply utilize a “rotation evaluation form” that includes check boxes to rate competencies as excellent, good, fair or poor/unacceptable. Program directors infrequently check off “poor/unacceptable” since they would then become saddled with a whole set of time-consuming headaches (i.e. remediation for unsatisfactory performance).

   Performance evaluations should be based on observed behaviors on a per-patient basis rather than more global impressions of performance across a rotation or period of time.^4,5 In addition, our profession should consider developing a standardized method, used by all programs, to evaluate resident performance such as the Accreditation Council for Graduate Medical Education (ACGME) has done.⁶

   Initially, institutions should continue to follow the current criteria for the establishment of a new residency program. Once adequate data becomes available on when residents actually meet competency in given procedures, we can develop new criteria for start-up programs.

   Greater attention to surgical training for students at podiatric colleges and clerkships will help ensure that graduates will be able to complete residency training successfully. Furthermore, colleges must prevent students from graduating if they do not possess an inherent ability to perform surgery. If students have a slim chance of successfully achieving surgical competency in a residency program, it is unethical to promote them through all four years of podiatry school.

   There may be concern that by instituting VSCDO, some residents will be unable to complete their residency program successfully due to an inability to achieve competency in all required categories and subcategories. Unfortunately, there is no alternative for this resident. Therefore, I propose the following modification to the draft for comment, proposed revisions to the CPME publication 320.¹

   • In addition to an added credential for RRA surgery, implement an added credential for foot surgery.

   • All residency programs would be required to provide surgical training in foot surgery (per the current proposal). However, residents who complete three years of training without successfully obtaining VSCDO would receive a residency certificate without the additional foot surgery and RRA surgery credentials. These residency graduates would only qualify for qualification/certification by the American Board of Podiatric Orthopedics and Primary Podiatric Medicine (ABPOPPM). They would not be eligible to sit for qualification/certification by the American Board of Podiatric Surgery (ABPS).

In Conclusion

   It is clear that the minimum activity volume requirement for surgical cases and procedures in CPME 320 has no validity. The desire to eliminate this requirement appears to be shared by others although no detailed alternative to MAVs has been offered. Therefore, I propose the implementation of the new VSCDO method. With the VSCDO method, residents would be more motivated to learn and perfect their skills. Residency programs would be able to direct surgical procedures to residents who need them and the CPME would be able to collect and analyze data that has never been available before.

   The CPME notes that all comments on the proposed 320 changes must be submitted by April 1.

Dr. Green is the Chief of Podiatry for the VA New Jersey Health Care System and is Director of the Podiatric Residency and Clerkship Programs. Over the past 22 years, he has helped train over 90 residents and several hundred externs. Dr. Green is a residency program evaluator and represents both the ABPS and ABPOPPM during on-site visits.

Dr. Green thanks Jeffrey M. Robbins, DPM, the Director of Podiatry Services for the Veterans Affairs Central Office at the Louis Stokes Cleveland Veterans Affairs Medical Center, his faculty and planning committee for providing a course entitled “Ensuring Quality Patient Care through Competency Based Evaluation.” Dr. Green notes that this training program, which was held in December 2009, inspired him to question the habitual way in which we utilize MAVs in verifying competency. Dr. Green also thanks Susan Padrino, MD, an Assistant Professor in the Department of Psychology and Internal Medicine, and the Medical Director at Douglas Moore Health Center in Cleveland, who incorporated her research and experience in leading the session.

Disclaimer: Dr. Green notes the views expressed in this article are his views alone. He says the article does not represent the views or opinions of the Department of Veterans Affairs, nor does the article represent the views or opinions of the Council on Podiatric Medical Education, the ABPS, the ABPOPPM, or any other agency or association.

References:

   1. Council on Podiatric Medical Education. CPME 320. Draft for comment by the residency community of interest. Standards and requirements for approval of podiatric residencies. October 2009.    2. Council on Podiatric Medical Education. CPME 320. Standards and requirements for approval of residencies in podiatric medicine and surgery. July 2007. nbsp;  3. Johnson C. Competencies as an evaluation tool. Clinics in Podiatric Medicine and Surgery 2007; 24(1):103-117.    4. Phelan S. Evaluation of the Non-cognitive professional traits of medical students. Academic Med 1993; 68(10):799-803.    5. Gray J. Global rating scales in residency education. Academic Med, 71(1 suppl):S55-63.    6. ACGME and ABMS. A product of the joint initiative of the ACGME Outcome Project of the Accreditation Council for Graduate Medical Education (ACGME), and the American Board of Medical Specialties (ABMS). Version 1.1. September 2000.