Attention IACME members and friends
Looking for an opportunity to chat with like-minded colleagues? Please join us for our initial Special Interest Conference Calls
- Innovations in Self-Assessment - on Friday, July 22, 2016 at noon (call highlights below);
- Preparing for the ACCME Self-Study - on Thursday, August 4th at 11:30am (call highlights below);
- Maintenance of Certification - on Wednesday, August 17th at noon-CANCELLED (will be rescheduled soon)
- 2017 schedule: TBD
If there are particular topics you would like to discuss please email Sue Piscoran at SPiscoran@aol.com
SIG - Innovations in Self Assessment
Highlights of the July 22, 2016 call:
Most traditional Q and C self-assessments described by participants were online enduring materials with multiple-choice questions. Among the most unique assessment mentioned was the Society of Gynecologic Oncology's Robotic Surgery Assessment where learners’ movements are tracked and learners' level of expertise and confidence are assessed.
Learning Management systems used by participants included: Extend Med – Joomla LMS (American Society for Gastrointestinal Endoscopy), BridgeLine Digital (American Academy of Pediatrics –AAP ), and Moodle/Totara (American Society of Anesthesiologists -ASA).
Innovations - enhancements being used and tested
Confidence ratings are being adopted in more assessments as they are linked to more reflective learning... Several organizations have used confidence ratings to help learners identify their greatest weaknesses with the theory that content areas where individuals had high confidence but scored poorly represent a learners greatest practice gap.
The ASA outlined how they provide learners with a score for each question that combines confidence ratings with correctness - learners receive a higher positive score when a question is answered correctly with confidence (up to 300%) and a lower negative score when a question is answered incorrectly with a high degree of confidence (up to -600%).
The AAP provides learners with various reports:
1. specific content areas are grouped together and sorted by confidence ratings scores;
2. confidence ratings are grouped together based on how a learner responded (ie, 100% confident answered correctly, 50% confident answered correctly, 0% confident answered correctly; 0% confident answered incorrectly, 50% confident answered incorrectly and 100% confident answered incorrectly (greatest gap); or
3. performance report categorized by specific content area showing average percent answered correctly and average assigned confidence rating per learner.
Peer-benchmarking (Comparison with peers) is oftentimes provided to learners so they can compare their performance with their peers. This is being done at both the ASA and the AAP. Oftentimes practitioners are able to see how their answer to a question compares to other practitioners; residents can see how their performance ranks compared to other residents. (The AAP provides all of their member residents with complementary access to their flagship self-assessment).
Discussion groups - The ASA is enhancing their self-assessment quizzes with online discussion forums.
Build your own assessment – The ASA has functionality in place to allow learners to build their own self-assessment based on topics, keywords and a preview glimpse of the question. A critical element for this product is the development of a sizable well-organized database of tagged questions.
Resources - While some organizations hyperlinked to external articles, other organizations only hyperlinked to their own materials.
IACME – SIG preparing for the self-study
8.4.2016 - Conference call highlights - Ms. Mindi Daiga facilitated this call
Nine simple things that make a big difference
1. Work ahead on the self-study before the files are selected; not that much changes in the 18 months before its due. (little changes over time)
2. Invest in a good Adobe Acrobat tool and be familiar with its features to consolidate everything. (Save sections in separate folders – break down into smaller chunks)
3. Use a snipping tool to grab relevant screen shots (or parts of screen shots) – helps focus in on relevant piece of a document.
4. Clearly label your responses – label your sections.
5. Make sure you answer the question being asked. If you’re not direct, it may suggest that you don’t know the answer/ or how to answer the question. Rephrase/restate the question that is being asked in the first sentence or two of your response. (Alternatively, it’s a good idea to copy the entire outline in your self-study).
6. A good/ clean self-study may give the impression that your program is organized. If your self-study is not professional looking and is hard to follow, it may give the surveyor cause to look more closely at the accreditation files and details.
7. Important to be succinct: 3 or 4 sentences often provides the key points!
8. Ask someone else to proofread the self-study, even someone who may not be actively involved in the CME circle of your organization.
9. Check out the Compendium of Case examples on the ACCME’s website – examples of compliance and non-compliance.
When telling the story of how you develop two CME activities – Best to give all of the details for one activity at a time– don’t comingle your ideas. If possible, you might include different types of examples/activities (i.e. live and enduring) to showcase more. This area is to demonstrate your overall process (not to include every single form collected)
For performance in practice structured abstracts --There is a tutorial on the ACCME website under education, with several examples of completed forms.
Be careful and intentional in how you phrase information on the PIP structured abstract. Pay close attention to the ACCME’s examples. Language need not be complicated. The more direct, the better.
Criteria 7 has the most non-compliance – common mistake is using the wrong definition of a commercial interest – important to use the complete definition of a commercial interest.
The next call for this group will be scheduled in 2017. Watch for future dates.