by Sarah A. Pierce, Pharm.D., PGY1 Pharmacy Practice Resident VA Maryland Health Care System
Most pharmacy residents are familiar with the use of ResiTrakTM to complete self-evaluations, an arduous process made more difficult by having to recall performance over a long period of time. Is this method of self-evaluation effective?
In both pharmacy education and residency training, self-assessment is a commonly utilized tool intended to encourage a learner to evaluate his or her performance, identify strengths and weaknesses, and note areas for self-directed learning and growth. In its accreditation standards for Doctor of Pharmacy programs, the Accreditation Council for Pharmacy Education (ACPE) discusses the importance of self-assessment for students, faculty, and staff.1 The theme of self-assessment and self-directed learning persists into post-graduate residency training. The American Society of Health-System Pharmacists (ASHP) includes “resident self-assessment of their performance” as a requirement in their accreditation standards for PGY1 pharmacy residency programs.2 The ASHP standards require “summative evaluations” at the end of each learning experience (aka “rotation”) and encourages optional spontaneous “snapshot” self-evaluations too.
Implicit in these requirements is an assumption is that self-assessment is valuable and accurately reflects a person’s strengths and weaknesses. However, according to Eva and Regehr, there is substantial body of literature which suggests that learners often cannot accurately self-assess their strengths and weaknesses and that self-assessment correlates poorly with actual performance.3 However, there may be a distinction between self-assessment and self-monitoring: “self-assessmentas a cumulative evaluation of overall performance, and self-assessment as a process of self-monitoring performance in the moment” [emphasis added].3
Eva and Regehr discuss the results of two studies which explored self-monitoring and self-assessment.3,4 In each study, participants answered sixty trivia questions divided into six categories. Participants were asked to evaluate their performance at different times during the testing. To measure “self-assessment” (that is, a cumulative evaluation of overall performance), the researchers had participants predict their overall score for each category both before and after completing all ten trivia questions in that category. To measure “self-monitoring” (that is, an evaluation of performance while in the moment), the researchers had participants rate their confidence in a given answer immediately after answering the question. The results showed that a “self-monitoring” measure demonstrated a higher correlation with actual performance when compared to the cumulative “self-assessment” measure.
They hypothesized a potential explanation for these findings. Self-monitoring likely requires a “fundamentally different cognitive process” than self-assessment. With self-monitoring, learners have many inputs and sources of information at their disposal to predict potential success or failure on a moment-to-moment basis. However, with self-assessment, the learner must rely on memory to aggregate information of multiple past events in order to determine the overall success or failure.3The concept of self-monitoring was replicated and expanded in work by McConnell and colleagues.4
What are the potential implications of these findings? In the Educational Theory and Practice course, the idea of self-directed learning was introduced.5 Self-assessment is a tool used to facilitate self-directed learning. However, if self-assessment is not as accurate as one may hope, then perhaps self-assessment is not the best tool to facilitate self-directed learning. I would argue that more attention should be directed to self-monitoring on a moment-to-moment basis, rather than on cumulative self-assessments. Through self-monitoring, individuals would develop a more accurate picture of their abilities and this could lead to more focused self-directed learning needs. As self-assessment is by far the most common self-evaluation tool used in pharmacy education and residency training today, new, creative ideas to transition to self-monitoring are needed.
Self-monitoring could be implemented in pharmacy education and residency training in several ways. In pharmacy school, an early and consistent emphasis on self-monitoring could prove to be more effective than intermittent, reflective self-assessments. By asking students to identify their strengths and weaknesses in real-time, this may motivate them towards focused self-directed learning. For example, students taking an exam on the pathophysiology of diabetes, the pharmacology of diabetes medications, and diabetes management could be required to rate their confidence on each test question. After the exam, students could receive a report with their self-monitoring responses and a breakdown of their actual performance in each domain (e.g. pathophysiology, pharmacology, and patient management). In this way, students would be better at identifying areas they struggled with, gradually improve the accuracy of self-monitor their performance, and note areas that require further study. Regularly repeating this process may help students develop stronger self-monitoring skills and help them become independent practitioners after graduation.
A similar argument could be made for pharmacy residency programs. I believe “snapshot” evaluations should be used more frequently. During my internal medicine rotation, I was asked to complete a snapshot evaluation related to my data gathering skills and treatment plan for a specific patient. This was much more focused than the summative self-evaluation done at the end of my rotation whereby I had to assess my overall performance related to several different goals and objectives. I believe I gained more insight into my strengths and weaknesses with the snapshot evaluation. I was able critical examine my performance on narrow set of tests “in the moment” rather than having to search my memory for past events that related to my performance.
While self-assessment is certainly a necessary component of pharmacy education and helps facilitate self-directed learning, I believe there should be a greater emphasis on “real-time” self-monitoring. Self-monitoring should be taught and required in Doctor of Pharmacy programs and frequent self-monitoring “snap shots” should be a mandatory component of pharmacy residency training.
A similar argument could be made for pharmacy residency programs. I believe “snapshot” evaluations should be used more frequently. During my internal medicine rotation, I was asked to complete a snapshot evaluation related to my data gathering skills and treatment plan for a specific patient. This was much more focused than the summative self-evaluation done at the end of my rotation whereby I had to assess my overall performance related to several different goals and objectives. I believe I gained more insight into my strengths and weaknesses with the snapshot evaluation. I was able critical examine my performance on narrow set of tests “in the moment” rather than having to search my memory for past events that related to my performance.
While self-assessment is certainly a necessary component of pharmacy education and helps facilitate self-directed learning, I believe there should be a greater emphasis on “real-time” self-monitoring. Self-monitoring should be taught and required in Doctor of Pharmacy programs and frequent self-monitoring “snap shots” should be a mandatory component of pharmacy residency training.
References
1. Accreditation Council for Pharmacy Education. Accreditation standards and guidelines for the professional program in pharmacy leading tothe doctor of pharmacy degree. Chicago: Accreditation Council for Pharmacy Education; 2011. 97 p. [cited 2012 Oct 8]
2. ASHP Commission on Credentialing. ASHP accreditation standard for postgraduate year one (PGY1) pharmacy residency programs. Bethesda (MD): American Society of Health-System Pharmacists; 2005. 23 p. [cited 2012 Oct 8]
3. Eva KW, Regehr G. Exploring the divergence between self-assessment and self-monitoring. Adv Health Sci Educ Theory Pract 2010;16(3):311-29. [cited 2012 Oct 8]
4. McConnell MM, Regehr G, Wood TJ, Eva KW. Self-monitoringand its relationship to medical knowledge. Adv Health Sci Educ Theory Pract 2011;17(3):311-23. [cited 2012 Oct 8]
5. Kaufman DM. Applyingeducational theory in practice. BMJ 2003; 326: 213-6. [cited 2012 Oct 8]