Introduction

Over a period of more than 10 years, medical education in general and nursing education in particular has changed substantially as a result of official decisions concerning academic accreditation. Consequently, nursing educators must ensure that students are suitably supported towards the demonstration of professional practice. Making assessment of this evolving practice is challenging and student portfolios may offer an opportunity to capture this evolving practice more appropriately than traditional examinations alone. The use of student portfolios has increasingly been used in medical education as a tool aimed at encouraging effective learning and assessment [1,2]. This study, using a case study approach, explores a group of undergraduate nursing students’ perceptions about portfolio use in education and about using them as assessment and learning tools in clinical placements in one hospital in Saudi Arabia. The current study may contribute to guiding faculty in nursing education in exploring the importance of using portfolios as a means of evaluating learning and teaching. The aims of this study were to explore the nursing students’ perceptions on the process of using a portfolio in their clinical education as well as their perceptions on the use of the portfolio as both an assessment and learning tool in clinical placement.

Background

Several assessment methods and learning methodologies have recently been adopted to assess the core competencies that students must possess by the end of their academic studies, to ensure safe nursing practice [3,4]. One of the learning methods and assessment strategies utilized by nursing education are student portfolios [5,6]. There are seen as providing an opportunity to encourage continuous assessment that has been integrated into nursing education and students, therefore introduced to the portfolio method early in their education. This provides one way to continuously encourage professional development, aiming to enhance cognitive thinking so that as future nurses they are more analytical, and critical [5]. A well designed portfolio involves the active processes of collecting, synthesizing, and organizing possible relevant items that most effectively provide evidence of learning achievement for an assessment process [7]. In addition, portfolios can enhance learning, as students are encouraged to reflect on their experiences, identify learning needs, and initiate further learning [8,9].

Portfolio-based learning has been increasingly implemented in a variety of educational and professional learning contexts [10]. Specifically, it has been utilized to stimulate and monitor students’ professional development and promote their ability to become lifelong learners [6]. They are often characterized as a collection of students’ work and achievements during their academic careers [9]. Increasingly portfolios have been utilized in higher education, as the sector seeks new teaching and learning assessment methods that promote students’ confidence, motivation and self-esteem as managers of their own learning environments [11].

These developments are reflected in medical education [12] and the use, for example, of portfolios to demonstrate learning in postgraduate nursing [13]. The use has been underpinned by a rationale that they can stimulate reflection among students [10]. However, Gannon, et al. [14] reported that there is confusion about how portfolios should be used in educational and professional learning environments and whether or not portfolio content can be used as an assessment instrument [14]. This is complicated by the fact that a student portfolio can take many forms and can be tailored depending on their purpose and an individuals’ needs. Zubizarreta [15] indicates that a learning portfolio becomes most useful when a student increases the quality of their evidence rather than, as can happen, the quantity. The learning portfolio may hold a reflective description or several short writings or longer narrative that outlines learning development [16]. An important element regarding the selection of evidence and the construction of the portfolio for a student may be the support received by a mentor or a teacher.

According to Zubizarreta [15], a simple model for a learning portfolio is to base them on the following three elements: reflection, documentation, and collaboration. Deeper learning, first identified, is most clearly demonstrated when these three elements are integrated and the portfolio moves beyond the mere description of practice. However, research has illustrated that preparing and completing student portfolios can be difficult and resource-intensive for both students and faculty. They take time to construct for the learner and significant time to prepare and review by faculty [17].

Meaningful reflection can be enhanced by utilizing the advantages of collaboration and mentoring, creating a suitable learning community of practice for students (Lave and Wenger reference on communities of practice). However, identifying portfolio mentors is not straightforward and raises a series of potential questions: Should the coach be the course teacher? Should trained students be enlisted as peer mentors? Can professionals or faculty advisors serve as guides? Can students developing electronic portfolios turn to information technology staff for mentoring resources? Are there ways to use interactive technology tools to provide virtual collaboration opportunities? In addition, context, resources, purpose, and other factors all play a role in helping find ways to connect students to knowledgeable and effective mentors who can assist them in cultivating substantive reflective judgment and the analytical skills needed to develop a purposeful portfolio [15].

Methodology

Methods

A descriptive and exploratory research design was used to elicit undergraduate nursing students’ perceptions about using portfolios for clinical learning and assessment. To gain the data a modified Portfolio for Clinical Practice Learning and Assessment tool (PCPLA) was used in this case study. The permission to adapt the tool was obtained by e-mail from the author Brennan and Lennie [6] (Appendix II). The PCPLA tool consists of three parts; the first captures demographic and educational details. The second part is the questionnaire, which identifies three specific portfolio-related domains as following: Domain I: "Student perceptions about the portfolio building experience". Domain II: "Student perceptions about using portfolios as an assessment tool". Domain III: "Student perceptions about using portfolios as a learning tool". The third part is an open-ended question for the students to write narratives. The questionnaire scores were reversed for questions 1, 3, 4 and 9 (Reverse scoring means that the numerical scoring scale runs in the opposite direction. So, strongly disagree would attract a score of 5 instead of score 1, disagree = 4, and so on. See Appendix I). The reliability and validity of the modified tool was tested. Reliability was assessed by internal consistency (Chronbach’s Alpha) that were 0.73, 0.89 and 0.77 for the three parts of the questionnaire respectively. A pilot study was carried out using ten students to test the validity and clarity of the questionnaire. The questionnaire was then modified by rephrasing some statements which had been reported as lacking in clarity (Appendix 1).

The study was conducted in the female section of the College of Nursing at one University in the Eastern Provence in Kingdom of Saudi Arabia. It was based on one single case study [18]. According to Anthony & Jack [19], “Case study is becoming entrenched in the nursing research lexicon as a well-accepted methodology for studying phenomena in health and social care”. Ethical approval was obtained from the Deanship of Scientific Research/the Institutional Review Board at the University. Ethical principles were adhered to in the study with participants able to withdraw at any stage and all completed consent forms. All current undergraduate female nursing students who agreed to participate were included. Male nursing students were excluded from this study because the male section in the College of Nursing had recently been opened and the student portfolio was not yet being used. A total of 244 female students participated. This included groups of second- (n = 39), third- (n = 91), and fourth-year students (n = 114). This study was conducted at the end of both the first and second semesters of the academic year, which involved students from all levels. A hard copy of the questionnaire was given to each student, which was collected and returned to the research assistant after completion.

Data were coded, entered, and analyzed using SPSS version 19.0. The scores were redistributed by summating strongly agreed with agreed and strongly disagree with disagree. Qualitative variables were presented as number and percent and a Chi square test used to compare the differences. Quantitative variables were summarized using means and standard deviations and median. The inter-quartile ranges and Krucal Wallis test and Man Whitney test were used to compare the differences. A Spearman correlation was used to test the association between the quantitative variables. Regarding the qualitative part of the data, The MAXQDA software (version 2007) was used in to facilitate the coding system and helped in data management. The data was analysed using a thematic approach using a thematic constant comparative method of data analysis [20]. The data was read over the transcripts for many times to gin comprehensive meaning behind participants’ experiences and to ensure that any potential themes were grounded in the data.

Findings

The total number of female undergraduate nursing students participated in this study was 244. Thirty-nine students from the second year (Y2s), ninety-one students from the third year (Y3s), and one hundred and fourteen from the fourth year (Y4s). As shown in (Table 1), the students reported spending an average of two hours per day preparing work for their portfolios. Students’ average daily hours spent preparing portfolios is 2 hours. Students also indicated that they spent an average of 0-6 hours a week working with a mentor, with a median of 1 hour. The median cost of portfolio preparation 100 Saudi Riyals (SR).

Portfolio Development

(Table 2) showed a majority (84%) of female nursing students agreed that portfolio development involved writing many papers. More than half (55%) of the students agreed that portfolio building took excessive training time, and this significantly increased as the study-year level increased (P < 0.001).

The students said portfolio “Consume time, money and effort”.

Wasting Time

Wasting time is the expression used by most of the students in all level even the ones who supported the presents of this file.

“It consumes most of my time just to prepare this unusual file” (Y2s).

Effort and Reflection

The expression used by most of the students in all levels, even those who were generally more positive felt they were generally “Wasting time”. The main reason given was that the amount of work spent to develop the portfolio after each clinical rotation was perceived to outweigh the benefit and they felt that the grades received did not reflect the amount of effort (Table 3).

Unfair

It was “Unfair” according to one of the students in her third year:

“First of all we have to buy folder, then print all material, we need ink, …all the effort will go because the clinical instructor will only look at it superficially and we do not read what is inside, all my university money award will go without benefit” (Y3s).

A fourth year added that the portfolio involved:

“Wasting time and effort, it has no meaning except as a way of collecting some papers which supposedly have to be graded. They have to evaluate the self-reflection and put grade on them” (Y4s).

Reflection on Portfolio Journey

The Median overall satisfaction scores significantly differed according to graduation year. A concern is that the satisfaction with the portfolio appears to decrease over time from second to fourth year P = 0.02 (Table 2). The second-year students reported interest in portfolio building (35.9%), as compared to third- (22%) and fourth-year students (14.3%) (P = 0.02) (Table 4).

Confidence

Second year students highlighted how important they felt the portfolio had been in building their confidence: “I am confident to present my work” Y2s; “Portfolio preparation give me the feel of confidence and supported” Y2s.

A Painful Experience

However, by the third and fourth year it was more likely that students reported less positively, suggesting more frequently that it was “A painful experience”. Using portfolio caused stress and anxiety as the following statement illustrates:

“Most of the clinical instructors don’t know how difficult and painful for us to prepare and be on time for submission also there is inconsistency in giving the grads as well as the grading is not congruent with the effort and stress that made for the work to finish” (Y4s).

“The expense of buying all the colored paper as partition used in the portfolio added more stress financially on us” (Y4s).

Time Benefits

The more experienced students stated that they would prefer to utilize the time consumed in portfolio preparation in studying or doing other learning activities:

“Alternatively, we could use our time to study, share other educational activity than preparing portfolio” (Y3s)

More than a quarter of the students agreed that portfolios could increase learning, self-dependence, assess knowledge and help to identify practical deficits. Agreement with these two items was again significantly higher for second-year students (33.3%), than older students regarding the building of knowledge (P = 0.005) and learning skill development (P = 0.03). In addition, second-year students reported the highest level of overall satisfaction, relative to fourth- and third-year students, although these differences were non-significant. Additionally, median overall satisfaction scores significantly decreased according to graduation year; P = 0.03 (Table 5).

Sense of Satisfaction

Satisfied

The second-year students were more satisfied in terms of the portfolio keeping them aware on their achievements and providing an easy to access and organized format:

“Reflecting in the portfolio and putting evidences keep me tracing my own development” Y2s;

“All my assignments and clinical works are kept organize and easy to access” Y2s.

Year 3 and 4 students also agreed that the portfolio was useful for maintaining an organized record of their achievements.

Unsatisfied

A major concern raised by third- and fourth-year students was the lack of consistency between instructors. According to the students that each course had their own instructions regarding student portfolio preparation. It was not always clear why they had to prepare the file and if it was being used for assessment alone or additionally as a teaching tool. At the same time the students believed that each clinical instructor had her own way of grading, some of the clinical instructors were seen to look at the portfolios periodically whilst others did not look at them at all:

“The student portfolio doesn’t reflect student educational development, because the clinical instructors have their own way of assessment and grading” (Y3s).

“Not all clinical instructors concerned about reviewing the file but only gathering papers and prepare them in a file, which takes time and effort of the student to arrange and prepare properly” (Y4s)

Despite the concerns expressed, as illustrated in (Table 6), less than half of the female nursing students agreed that they preferred practical exams to the paperwork involved in portfolio building (42.3%). Again, it was second-year students who were more satisfied than other student's in terms of using portfolios to measure educational skill development and using portfolios for assessment and preference for using portfolios rather than written exams to measure practice development (P < 0.05). There were significant differences in agreement about overall satisfaction for the use of a portfolio in assessment. It was higher among second- (35.9%) than among fourth- (18.5%) and third-year students (13.1%) (P < 0.001).

Why We Prepare Portfolio

To Get Marks Only

Third and fourth year students questioned whether the reflections that portfolios are designed to encourage occurred. They argued that rather than being to do with learning achievement they were more to do with:

“Getting marks rather than learning from it. We could not see how we can learn from collecting papers only without feedback from our teachers” (Y4s).

Affect Learning Process

According to students that “Learning could be developed by clinical experience not by collecting papers” (Y3s)

Role Awareness

In addition, they also felt that the faculty were sometimes unaware of their role as mentors and collaborators with the student’s work:

“Not all instructors care about the quality, and there are requirements that are not necessary” Y4s and Y3s)

As a result, the students felt that the role of their teachers was limited: “The only thing the instructors do is sign the papers. We see that having written exam or clinical test is more effective than portfolio where their inconsistency in grading” (Y4s)

Discussion

Portfolios are used in healthcare education for a range of purposes, including supporting reflective practice, delivering summative assessment, and aiding knowledge management processes. In particular, portfolios are viewed as providing a key connection between learning at organizational and individual levels [21]. The current study aimed at exploring undergraduate nursing students’ perceptions about the use of portfolios as an assessment and learning strategy in one educational setting. In the theme 3 (Sense of satisfaction) of this study few students argued that the that portfolio kept them academically focused and on track. Rather than providing a connection from the individual to the organization, the portfolios were viewed as a painful experience (a painful experience) that added pressure for the majority of learners. This reflects other studies undertaken recently. For example, Birks, et al. [22] found that 51% of the students who used an e-portfolio reported feelings of "Uncertainty and confusion". It is evident that merely adopting a portfolio, paper based or an e-version, requires careful preplanning and thought. If this is not undertaken the risk is that, as we found, students believe that the time spent creating and maintaining the portfolio is mostly wasted. 

In addition, (Role awareness), students reported that they believed that most of the clinical instructors did not know how to use the portfolio to provide effective feedback. Additionally, they reported inconsistency in giving grades and felt that the grading was not congruent with the effort that they had made. This finding reflects the need for training of supervisors on the portfolio and providing effective feedback [11]. Joshi, et al. [10] said the "Assessment using portfolios can be further improved by using some uniform ways of scoring portfolios".

Teachers have previously reported several barriers to providing feedback to students. These include knowledge of the subject matter, a lack of skill about giving feedback, large students numbers placing pressure on time, and the fear of unexpected students’ reaction to feedback [23]. This study found that students (Y2s, Satisfied) using the portfolio for the first time were most positive about portfolio preparation. They argued that the portfolio enabled them to put their work in one place and that it was therefore easier for them to track their achievements. However, this positive experience reduced each year.

Less than half of the nursing students believed that the use of portfolios as an assessment tool adequately reflected their practical competencies (42.3%). The students’ negative perceptions and attitudes regarding building and using portfolios was explored further through an analysis of their written statements, elicited during data collection. Through these it was noted that portfolio building was perceived to interfere with clinical learning, due to the excessive amounts of paperwork required (Affect learning process). They felt the substantial amount of time invested was disproportionate, and that there was a need for additional information and instruction about the process so that the time could be used more effectively. The lack of clarity over the purpose of building the portfolio and the variations between instructors and courses left students with a negative experience and a number stated it left them anxious (Unsatisfied). This resulted in a lack of motivation and not valuing the potential importance of the portfolio for learning and assessment: “The clinical instructor did not assess my cognitive or practical achievement. This work I think only reflect my ability to organize my work not for assessment purposes” (Y4s).

The concerns raised resonate with findings from previous studies. For instance, Corcoran and Nicholson [24] found that students negatively responded to portfolio use. Specifically, the majority of students reported that portfolios did not motivate them to learn. They reported high levels of anxiety caused which may have had a negative effect on motivation. In particular, this anxiety might be related to the uncertainty of expectations and the time-consuming nature of keeping a portfolio [24].

In order to help each student, build up a learning portfolio that demonstrates the quality of a students’ learning, teachers must find ways of effective collaboration and mentoring. Zubizarreta [15] says “That mentors and collaborators are vital to the success of any learning portfolio project”. Without this support, students may be reluctant to engage in self-reflection and not value or have developed the knowledge or skills to do it. In addition, faculty may require help in developing and promoting students’ self-reflection and self-assessment abilities [23]. Teachers need to be trained and educated how to work with learners to ensure they are able to use the portfolio in constructive ways that demonstrate how knowledge and understanding have been applied and the implications [11]. One of the challenges teachers assigning learning portfolios face is to require their students to demonstrate the complexity of content and process knowledge, based on the values they bring and that guide and underpin the portfolio, and to provide an effective balance within the narrative [10].

Contrary to the results of this study, several studies addressing the same issue have found that students did develop positive attitudes about their portfolio. These studies report students who believed a portfolio provides an opportunity to incorporate theory with practice and increase personal responsibility for learning, supporting skill development and engagement in reflection [13,25]. In these cases, the portfolio preparation process yielded positive academic and affective outcomes. This difference appears to occur because the students were well prepared about the portfolio process, having regular and effective feedback from a mentor who had experience. Furthermore, "Awareness and understanding of the elements of feedback delivery can aid the process and ensure that both supervisor and student have a positive interactive experience” [11].

Limitation

The study only involved female student nurses and as such, there is potential that this is not entirely representative. It would be useful to extend the study to include the perceptions of male nursing students and faculty regarding the use of portfolios in teaching and learning. In addition, the questionnaire provided data that could be investigated further in dialogue that is more open with the students, through for example interviews of focus groups, to examine more closely some of the themes, such as anxiety, that emerged from the data.

Conclusion & Recommendations

Nursing students identified a range of issues and concerns regarding portfolio use. Specifically, they indicated that the intent of the process, guidelines for portfolio compilation, and students’ and faculty members’ roles require clarification. The findings of this study also suggest that portfolio paperwork should be kept at a manageable limit. Teachers supporting the development of the portfolio need to be aware of their role, be effectively trained as mentors and how to provide effective feedback. In addition, a clear portfolio structure was identified as one of the key ingredients contributing to the effectiveness of portfolio assessment by undergraduate nursing students.

The findings of this research highlight several important aspects for using student portfolios. One of these is the rethinking and bringing clarity to the purpose, process, and expected outcomes of a portfolio so that they can contribute as a valuable method of student assessment. Secondly, additional training about the purpose, process, and expected outcomes of portfolios should be provided to faculty to ensure consistency. Thirdly, there is a need to provide students with training in collaborative learning strategies, so that they can fully benefit from the mentor-based learning that might be initiated during portfolio preparation.

A student portfolio can be used to enhance learning, and previous studies have demonstrated this. However, this study identifies that the implementation needs to be carefully planned so that the emphasis is on learning that is identifiable and clearly understood by all those involved: students and faculty. If this is not achieved the portfolio development can lead to a lowering of motivation, added anxiety and frustration and therefore remove the very engagement it seeks to achieve through self-reflection. This is made worse if the administrative aspects are too onerous so that the time is less focused on reflection and relates more to the collection of evidence.

Highlights

·         The Portfolio-based learning had limited influence on students' professional development, self-dependence, and self-confidence.

·         Unless well thought through portfolio building interferes with clinical learning, due to the excessive amounts of paperwork required, the substantial amount of time invested, and the need for additional information and instruction about the process.

Conflicts of Interest

The author has no conflicts of interest to declare (no financial/personal interest). Potential conflicts don't exist.

Acknowledgements

I would like to thank Imam Abdulrahman Bin Faisal University/ Deanship of scientific research for their great support financially through funding this work (IRB -2014-05-198); As well as providing all resources to conduct this study. Also, I would like to thank Dr. Simon Lygo and Professor Ian Kinchin for their support in revising the manuscripts and providing help during submission of the research.

Appendix I and II

1.       Oudkerk Pool A, Govaerts M, Jaarsma D, Driessen E (2017) From aggregation to interpretation: How assessors judge complex data in a competency-based portfolio. Advances in Health Sciences Education: Theory and Practice. Advances in health sciences education: theory and practice 23: 275-287.

2.       Williams M (2003) Assessment of portfolios in professional education. Nursing Standard 18: 33-37.

3.       Paul S, Grunwald, Cristie, Bonner, Julia, & Musial, Diann (2010) Assessment of Critical Thinking in Nursing: How Should Critical Thinking Be Assessed in the Clinical Setting? ProQuest Dissertations and Theses.

4.       Warburton Darren ER, Bredin Shannon SD, Charlesworth Sarah A, Foulds Heather JA, McKenzie DC, et al. (2011) Evidence-based risk recommendations for best practices in the training of qualified exercise professionals working with clinical populations. Applied Physiology, Nutrition, and Metabolism 36: S232.

5.       Ticha V, Fakude LP (2015) ‘Reflections on clinical practice whilst developing a portfolio of evidence: Perceptions of undergraduate nursing students in the Western Cape, South Africa’. Curationis 38: a1502.

6.       Brennan K, Lennie S (2010) Students’ experiences and perceptions of the use of portfolios in UK preregistration dietetic placements: a questionnaire-based study. Journal of Human Nutrition and Dietetics 23:133-143.

7.       Cooke M, Walker R, Creedy D, Henderson A (2009) Clinical progression portfolio: A resource for enhancing learning partnerships. Nurse Education in Practice 9: 389-402.

8.       Endacott R, Gray M, Jasper K, McMullan M, Scholes J, et al. (2004) Using portfolios in the assessment of learning and competence: the impact of four models. Nurse Education in Practice 4: 250-257.

9.       McMullan M (2008) Using portfolios for clinical practice learning and assessment: the pre-registration nursing student’s perspective. Nurse Education Today 28: 873-879.

10.    Joshi M, Gupta P, Singh T (2015) Portfolio-based learning and assessment. Indian Pediatrics 52: 231-235.

11.    Clynes M, Raftery S (2008) Feedback: an essential element of student learning in clinical practice. Nurse Education in Practice 8: 405-11.

12.    Sidhu N (2015) The teaching portfolio as a professional development tool for anaesthetists. Anaesthesia and Intensive Care 43: 328-334.

13.    Tochel C, Haig A, Hesketh A, Cadzow A (2009) Medical teacher: The effectiveness of portfolios for post-graduate assessment and education: BEME guide no 12. Taylor & Francis 31: 299-318.

14.    Gannon F, Draper P, Watson R, Proctor S, Norman I (2001) Putting portfolios in their place. Nurse Education Today 21; 534-540.

15.    Zubizarreta J (2004) The Learning Portfolio: Reflective Practice for Improving Student Learning. Anker, Bolton, MA.

16.    Seldin P, Higgerson M (2002) The Administrative Portfolio: A Practical Guide to Improved Administrative Performance and Personnel Decisions. Anker Publishing, Bolton, MA.

17.    Hayes E, Chandler G, Merriam D, King MC (2002) The master’s portfolio: validating a career in advanced practice nursing. Journal of the American Academy of Nurse Practitioners 14: 119-125.

18.    Creswell J (2007) Qualitative inquiry & research design: Choosing among five approaches Sage, London.

19.    Anthony S, Jack S (2009) Qualitative case study methodology in nursing research: An integrative review. Journal of Advanced Nursing 65: 1171-1181.

20.    Glaser B, Strauss A (1967) Discovery of grounded theory. Chicago, IL: Aldine.

21.    Seldin P (2004) The Teaching Portfolio: A Practical Guide to Improved Performance and Promotion/Tenure Decisions 3^rd Edition Anker Publishing, Bolton, MA.

22.    Birks M, Hartin P, Woods C, Emmanuel E, Hitchins M (2016) Students' perceptions of the use of eportfolios in nursing and midwifery education. Nurse Education in Practice 18: 46-51.

23.    Moonaghi H, Vafaee M, Ildarabadi Es, Rice S (2016) Experiences of Medical Teachers about Methods. Types, and Barriers of Giving Feedback 6: 26-31.

24.    Corcoran J, Nicholson C (2004) Learning portfolios - evidence of learning: an examination of students’ perspectives. Nursing in Critical Care 9:230-237.

25.    Buckley S, Coleman J, Khan K (2010) Best evidence on the educational effects of undergraduate portfolios. Clinical Teacher 7: 187-191.