Introduction

The literature shows evidence that the use of technology in both primary and secondary education results in positive learning outcomes. Likewise, the literature also shows comparable educational results when technology is used to educate professionals. Pannabecker [1], studied college level learning experiences of various majors and concluded use of technology in education contributed to substantive learning. Additionally, the literature show evidence that learners benefit from the use of technology in education and technology can be successfully used to educate professionals in various professions [2-5]. Finally, the literature show evidence that demonstrate achievement of learning objectives and even improved learning outcomes with the use of interactive learning modalities compared to the use of traditional teaching methods [6]. This project involves the implementation of a new computer based educational program to medical device professionals with varied educational backgrounds, ages and computer skills.

Adults learn best when education is presented in a self-directed (learner controlled) and purposeful manor [7]. Additionally, adults connect the importance of learning to social responsibilities such as employment or the need to create a solution for a problem [7]. Active learning involves the learner obtaining new knowledge, then putting it into action. This type learning connects information received to how it is applied for a specific use [8]. For example, a student reads instructions on formatting an object in Microsoft^® Word, then practice formatting an object (ex. A triangle) using the information gained from reading the instructions for this skill.

Method

This educational process improvement project was completed at Medtronic, Inc., in Fridley, Minnesota. The learning outcomes focused on preparing New Field Employees (NFE), an employee who worked out of a home-based office and assigned to support business activities in a specific geography, to use a computer based Learning Management System (LMS) to complete required Intrathecal Drug Delivery Systems (IDDS) training and education. Participants included both domestic and international employees. To attend this program, international employees were required to be fluent in English. The educational background of the NFE included some of the following: A sales representative position required a Bachelor Degree in Business Administration, Biology, Life Sciences or a related field is a standard educational degree requirement for Sales Representatives [9]. The Clinical Specialists standard education and skills include proven history showing 6 years of patient care/clinical experience with Associated Degree or 4 years of patient care/clinical experience with a Bachelor Degree or 2 years’ medical sales experience with a Bachelor Degree [9]. As a result of this process improvement implementation, stakeholders identified three desired outcomes: Complete home study assignments prior to attending the mandatory two week face to face course, complete daily in class tests and documentation of job competency activities, which new employees were required to performed during field training assignments.

To evaluate this process improvement program a multiple group, time series approach is utilized. There are two groups: the project group, which consists of NFE who used the LMS for their training and the comparison group, who completed their training prior to implementation of the LMS and therefore used printed materials (traditional method). The project group (75 participants) consists of NFE hired into the training programs in 2012 during March, May, July, September and November. The comparison group, educated using the traditional method, is composed of 93 NFE participants who completed one of the Medtronic Intrathecal Drug Delivery System (IDDS) training courses from November 2011 through February 2012. The curriculum plan includes a minimum of 2 weeks to complete home-study assignments, and a 2 week (10 day) face to face classroom didactic instruction with practicum activities followed by specific field training competencies. All face to face sessions were conducted in the same location where presentations were delivered from 9 am to 12 noon, and practicum (hands on practice of training content presented in the morning session) occurred between 1pm to 4:30 pm. Also, a daily quiz is administered at the beginning of each day in week one on days 1 through 5, then a final exam on day 3 of week two. Additionally, on the last day of class, students complete a survey which ask questions regarding use of the assigned technology and electronic resources used in the IDDS training course. Finally, I, being the sole educator during the periods in this study, have access to historical information analyzed for this project.

Implementation Methods

To begin the project, initial meetings occurred between the course educator, business IT, instructional designer, program coordinator and SABA administrator. The course educator managed the curriculum and identified required course content (Reading materials, tests, handouts) to be converted into the LMS. The Instructional designer formatted designated materials provided by the Course Educator into the LMS. The system administrators deployed a testing environment, which allowed both the instructional designer and course educator to test content for accuracy and system functionality testing. Prior to implementing the program to all new employees and to assess and correct steps in the work plan, one new hire group received the curriculum via the LMS as a test group. To set up newly administered company personal computers and access the LMS, each class received printed directions by mail prior to the first onsite orientation or as a handout during the onsite company orientation. Also, participants received a follow up email which provided a direct link to the PPT training of the LMS instead of the conference calls conducted for the test group.

The NFE used several forms of mobile technology during each work day. The required mobile technologies included use of a laptop computer, iPad, and smartphone delivered directly to the NFE’s home prior to face to face internal computer training. The delivery of all devices contain general set up instructions, however, detailed electronic training is conducted during the initial in person company orientation in Minneapolis, Minnesota. Additionally, each NFE is assigned a field trainer whose responsibility includes basic orientation to assigned technology tools and ensuring all field curriculum activities are completed. Upon accessing their company issued personal computers or iPads, the NFE completed the assigned home study readings and field activities in preparation for the face to face training. The face to face course content is stored on secure Medtronic distribution service but accessible to NFE once they entered class. Next, The NFE will complete daily tests accessible through the LMS and finally a survey of participants will be administered at the beginning of training to evaluate their technology use and perceived level of competence.

Results and Analysis

Test results from three classes of both the Traditional and LMS NFE groups were analyzed. The LMS Group contained 75 participants and the Traditional Group contained 93 participants. The data were organized to determine test mean scores and standard deviations. Next the data were placed into stem leaf plots to evaluate frequency distribution scores for each tests in both the traditional and LMS groups (Appendix A and Appendix B). The comparisons between the LMS and the traditional tests results were performed by calculating a pooled standard deviation and the standard error of the mean differences. The pooled standard deviations were then used to obtain a t test comparison for each test. An alpha value of 0.05 is the established significance level and a two tailed t test analysis is conducted between the 6 tests pairs to determine if there is a statistical difference in either direction for each pair. The in-class qualitative data were obtained through observation of student behaviors during the face to faces sessions of both the test and study phases.

A Survey Monkey tool was administered to all NFE in the LMS group (75), and 30 participants responded. The responses to online surveys are noted to be significantly lower compared to in class hand completed surveys. The on line surveys were sent to NFE after they completed the course, therefore it is estimated that this delay may have affected the response rates. It is more challenging to complete the NFE survey once they return to their districts due to the high demand of the job and the required learning activities. The total NFE who replied to the online survey were 26 out of the 75 NFE who made up the project group. The qualitative data were obtained by observing NFE classroom interactions with computers, internet connectivity, tests completion time and ease of system use during the face to face testing period. Furthermore, qualitative data were obtained using Survey Monkey, an online survey tool, electronically delivered to NFE at the end of the face to face IDDS course (Appendix C). The survey administered at the end of the face to face course is entitled TDD course survey and is used to analyze NFE perceptions on the use of technology and eLearning tools in this course. The detail questions and responses to the TDD end course survey can be viewed in Appendix D. I acknowledge that in this project I am the only observer of behavior during in class activity and during the historical recall, therefore, inherently some observations may be having been missed and represents a weakness in this report.

Results and Interpretation

Five of the six test comparisons showed no statistical difference and thus the null hypothesis could not be rejected. However, the mean test scores between the Bridge Bolus tests showed a statistical difference and thus the null hypothesis is rejected for this test comparison. The mean tests comparison results are displayed in (Table 1).

The design of each test varied but included a combination of multiple choice and fill in the blank format. The fill in the blanks questions required either text or numerical entries and there were between 10-20 questions for each test. I also observed participant’s test completion times ranged from 15 minutes to 1 hour. Students who required connectivity assistance occasionally exceeded the system launch time thus requiring the IT support team to reset the test. However, each group required notably less assistance with each test and they demonstrated increasing comfort with successful launches. We also notice that the NFE expected the LMS to navigate quickly and similarly to other electronic devices they were assigned. This discovery was identified by our IT professionals who determined that NFE on occasion thought the system did not recognize the launch command thus the NFE re-launched the test. The re-launched test action was recognized as an attempt to access the test, therefore, exceeding the preset number of attempts and consequently locked the test, preventing the student access to complete the test. To prevent unwanted lock outs, NFE were instructed to be “patient” and allow the system to completely launch. The launch speed was a limitation as part of the design and system capacity purchased by Medtronic, thus no technical correction is available at the time of the project.

The participant behaviors observed during testing included activities seen by those completing tests with traditional tools. Such behaviors included completing test then exiting the testing room as instructed. Unexpected behaviors involved students discussing test results with other students who remained in the testing room while one or both students were testing. In an effort to maintain test integrity, the following instructions were emphasizing: “Participants are not permitted to discuss tests results among each other in the testing room”. Furthermore, each student has the opportunity to repeat tests if they do not achieve a passing score; therefore, a “No talking during testing” rule is implemented to maintain exam integrity. The online survey results contained comments consistent with continued IT challenges which occurred during in class testing. Furthermore, three themes were identified from the online survey. The three themes are:

• System “Glitches” were frustrating

• Online home study helpful but quantity of online felt overwhelming

• Online assessment questions could be more challenging

A Survey Monkey, an online survey tool, was used to obtain anonymous responses regarding participant’s evaluation of the course training with use of the LMS. Results were mixed regarding satisfaction with LMS ease of use. The test group (Feb 2012) revealed a required enhancement to increase connectivity during simultaneous testing. Participants reported LMS was not easy to access in the class. However, the students were observed to access the LMS tests by the 3^rd day with minimal or no assistance. Detail responses to survey question: “Please rate your online learning and assessment experience” are intended to obtain feedback on using the LMS and can be found in Appendix D at the end of the paper. To evaluate initiation of the IDDS education plan, I examined the communication process for delivery of home study materials, monitored the spread sheet used to document the delivery of the welcome letter which contained instructions and electronic links to the assigned LMS. I also met weekly with the course coordinator who maintained the spread sheet and emailed the planned communication to all NFE.

Next, during testing I remained in the classroom to monitor connectivity, assess participant’s behavior and determined computer response time to display test results. To monitor connectivity, I observed participant’s ability to successfully connect to the Medtronic secure intranet with or without the help support from myself or other IT staff in the room. I recorded notes regarding connection success or challenges for communication to the IT team as needed. Additionally, I observed participant’s behavior displayed while testing. These behaviors where considered activities that were outside what would be done while taking a printed test. Examples of these behaviors include obviously leaning to see one’s neighbor computer screen and ask a question of them versus asking questions to the course instructed as directed at the beginning of the testing. Another example involves students showing other students their results on the screen immediately after completing the test. A final observation involved assessing how quickly test results would display after participants selected the submit button.

From the LMS test group two areas of enlightenment regarding the IT infrastructure and the LMS internet functionality were realized. First, the building used for this project was built in 2001, just prior to the explosion of technology and subsequent assignment of multiple wireless devices to field employees. Consequently, the increased use of multiple mobile devices over extended the network capacity in the education building. The increase use of multiple devices necessitated that participants disabled wireless devices not in during testing to minimize the amount of wireless traffic which lead to the intranet overload due to multiple class administering online test via the LMS simultaneously.

Discussion

Although there is no evidence to guide specific methods for education using technology, the basic principles of pedagogy remain the blue print to designing an education program. The innovation of technology into education provides efficiency, relative cost savings and a creative learning environment which can occur anywhere at any time. The evolution of technology continues which requires movement of all direct or indirect health care personnel become both computer and information literate. Furthermore, the potential benefit of technology in patient safety and quality has increased policies promoting technology adoption with medical documentation and decision support in all areas of health care. This transition creates a domino effect among direct health care providers and all associated businesses relate to various health care entities. Additionally, Registered Nurses, the largest direct patient care group, and non-nurse medical industry professionals working in the health industries must shift previous non-technology thinking to the new concepts which incorporate technology into everyday healthcare life (Appendix E).

The implementation of the LMS in the training program has changed the accessibility of training content to the new employees by providing a convenient access when and where a participant needs the content. The course content in the LMS is accessible from all their assigned technology thus the educational resources are now the convenient for this highly mobile group. Strengths of this program include addressing a problem identified by the stakeholders, simple data collections, feasibility and ground work set for future system upgrades. Stakeholders identified training record management and method of education delivery as an area for process improvement which could be enhanced by implementing a LMS to all new field employees in the Neuromodulation division. Financial authorization to upgrade the method of delivering new hire education to an electronic format triggered the start of this program.

Data collection is simple due to electronic entry by participants into the LMS; in turn this information is accessible by the program evaluator. Additionally, comparison data of non-computer based test scores were archived using a scanner converting records into an electronic form. The records are stored on the company secure network server. Records are easily accessible by authorized personnel in the training department; therefore, this information can be collected for this project and other improvement process evaluation as they occur.

The comparison group, post-test design is a feasible method for evaluation because all new employees are required to receive the program. Additionally, there are variables such as prior use of a LMS, computer use skill set and experience with on-line courses. These variables are not controllable and have the potential to moderate the participants’ use of the Medtronic LMS. Furthermore, this design is feasible since all participants will be in the same room during testing allowing the instructor/evaluator to observe test completion, time to displayed test results and system connectivity during face to face training. Finally, all new hire field employees are assigned home study in the LMS; therefore, this data is easy to obtain by the evaluator.

Limitations to this design include uncontrollable variables, unfeasibility to conduct randomized interventions and potential for bias introduced by the evaluator. As a result, the exact effects of the intervention are difficult to quantify and attribute to a specific intervention. Variables experienced by participants will vary and can include prior computer skills, use of LMS with prior employers and general computer skills. These variables can have either a positive or negative effect on the participant using the Medtronic LMS.

The transition of educational tools from traditional printed materials to an electronic format introduced new terminology such as computer based training, technology based training and online training into the learning environment [10]. As a consequence, Americans born after 1990 have never known a world without technology and its presence is expected in every facet of life. Currently, the twenty first century is known as the age of technology innovation and it is shaping both how we learn and perform most any job regardless of the educational need to perform the job. And even though health care did not lead in the innovation of technology, all areas of the health care industry must embrace the change and find ways to both effectively and efficiently integrate innovative educational resources into education and training curriculums.

The learning environment is no longer confined inside hard wall structures instead occur in flexible environments such as coffee shops, cars or where every there is an internet access. Use of technology also changes educational materials into interactive, accessible anywhere content allowing learning to occur at any time of the day and instantaneously. This project demonstrates the use of technology can be effectively used in the educational process. Furthermore, a clear strategy or educational process must be included in the curriculum to ensure a successful learning experience for the diverse experience of adult learners.

Table 1: LMS and Traditional Method Mean Test Scores.

References

1. Pannabecker JR (1995) For a history of technology education: Contexts, systems, and narratives. Journal of Technology Education 7: 43-56.
2. Ishtaiwa FF, Abulibdeh ES (2012) Asynchronous e-Learning tools on interaction and learning in a blended course. International Journal of Instructional Media 141-159.
3. Garland K (2010) E-Learning vs. classroom instruction in infection control in a dental hygiene program. Journal of Dental Education 74: 637-643.
4. Feeg VD, Bashatah A, Langley C (2005) Development and testing of a CD-ROM based tutorial for nursing students: Getting ready for HIPPA. Journal of Nursing Education 44: 381-386.
5. Thompson TL (2010) Simulated learning experiences in Health Care. Clinical Nurse Specialist 24: 279-280.
6. McCormick MJ, Romero de Slavy J, Fuller B (2013) Embracing Technology: Using an unfolding case simulation to enhance nursing students' learning about Parkinson Disease. Journal of Neuroscience Nursing 45: 14-20.
7. Bastable S (2008) Nurse as educator: Principles of teaching and learning for nursing practice (3^rd Edition). Sudbury: Jones and Bartlett.
8. Bonnel WE, Smithe KV (2010) Teaching Technologies in Nursing and The Health Professions. Beyond Simulation and Online Courses. New York: Springer Publishing Company.
9. Medtronic Inc. (2012) Careers.
10. Sangra A, Vlachopoulos D, Cabrera N (2012) Building an inclusive definition of e-learning: An approach to the conceptual framwork. The International Review of Research in Open and Distance Learning 13: 145-159.