Validation and Reliability of the Comprehensive Difficult IV Access Scoring Tool

Objective: To assess the face validity and inter-rater reliability of the Comprehensive Difficulty IV Access (C-DIVA) tool for identifying difficult vascular access (DIVA) patients and directing care for clinicians without specialized vascular access experience (generalists). Methods : A prospective observational study was conducted using nurses experienced in vascular access to internally validate the C-DIVA tool. After validation of the tool, scenarios were provided to a vascular access specialist group for inter-rater reliability measurement. Results : A survey to assess face validity received responses from 7 nurses in active practice. Using a 1-5 Likert scale, the tool was rated at 4.43 for usability. Additional assessments by 12 nurses provided 72 independent assessments of case scenarios to establish inter-rater reliability. Intra-class correlation coefficient of 49.4% with 55.6% agreement was achieved. Conclusion : The C-DIVA tool was validated by a sample of nurses with vascular access experience from different clinical areas. The inter-rater reliability of the C-DIVA tool is moderate. The use of the tool should facilitate the identification of DIVA patients by generalist providers (those without specialized training or advanced skills in vascular access placement) to facilitate the allocation of appropriate clinicians and resources to achieve successful vascular access for patients with difficult access.


Introduction
In the mid-2010's more attention was given by researchers to patients who presented with difficulty in establishing venous access. The term DIVA (difficult intravenous access), first established by Keyes in the 1990s, defined DIVA as patients experiencing multiple intravenous catheter (IV) failed attempts [1]. At the same time, leaders in some health systems questioned whether DIVA was a significant concern and if there was a need to escalate to more skilled clinicians for successful IV insertion. A literature review was performed to validate these issues. Several papers referenced difficult access patients and nearly all identified those considered DIVAs by the number of failed attempts [2][3][4][5][6].
Starting in the 1990s, pediatric nurses recognized a need for tools to assist with identifying patients that would benefit from escalation when it came to vascular access [2,3,7,8]. Tools were later developed for adult patient populations undergoing chemotherapy or surgery [9,10]. These tools demonstrated methods to objectively assess and escalate patients to decrease failed attempts and gain vascular access expeditiously. However, a comprehensive tool for application throughout health systems and across the lifespan of patients has yet to be published.
Despite advances in research and technology and specialization of clinicians in vascular access, patients continue to suffer multiple venipunctures during the insertion of vascular access devices. Research studies and literature continue to identify patients after multiple attempts rather than via objective assessment. The Comprehensive Difficult IV Access Score (C-DIVA) was developed to rapidly identify patients with DIVA at the point of care and escalate when appropriate to increase the likelihood of successful first-time attempt and is usable in all patient populations. The tool is simple and easy to apply effectively in order to determine which patients are difficult and in need of further intervention.

Materials and Method
The C-DIVA tool was developed following a review of van Loon's Adult Difficult IV Access (A-DIVA) tool [10,11]. It was determined that the C-DIVA tool might be appropriate for use in emergency departments. The C-DIVA tool was integrated into use in a community emergency department (ED) and was later established as the process for device selection by the department's formal vascular access team [12]. The tool was taught to all ED nurses in huddles and displayed in prominent areas around the department. The tool consists of four categories that each score from 0-2. The categories include visible appearance of the vein, palpable sites, history of difficult access and clinical factors. A total score of 0-8 is calculated and used to rank the patient as low, medium, or high risk of difficulty. The higher the score, the more difficult the patient and the need for escalation. The intention was to keep the tool simple enough to be easily remembered and calculated without the need of a written form ( Figure 1).
The greatest difficulty in using the C-DIVA is the fourth scoring category: Clinical Factors. In van Loon's original publication of the A-DIVA score, the fourth category identified whether patients were having a scheduled surgery or an emergency procedure. It was found that this was a contributor to difficulty establishing vascular access and was incorporated into the tool. To generalize from that it seems to the authors that various clinical factors could function in a similar manner such as identification of clinical issues that would indicate need for more urgent care.
The emergency severity index (ESI) is widely used in emergency departments across the United States to triage or sort patients upon arrival, stratifying patients based on acuity and the number of resources needed for care. The ESI scores patients between 1-5 with 1 being those with the highest acuity and in need of the most resources and immediate care [13]. This would seem to correlate to increasing difficulty in a similar manner since presumably critical patients would be more likely to be in vascular collapse due to their condition and thus correlate to DIVA.
Utilizing ESI requires some ability of the user to recognize the clinical factors that would be associated with DIVA. In the ED the ESI and certain clinical conditions would be heavily tied to the Clinical Factors score. Stroke, acute coronary syndrome, and trauma, for example, would score high in this section. However, ESI is not used outside of the ED, thus to have a truly comprehensive tool it cannot simply be tied to the exigencies of a particular care area such as ED or OR. This led to us adopting a more generalized "Clinical Factors" category by including urgent and emergency conditions that would be able to be applicable across all clinical areas. In the ICU, the admitting diagnosis and condition of the patient would be used to score Clinical Factors. While in an outpatient infusion setting the medication type, length of treatment, or use of limb restrictions would be considered in this section.
To validate the C-DIVA tool, the authors contacted nurses with known experience in vascular access to participate in the survey. Seven respondents scored the tool on understandability, clarity, and usability using a 1-5 Likert scale. Four questions were asked including: a) Is the tool easily understandable by generalist clinicians? b) Is it unambiguous? c) Do you think the tool is usable by generalist clinicians? And d) Do you believe the tool would assist a generalist nurse to know when to call a specialist? ( Table  1).  The respondents were in agreement that the tool is understandable (85.71%) and unambiguous (82.86%). They were in agreement that the tool is usable by generalists (88.57%). The expert consensus was lower (82.86%) on whether the use of the tool would help improve vascular access first-time attempt success. Several stated skepticism about whether the tool would be used consistently or appropriately.
This survey represents the current state of practice reflecting subjective clinical expertise. Inter-rater reliability was measured by using six scenarios. Scenarios covered a variety of patients (pediatric to geriatric) in different areas (i.e. ED, OR, ICU). Twelve respondents at the 2022 Association for Vascular Access Annual Scientific Meeting were surveyed. The initial survey asked whether the scenario indicated a possible or absolute need to escalate the patient to the use of technology (either near-infrared or ultrasound) to assist with placement. The responses were nearly uniform in this specialist group in recognizing when patients were appropriate to escalate for additional intervention to increase firsttime success.
Subsequent to the measurement of the current state, the respondents were asked to score the same scenarios with the C-DIVA tool. The scoring was in line with the intent of the scenario 70.9% of the time. There were some over scores (20.8%) and under-scores (8.3%) on individual factors. However, when looking at the total score there were no under-scores and only 1 out of 6 scenarios had an average score over that of the design. The over-scored scenario was designed to be marked a 5 which would indicate moderate difficulty in placing an IV. The tool would recommend that the assessor consider calling a specialist to use ultrasound. The respondents' scores averaged 7 meaning that they identified the scenario as high risk for difficulty and requiring the use of ultrasound. When considering DIVA patients, the risk would be underscoring the patient and not recognizing difficulty. Giving the patient a higher score and calling a vascular access specialist would only be a concern if care is delayed.

Results and Discussion
The data was analyzed using the built-in Data Analysis in Excel (Microsoft, Redmond, WA). An ANOVA: Two Factor Without Replication test was run with an alpha of 0.05. This yielded an Intra-class Correlation Coefficient (ICC) of 0.4942. Assessment of inter-rater agreement was assessed first by percentage agreement. This yielded a 55.59% agreement. However after more rigorous statistical analysis, the ICC was found to be 49.42%. (Table 2). The focus of this study was to refine and validate the C-DIVA tool for the assessment and early identification of DIVA patients across multiple patient care settings. The results obtained show that it is applicable for the intended purpose. Although other tools have been published and validated, they are limited to specific groups or populations. This is the first tool, to our knowledge, that is applicable to all patient populations. The tool is also relatively simple to use and straightforward in comparison to some others which have more factors to consider.
The tool has specific recommendations for the generalist to implement for successful vascular access placement. Other tools identify DIVA but leave the clinician to determine the best course of action. Following van Loon, we felt that it was crucial to give structured guidance to the tool results and assist clinicians to proceed with the best course of action. Without the specific guidance of the tool, identified DIVA patients might still be faced with multiple failed attempts prior to escalation.
It must also be noted that this tool gives a point-in-time scoring and not an absolute score relevant across the entire treatment regimen. A patient who arrives in the hospital via the ED may score high initially but may not be a high risk for difficulty later when admitted to a medical floor. At the same time, DIVA patients that consistently present with 'venopenia' or, lack of suitable veins, and relate a history of multiple venipuncture attempts should always score high enough to warrant immediate escalation without multiple failed attempts.
The face validation showed that there was strong agreement that the tool was understandable, unambiguous, and usable. There was some uncertainty about whether it would be used to appropriately impact care decisions but this seems to be more of an educational or organizational culture issue rather than a shortcoming of the tool.
The results of the inter-rater reliability of the tool were lower than expected. It is likely that this is attributable to the randomness of the surveying and the limited time to present the tool to conference attendees. Each surveyed clinician was given a sheet with the DIVA scoring tool and very brief instructions. It is evident from the responses that some participants did not feel that the tool was as clear on quick examination as those who responded to the face validation survey. Because of the general nature of the tool and its purposeful ability to be applied to multiple situations, some discussion and training is helpful to establish in the rater's mind what the Clinical Factors item is intended to measure. This lack of prior training for the tool introduced a degree of variability into the results. Despite a moderate ICC, the results are suggestive that the tool is applicable for assessing patients and may lead to more appropriate escalation of treatment for those patients identified as DIVA.
The Comprehensive Difficult IV Access Tool moves nursing away from a subjective assessment and toward an objective standard to escalate appropriate patients needing IV insertion to a nurse with advanced IV insertion skills. The status quo of attempting multiple times to access patients and then calling for advanced resources can now be replaced with a more efficient use of time and resources and provide successful IV access with a lower chance of failure. This could lead to improved patient satisfaction [14] and less nurse frustration as well as decrease delays in care [15].
The tool was tested on a group of clinicians with extensive expertise in vascular access. It may require some training for generalists to fully understand and use the tool. Additionally, since it is intended to be used in multiple settings across the healthcare industry, it will likely need education support to ensure that individuals understand the appropriate use in their department.

Conclusion
The C-DIVA tool has been validated by a sample of clinical experts as being useful for generalist clinicians to objectively determine when patients are difficult to access and in need of specialized care to increase the likelihood of first-time vascular access device placement success. Inter-rater reliability indicates that clinicians will consistently be able to use the tool to escalate care. It is therefore likely that the use of the tool would decrease failed attempts, limit the financial burden of wasted supplies and clinician time, and mitigate delays in the delivery of patient care.