Research Implications

The known: Exposure to natural disasters may result in adverse psychosocial outcomes. While some experience early symptom resolution, others may require ongoing psychosocial care. Intervention studies often include a limited number of participants residing in a localised region.

The new: This Australian naturalistic study of disaster-affected people demonstrates a standardised intervention program for those with chronic symptoms (>6 months duration) can be effective and conducted across rural, regional, and metropolitan centres.

The implications: The study highlights the effectiveness of a state-wide Post-disaster Trauma-Focussed Cognitive Behaviour Therapy. The outcome data demonstrates that ongoing therapy or monitoring may be required beyond the current post-disaster funding model.

Introduction

Australia's susceptibility to natural disasters is emphasised by the 1983 Ash Wednesday and the 2009 Black Saturday Bush Fires[1], the floods and cyclones that affected Queensland in 2010-2011 and the more recent flooding that has affected eastern Australia [2]. These events affect the psychological and economic well-being of communities and individuals in the short term and over time [3].

Post-disaster outcomes depend on various factors, including pre-disaster physical and psychological health, previous life events and social and economic factors [4]. Although many people recover, others may develop post-traumatic stress disorder (PTSD), depression, anxiety, and substance use disorders. Factors such as the severity and duration of the event, access to services, social connectedness, and change in work status influence psychological outcomes [5], with the response capacities and effects in urban and rural areas differing [6].

Timely and effective responses to disasters are an issue for all countries. Variability in the rate of mental disorders after disasters is emphasised by studies that report anxiety rates of 7-42%, complicated grief (28-41%), depression (6.5-38%), PTSD (11-89%), and substance misuse (1.3-24%) [7]. Post-disaster treatment studies are affected by this variation in psychological morbidity, with the broad range of psychological interventions that include cognitive behaviour therapies (CBT), eye-movement desensitisation and reprocessing (EMD-R), pet-assisted therapy and complementary medicines and a limited number of longitudinal studies [8, 9]. 

A systematic review by Lopes et al. (2014) further highlights the difficulty of determining the most effective post-disaster intervention treatment. The review noted few studies met the criteria for CBT treatment post-disaster, with most related to victims of earthquakes [10].

The 2022 floods affecting the eastern states of Australia, the floods and cyclones in Queensland, Australia, between December 2010 and February 2011, underline the challenges of responding to large-scale disasters. The Queensland events affected 68 of Queensland's 73 Local Government Authorities (LGAs) and 1.3 million people. Cities and towns became isolated, and a major provincial hospital required the evacuation of all their patients [11].

The Queensland disaster response included the establishment of The Queensland Mental Health Natural Disaster Recovery Plan 2011–13 (The Plan). The Plan linked primary health care, non-government organisations (NGOs), community-based supports, psychological first aid programs and established a specialised mental health program (SMHP). The response sought to encourage capacity building and provision of accessible services to at-risk populations through a multilayered approach that included the non-government and primary health sectors and evidence-informed specialist mental health care [12]. SMHP teams were located in areas most affected by the floods and cyclones and focused on treating those 18 and over (Table 1).

Table 1: Distribution and staff numbers of Recovery and Resilience across State.

Specialist mental health clinicians employed by the program were trained in Skills for Psychological Recovery (SPR) and Trauma-Focused CBT (TF-CBT). Clinicians participated in either face-to-face training or a web-based program focused on providing recovery-orientated trauma-informed care. The TF-CBT program consisted of 12 sessions and included psychoeducation, the development of social skills and problem-solving capacities, relaxation training, affect regulation, cognitive therapy, trauma-focused therapy and stabilisation and safety sessions. The treatment modules were linked with outcome goals established in conjunction with the individual and clinical assessment. Materials developed for the program included a clinician's field guide and psychoeducation handout resources. Supervision was provided by a senior local team member and senior staff attached to the Centre for Trauma Loss and Disaster Recovery [13].

Method

This paper reports a retrospective evaluation of the effectiveness of the TF-CBT program provided by the SMHP to 215 patients affected by the 2010-11 disasters during 2012. Ethics approval was granted by Metro South Health Human Centre for Health Research Ethics Committee (HREC/14/QPAH/472) and Queensland University of Technology (Ethics approval number 1500000016) - A retrospective evaluation of the outcomes of State-wide disaster mental health programs established and delivered following the Cyclones and Floods of 2010-2011.

SMHP clinicians assessed each person after referral to the program. The psychosocial and general health measures assessed the impact of the disaster and reflected assessments identified by Cénat et al. (2020) and a local panel of experts [14]. The assessments included patient narratives related to the event and quantitative measures to assess behavioural, psychological, and social phenomena. Data were obtained at triage, intake and discharge (Table 2).

Table 2: Triage, Pre-program, and Discharge Assessments.

The pre-and-post treatment repeat measures used to evaluate treatment efficacy were the Post-traumatic Symptom Check List Civilian version (PCL-C) [15], Kessler Psychological Distress Scale (K10) [16], Short Form Survey (mental and physical health) (SF12) [17], Global Assessment of Function (GAF) [18] and Health of the Nation Outcome Scales (HoNOS) [19].

The study population was drawn from the 881 people referred to the SMHP in 2012 following the 2010-2011 natural disasters. The duration between disaster exposure and referral ranged from 9 to 22 months. The analysis criteria for the study group (n=215) were that an individual completed an episode of care as evidenced by; delivery of three (3) or more face-to-face sessions, participation in at least one standardised 'Trauma-Focused Therapy Module', and completion of pre-and post-outcome assessments.

Data and clinical information were recorded in an electronic record. Following cessation of the program, de-identified information was entered into a secure database for subsequent evaluation. Independent data entry officers collated the de-identified information for storage in password-protected IBM SPSS (V23) files using a standardised data dictionary.

IBM SPSS Statistics was used to evaluate pre-and post-treatment outcomes. The treatment effect size was calculated by computing the difference between pre-and post-treatment outcome measure means and dividing this by the pooled standard deviation of those means. Effect size values (Cohen d) of 0.20 indicate small, 0.50 moderate, and 0.80+ large treatment effects. Larger effect sizes indicate more symptom reduction and fewer residual symptoms at the end of treatment [20].

Results

Program participants lived in Central Queensland( 22%), Metropolitan region and Gold Coast (15%), West Moreton (23%), Darling Downs (18%) and South-West Queensland (11%).

The PCL (n=173) pre-and post-treatment scores confirmed symptom reduction (m=49.8, SD:16.73 to m=33.77, SD:15.44; p<0.001). Similarly, the K10 (n=171) showed a decline in scores following treatment (m=27.98, SD:9.2 to 18.81, SD:8.6; p<0.001). The GAF (n=179) mean pre-treatment (64.04, SD:10.99) signified social, occupational, and interpersonal functioning difficulties. The post-treatment GAF scores (m=79.56, SD:11.56) reflect improved daily functioning and symptom reduction (pre-post GAF, p<0.001). Analysis of the SF12-Physical Health (SF12-PCS) (p=0.97) and SF12-Mental Health (SF12-MCS) (p=0.24) did not identify a significant change in pre-and post-treatment scores (Table 3). The HoNOS data were excluded from the study due to the number of subjects with missing data.

Cohen d pre-and post-treatment scores (pooled variance) were obtained for the PCL (0.996), K10 (1.029), GAF (1.377), SF12-PCS (0.007) and SF12-MCS (0.19) (Table 3). This represented a large effect size for the GAF, PCL and K10 but nil effect for the SF12-PCS and SF12-MCS.

Table 3: Treatment Outcomes:  Effect Size.

To further assess the effectiveness of the TF-CBT program, a matched-pairs analysis was undertaken (Table 4). Bonferroni corrections were used to account for multiple comparisons. The matched paired pre-and post-dependent t-test for the GAF, PCL, K10 and SF12-MCS found a significant difference for both one-sided and two-sided comparisons (p<0.001) for all these assessments.

Evaluation of the pre-and post-intervention paired scores (Table 4) with a Bonferroni correction was applied to all analyses. Following treatment, the PCL score decreased from 49.54 ± 17.11 to 34.15 ± 15.98 (t(154) = 18.12, p<0.005). The K10 score decreased from 27.96 ± 9.45 to 19.12 ± 8.72 (t(152) = 16.79, p<0.005. The GAF score increased from 64.79 ± 10.69 to 79.02 ± 11.91 (t(159) = -18.243, p<0.005).The matched pairs SF12-MCS demonstrated a decrease from 45.30 ± 11.31 to 36.33 ± 11.10 (t(62) = 4.87, p<0.005). In contrast, the SF12-PCS pre (43.61 ± 11.83) and post-treatment scores (44.56 ± 12.90) did not differ (t(61) = -0.48, p= 0.63).

Table 4: Treatment Outcomes Matched Pairs Analysis: Comparison Dependent t-test (Pre-Post).

As shown in Table 5, the ES analyses revealed a large Cohen d and Hedges d ES for the pre-and post-GAF, PCL and K10. The SF12-PCS ES was less than zero, while the SF12-MCS demonstrated a moderate impact. This score was in reverse, indicating that the matched pairs analysis suggested a deterioration in mental health as assessed by the SF12-MCS. The Hedges’ g was undertaken to address the risk of bias related to sample size [21].

Table 5: Treatment Outcomes Matched Pairs Analysis Effect Size.

Discussion

The importance of this naturalistic study is highlighted by the systematic review by Lopes et al. (2014) that from over 800 papers, only 11 studies identified (n=742) were related to natural disasters and included standardised pre-and post-intervention assessments, a diagnosis of PTSD, and treatment with cognitive behaviour therapy. Ten of the studies focussed on people affected by earthquakes [10]. A 2018 Cochrane Review of treatment provided to people living in humanitarian settings identified 36 studies; however, the treatment groups also included those exposed to armed conflict. Two studies used EMD-R and non-used TF-CBT [22].

Unlike other studies, this evaluation includes people living in metropolitan, regional and rural communities. The study findings support the efficacy of TF-CBT with a reduction in PTSD (PCL) and psychological distress (K10) and improved Global Function (GAF). There was, however, no improvement in general health (SF12-PCS). Although the self-report measures (PCL, K10) demonstrated improvement, the scores indicate many people remained symptomatic, suggesting the need for further treatment, and continued monitoring of their mental health status (Table 3).

Post-traumatic symptom change (PCL)

The study findings compare with other evaluations and demonstrate that an evidence-informed program can be provided and supported across an area several times the size of Great Britain [23]. The pre-post symptom reduction demonstrates similar efficacy to programs provided to earthquake victims in Turkey and China [10].

Psychological Distress (K 10)

The high level of pre-treatment distress in the study group compares with Australian community samples that indicate around 10% of the population score in the high range (K10=22-29) and 3-4% in the very high range (K10=30-50) [24]. The level of distress in the study group is further highlighted when comparing a cross-sectional online survey of 547 Australians during COVID-19 (mean K10 score of 19.6 (SD=7.6)) [25].

SF12

The SF12 has two summary scales describing mental health wellbeing (SF12-MCS) and physical wellbeing (SF12-PCS). Higher scores indicate better health, with an assumption the scoring algorithms for physical and mental health are not correlated [26].

The pre-treatment SF12-MCS and SF12-PCS group and matched pairs means compare with an Australian community sample (n=10,641) that found means of SF12-MCS 49.05 (SD:10.36) and SF12-PCS 48.85 (SD:10.05) [27]. Interestingly neither the group nor matched pairs SF12-PCS (Tables 3, 4 and 5) identified a significant change, while the matched pairs SF12-MCS demonstrated a decline in the mean score, indicating an increase in psychological morbidity.

The decline in the SF12-MCS score for both the group and matched pairs occurred despite the reduced PCL and K10 scores and the improved GAF score. One explanation may relate to the observation that despite the decrease in symptoms as measured by the PCL and K10, both measures indicate the presence of ongoing morbidity. The decline in score may also be explained by factors such as new stressors related to the disasters or other events that may have occurred during the treatment period.

Global Assessment of Functioning (GAF)

Despite the growing number of natural disasters, knowledge of functioning over time versus cross-sectional functioning remains limited. Most studies are cross-sectional rather than longitudinal and unrelated to therapeutic interventions [28]. Even the outcome measures in many of the 9/11 studies focussed on symptom reduction rather than function [29, 30]. This study demonstrates the impact on global function of those affected by the events of 2010-11 before treatment and post-intervention (Table 4).

Integrating evidence-based care into day-to-day clinical practice when services are confronted with the challenge of damaged infrastructure, the potential individual and community economic losses and the adverse psychosocial outcomes of disasters places significant demands on clinicians. Factors such as access to supervision, therapist beliefs and organisational imperatives may negatively influence the application of evidence-based interventions within a disaster environment. These aspects and variability in adherence to International and Australian post-disaster treatment guidelines can influence treatment outcomes [31-33]. The results highlight that while symptom reduction may be significant that psychological symptoms may remain elevated suggesting the need for ongoing therapy beyond the disaster funding timescale of two years. In addition, mental health assessment and counselling grants are often a fraction of the allocated aid funds and the spending on infrastructure [34-35].

The evaluation of the TF-CBT implemented following the Queensland 2010-11 disasters, like other naturalistic studies, is affected by clinician experience, level of supervision, patient psychosocial factors and non-adherence to protocols as evidenced by the failure to complete some assessments. While these factors place limitations on the evaluation, it does, like other naturalistic studies, place the assessment of the program outcomes within the 'real world setting' in which clinicians and health services work [36].

Strengths

This retrospective evaluation adopts a naturalistic format to appraise the outcome of interventions utilised in response to the Floods and Cyclones of 2010-11. While Randomised Controlled Trials (RCTs) are recognized as the 'gold standard' for evaluating efficacy[37], RCTs have limitations. RCTs generally use highly trained clinicians with high levels of supervision. The programs are often conducted in academic centres using strict inclusion and exclusion criteria. Evaluation of naturalistic studies provides an opportunity to assess the effect of clinical interventions and whether the treatment outcomes have real-world meaning in the study population [38]. Although naturalistic studies may not aid in establishing the influence of a treatment on the outcome to a therapy, naturalistic evaluations can be seen as complementary to RCTs [39, 40]. The SMHP treated a population with chronic symptoms between 9-22 months following the natural disasters. Participants were experiencing persistent and significant symptoms. Importantly the program provided treatment across a broad range of geographically diverse communities, delivered by local clinicians with a standardised protocol and supervision.

Limitations

The study did not evaluate whether those who attended more sessions achieved better outcomes. Some participants did not complete all the assessment measures, and the HoNOS data could not be included in the evaluation due to the amount of missing data. Information related to post-disaster experiences between the commencement and completion of the program was not recorded in the data set, and reasons for ceasing therapy were not recorded. Like other naturalistic studies, clinician experience, level of supervision, and non-adherence to protocols may influence the outcomes.

Conclusion

The naturalistic evaluation demonstrates the treatment program's effectiveness; however, the post-intervention mean PCL and K10 scores for the group and matched pairs continue to reflect significant morbidity levels. The pre-and post-treatment results indicate that despite treatment, many participants continue to exhibit considerable morbidity suggesting that psychosocial support interventions should continue beyond the two-year timeframe provided by the Australian Government disaster relief funding.

References

1. Dwyer G, Hardy C (2016) We have not lived long enough: Sensemaking and learning from bushfire in Australia. Management learning 47: 45-64.
2. Bureau of Meteorology (2022) Australia in summer 2021–2022 Canberra: Commonwealth of Australia Bureau of Meteorology.
3. Alderman K, Turner L, Tong S (2013) Assessment of the health impacts of the 2011 summer floods in Brisbane. Disaster medicine and public health preparedness 7: 380.
4. Rosen CS, Matthieu MM, Norris FH (2009) Factors predicting crisis counselor referrals to other crisis counseling, disaster relief, and psychological services: a cross-site analysis of post-Katrina programs. Administration and policy in mental health 36: 186-94.
5. Furukawa H, Takeuchi T, Yano E, Muto S (2015) Factors influencing psychological distress after the great east japan earthquake and Tsunami. Journal of community psychology 43: 521-6.
6. Handley TE, Rich J, Lewin TJ, Kelly BJ (2019) The predictors of depression in a longitudinal cohort of community dwelling rural adults in Australia. Social Psychiatry and Psychiatric Epidemiology 54: 171-80.
7. Crompton D (2019) Responding to Disasters: More Than Economic and Infrastructure Interventions. Prehospital and Disaster Medicine 34: s70-s.
8. Beaglehole B, Mulder RT, Frampton CM, Boden JM, Newton-Howes G, et al. (2018) Psychological distress and psychiatric disorder after natural disasters: systematic review and meta-analysis. British journal of psychiatry 213: 716-22.
9. Cusack K, Jonas DE, Forneris CA, Wines C, Sonis J, Middleton JC, et al. (2016) Psychological treatments for adults with posttraumatic stress disorder: A systematic review and meta-analysis. Clinical psychology review 43: 128-41.
10. Lopes A, Macedo T, Coutinho E, Figueira I, Ventura P (2014) Systematic review of the efficacy of cognitive-behavior therapy related treatments for victims of natural disasters: A worldwide problem. PloS one 9: e109013-e.
11. Arklay T (2012) Queensland's State Disaster Management Group : an all agency response to an unprecedented natural disaster. Australian journal of emergency management 27: 9-19.
12. Wade D, Crompton D, Howard A, Stevens N, Metcalf O, et al. (2014) Skills for Psychological Recovery: Evaluation of a post-disaster mental health training program. Disaster Health 2:138-45.
13. The Centre for Trauma Loss and Disaster Recovery. The Centre for Trauma, Loss and Disaster Recovery: Annual Report for 2012-13: Queensland Health.
14. Cénat JM, McIntee S-E, Blais-Rochette C (2020) Symptoms of posttraumatic stress disorder, depression, anxiety and other mental health problems following the 2010 earthquake in Haiti: A systematic review and meta-analysis. Journal of affective disorders 273:55-85.
15. Conybeare D, Behar E, Solomon A, Newman M, Borkovec T (2012) The PTSD checklist-civilian version: Reliability, validity, and factor structure in a nonclinical sample: PCL-C in a nonclinical sample. Journal of clinical psychology 68:699-713.
16. Andrews G, Slade T (2001) Interpreting scores on the Kessler Psychological Distress Scale (K10). Australian and New Zealand journal of public health 25:494-7.
17. Lundberg L, Johannesson M, Isacson DG, Borgquist L (1999) The relationship between health-state utilities and the SF-12 in a general population. Medical Decision Making : An International Journal of the Society for Medical Decision Making 19:128-40.
18. Miller MW, Wolf EJ, Martin E, Kaloupek DG, Keane TM (2008) Structural equation modeling of associations among combat exposure, PTSD symptom factors, and Global Assessment of Functioning. Journal of Rehabilitation Research and Development 45:359-70.
19. Trauer T (2010) Outcome measurement in mental health: Theory and practice. Cambridge: Cambridge University Press.
20. Ivarsson A, Andersen MB, Johnson U, Lindwall M (2023) To adjust or not adjust: Nonparametric effect sizes, confidence intervals, and real-world meaning. Psychology of sport and exercise 14:97-102.
21. Lin L, Aloe AM (2021) Evaluation of various estimators for standardized mean difference in meta‐analysis. Statistics in medicine 40:403-26.
22. Purgato M, Gastaldon C, Papola D, van Ommeren M, Barbui C, et al. (2018) Psychological therapies for the treatment of mental disorders in low- and middle-income countries affected by humanitarian crises. Cochrane Database of Systematic Reviews.
23. Queensland Government (2017) Interesting facts about Queensland Brisbane: Quensland Government.
24. Enticott JC, Meadows GN, Shawyer F, Inder B, Patten S (2016) Mental disorders and distress: Associations with demographics, remoteness and socioeconomic deprivation of area of residence across Australia. Australian & New Zealand Journal of Psychiatry 50:1169-79.
25. Rahman MA, Hoque N, Alif SM, Salehin M, Islam SMS, et al. (2020) Factors associated with psychological distress, fear and coping strategies during the COVID-19 pandemic in Australia. Globalization and health 16:95.
26. Hagell P, Westergren A, Årestedt K (2017) Beware of the origin of numbers: Standard scoring of the SF‐12 and SF‐36 summary measures distorts measurement and score interpretations. Research in nursing & health 40: 378-86.
27. Andrews G (2002) A brief integer scorer for the SF-12: validity of the brief scorer in Australian community and clinic settings. Australian and New Zealand journal of public health 26: 508-10.
28. Lahiri S, van Ommeren M, Roberts B (2017) The influence of humanitarian crises on social functioning among civilians in low- and middle-income countries: A systematic review. Global public health 12:1461-78.
29. Lowell A, Suarez-Jimenez B, Helpman L, Zhu X, Durosky A, Hilburn A, et al. (2018) 9/11-related PTSD among highly exposed populations: a systematic review 15 years after the attack. Psychological medicine 48: 537-53.
30. Hamwey MK, Gargano LM, Friedman LG, Leon LF, Petrsoric LJ, et al. (2020) Post-traumatic stress disorder among survivors of the september 11, 2001 world trade center attacks: A review of the literature. International journal of environmental research and public health 17:1-19.
31. Couineau A-L, Forbes D (2011) Using predictive models of behavior change to promote evidence-based treatment for PTSD. Psychological Trauma: Theory, Research, Practice, and Policy.
32. Sundram S, Karim ME, Ladrido-Ignacio L, Maramis A, Mufti KA, et al. (2008) Psychosocial responses to disaster: An Asian perspective. Asian Journal of Psychiatry 1: 7-14.
33. Crompton D, Young R, Shakespeare-Finch J, Raphael B (2018) Responding to Disasters: More than economic and infrastructure interventions. Insights Depress Anxiety 2.
34. Voelker R (2006) Post-katrina mental health needs prompt group to compile disaster medicine guide. JAMA : the journal of the American Medical Association 295:259-60.
35. Fulton AE, Drolet J, Lalani N, Smith E (2020) Prioritizing psychosocial services for children, youth and families postdisaster. Disaster Prevention and Management: An International Journal 29:591-607.
36. Jobes DA, Bryan CJ, Neal-Walden TA (2009) Conducting suicide research in naturalistic clinical settings. Journal of clinical psychology 65:382-95.
37. Lilienfeld SO, McKay D, Hollon SD (2018) Why randomised controlled trials of psychological treatments are still essential. The Lancet Psychiatry 5: 536-8.
38. Leichsenring F (2004) Randomized controlled versus naturalistic studies: A new research agenda. Bulletin of the Menninger Clinic 68: 137-51.
39. Van Wyk S, Schweitzer RD (2014) A systematic review of naturalistic interventions in refugee populations. Journal of Immigrant and Minority Health 16:968-77.
40. Vázquez GH, Holtzman JN, Lolich M, Ketter TA, Baldessarini RJ (2015) Recurrence rates in bipolar disorder: Systematic comparison of long-term prospective, naturalistic studies versus randomized controlled trials. European neuropsychopharmacology 25: 1501-12.