Biomarkers and Applications (ISSN: 2576-9588)

research article

  PDF Download

Improved Performance of CSF Dementia Biomarker Measurements Over Time: The Effect of Quality Control and Harmonization Programs

Mara Bourbouli, Elisabeth Kapaki*, Olga Petropoulou, George P. Paraskevas

Department of Neurology, Athens National University, Eginition Hospital, 74 Vas Sophias Ave, Athens, Greece

*Corresponding author: Elisabeth Kapaki, Department of Neurology, Athens National University, Eginition Hospital, 74 Vas. Sophias Ave, Athens 11528, Greece. Tel: +302107289125; Email: ekapaki@med.uoa.gr

Received Date: 22 June, 2018; Accepted Date: 29 June, 2018; Published Date: 05 July, 2018

Citation: Bourbouli M, Kapaki E, Petropoulou O, Paraskevas GP (2018) Improved Performance of CSF Dementia Biomarker Measurements Over Time: The Effect of Quality Control and Harmonization Programs. Biomark Applic BMAP-126. DOI: 10.29011/BMAP-126. 100026

1.   Abstract

1.1 Introduction: Cerebrospinal Fluid (CSF) biomarkers Total Tau Protein (τT), Amyloid Beta Peptide (Aβ42) and tau protein hyper phosphorylated at threonine 181 (τP-181) are important in the (differential) diagnosis of dementia and analysis of these biomarkers is now incorporated in the diagnostic criteria of Alzheimer’s disease. However, lack of standardization has led to considerable inter- and intra-laboratory variation. Various international programs have been launched aiming in the reduction of variability and harmonization of biomarker measurements.

1.2 Objectives: To explore whether experience gained from international quality control and harmonization programs had any effect on the analytical performance of our laboratory for CSF dementia biomarkers [Total Tau (τT), Amyloid Beta (Aβ42) and Phospho-Tau (τP-181)].

1.3 Methods: We retrospectively analyzed internal standard measurements during ELISA runs in 3 time periods: before 2010, 2010-2012 (experience from workshops and quality control programs) and after 2012 (JPND-BIOMARKAPD harmonization program).

1.4 Results: During the 1st period, coefficients of variation were 8.6%-17.1%. Subsequently, they were reduced, reaching 4.5%-6.6% at the 3rd period. Measurement error was reduced for τT and Aβ42 from 9.2% and 22.1% to 1% and 3.3% respectively. Median values for Aβ42 were significantly lower compared to the expected values during the 1st period but, came closer to (at the 2nd period) and finally reached the expected value at the 3rd period. 

1.5 Conclusion: The improvement noted, indicates a beneficial effect of quality control and harmonization programs on analytical performance, by lowering measurement errors to levels which are not expected to adversely affect diagnostic performance in every day practice.             

2.   Keywords: Alzheimer’s Disease; Amyloid Beta; Cerebrospinal Fluid Biomarkers; Dementia; Phospho-Tau; Tau Protein

1. Introduction

Cerebrospinal Fluid (CSF) levels of amyloid beta peptide with 42 Amino Acids (Aβ42), Total Tau Protein (τT) and tau protein hyper phosphorylated at the threonine residue 181 (τP-181), are well established biomarkers [1,2] for the discrimination among normal ageing, Alzheimer’s disease and other dementing disorders in clinical practice [3-7]. The use of these biomarkers has been incorporated in current criteria for the (differential) diagnosis of Alzheimer’s disease in the dementia and pre-dementia [8-10] or even the preclinical stage [11]. The development of new, disease-modifying treatments acting at a pre-dementia stage, makes the use of such biomarkers important for the selection of patients for clinical trials [12,13]. However, after almost 2 decades of research there is still a significant inter- and Intra-Laboratory variability in the Determination of CSF biomarkers, as a result of pre-analytical, analytical, post-analytical and kit-related factors [14-18]. Recently, it has been suggested that measured concentration shifts may not affect every day practice so seriously as previously thought and, indeed, the effect on AD diagnostic accuracy may be minimal (~8% or less) when deviations of ±20% are present in only one of the three biomarkers [19]. However, when deviations are present in more than 1 biomarker and/or shifts are >20%, diagnostic performance may decrease significantly.  

International Workshops and International Quality Control Programs have been organized since 2009, in order to reduce variability and harmonize the levels of biomarkers universally [14,15]. More recently, the “Biomarkers for Alzheimer’s disease and Parkinson’s disease” project of the Joint Programming Neurodegenerative Disease (JPND-BIOMARKAPD) was launched at June 2012 [20]. The above projects led to an ongoing research on various parameters and confounding factors of the methodologies used [14,21] and to the formulation of recommendations for (pre)Analytical Standardized Operative Procedures (SOPs) on CSF biomarker assays [22], emphasizing the need for strict adherence to these SOPs at the highest level possible. The aim of the present study was to investigate whether this strict adherence to SOPs had any effect on the quality of CSF biomarker measurements in an individual laboratory, with expertise in biomarker research. 

2. Materials and Methods 

2.1  Time Periods 

We retrospectively analyzed the results of internal standard measurements during biomarker assays, performed in our laboratory from October 2000 to December 2016 divided in 3 periods: (a) From 2000 to the end of 2009. (b) From January 2010 to June 2012; during this period experience and ideas gained from quality control programs and workshops, were incorporated in routine practice and definitely influenced procedures in our laboratory. (c) From June 2012 to December 2016; during this period additional experience and guidelines from the BIOMARKAPD project were used as standard procedures. 

2.2    Measurements 

All measurements of τT, Aβ42 and τP-181 have been performed in duplicate by double sandwich, Enzyme-Linked Immunosorbent Assay (ELISA) as provided by commercially available kits according to manufacturer’s instructions (“Innotest® hTau antigen”, “β-amyloid1-42” and “phospho-tau181” respectively, provided initially by Innogenetics and then by Fujirebio Europe, Gent, Belgium). For each ELISA run a fresh in-house Internal Standard (IS) was prepared at the beginning of the assay in polypropylene tubes, using the concentrated standard provided by the manufacturer, dissolved in the provided sample diluent, exactly by the same way used to prepare the standards for the standards curve. Since, for practical reasons, assay precision has to be maximal at concentrations around the cut-off values, the IS concentrations for τT, Aβ42 and τP-181 were 300 pg/ml, 500 pg/ml and 62.5 pg/ml respectively, based on cut-off values suggested from previous studies in our laboratory [3,4,23]. The IS was measured in duplicate, as the last unknown sample (plate wells numbered 95 and 96). Absorbance was measured at dual wavelength (450 and 620 nm) by a Lab systems Multi scan EX ELISA reader, controlled by a computer-based program (Lab systems Genesis 3.03) which automatically calculated the concentrations by the use of a sigmoid curve. 

2.3 Statistical Analysis

Both the measured concentrations of the ISs and the calculated error (%deviation) from the expected (true) concentration were analyzed. This error was calculated by the formula: error= (measured concentration -true concentration)/true concentration and expressed as the absolute value. Due to deviations from normality and/or heterogeneity of variances, non-parametrics were mainly used. Median values of the measured concentrations of ISs in each period were compared to the expected concentration by the Wilkoxon-signed rank test. Measured concentrations and errors among the studied periods were compared by the Kruskal-Wallis test followed by Bonferroni-corrected individual Mann-Whitney tests. Variances were compared by the Bartlett’s test for equality of variances followed by Bonferroni-corrected individual F-tests. Statistical analyses were performed by GraphPad Prism 2.01 (GraphPad Software Inc, San Diego, CA) and StatSoft Statistica 8 (StatSoft Inc, Tulsa, OK) 

3. Results         

Results are summarized in Table 1 and Figure 1.

Median values of ISs for τT did not differ significantly among each period and as compared to the expected concentrations (300 pg/ml) at any time period. However, a significant reduction by ~70% in the coefficient of variation was noted, accompanied by a gradual and significant drop of the measurement error in both the 2nd and 3rd periods, compared to the 1st period. Although the difference between 2nd and 3rd periods did not reach statistical significance, numerically the reductions were more obvious at the 3rd period.

Median value of the IS for Aβ42 in the 1st period was significantly lower from the expected (500 pg/ml) concentration by 22% (P = 0.0003). At the 2nd period the median value increased significantly, but it was still significantly lower, compared to the expected concentration (P = 0.0021). Median value of the 3rd period did not differ significantly from the expected and, it was significantly higher compared to the median value of the 1st and 2nd periods. A significant reduction by ~ 60% in the coefficient of variation was noted, that was accompanied by a significant reduction of the measurement error in both the 2nd and 3rd periods compared to the 1st period. Again, the difference between 2nd and 3rd periods did not reach statistical significance; however, the reductions were numerically more obvious at the 3rd period. 

Median values of ISs for τP-181 did not differ significantly from the expected concentrations. Additionally, measured concentrations and errors did not differ among each other in the 3 periods. However, a significant reduction in the coefficient of variation and measurement error was noted after the 1st period. 

4.  Discussion

Before 2010, in our laboratory, two problems were noted: high CVs and high measurement error for Aβ42.  Τhe CV of Aβ42 was the highest (~17%), followed by τT (~16%) and τP-181 (8.6%). This is in agreement with other observations suggesting higher intra-laboratory variability for Aβ42, as compared to τT and τP-181 [18]. Such a high CV is not surprising, since even higher CVs have been reported in multicenter studies, even in reference laboratories. In a world-wide multicenter study, intra-laboratory coefficients of variation for Aβ42 and τT as high as 25% and 18% were reported in 2008 [24]. During 2009-2010 the intra-laboratory CVs in reference laboratories participating in a quality control program, ranged at levels of 6.4%-19% for Aβ42, 3.2%-24% for τT and 3.8%-14% for τP-181 [15]. Based on cumulative results of the same quality control program, the mean CVs for all laboratories participated were 19% for Aβ42, 16% for τT and 12% for τP-181 [17]. Thus, it has been suggested that not only the inter-laboratory, but also the intra-laboratory variation still remains high, sometimes contributing to diagnostic uncertainty, especially at levels near the cut-off value, which may form a “grey zone" [17]. However, such studies have mainly focused on the inter-laboratory variation and, although presenting intra-laboratory data, they did not compare the intra-laboratory accuracy and variance in different periods over time. To our knowledge, this is the 1st study directly addressing this question.

Additionally, before 2010, the median measurement error for Aβ42 was 22%. For τT the median error was much lower (9.2%); however, for some individual ELISA runs, the error was higher, even reaching 40%. Such a deviation from true values may be expected to affect clinical diagnostic accuracy [19]. 

Three types of change have been observed from 2010 onwards. First, a reduction of the inter-assay variation was noted, regarding the ISs of all three markers. At the 3rd period all CVs were at the range of 4.5%-6.6%. Second, the measurement error was reduced, being <3.3% for all 3 biomarkers during the 3rd period, which is not expected to significantly affect diagnostic accuracy in clinical practice [19]. Third, median values for Aβ42 came closer to (at the 2nd period) and finally reached the expected value at the 3rd period.  Thus, an improvement was noted since 2010 that was ongoing and reached maximal levels after June 2012. 

The present study deals neither with stored standards nor samples, but with freshly prepared ISs. Thus, it evaluates analytical accuracy and, the improvement noted, reflects an improvement in (post)analytical performance. The reason of this improvement may involve analytical and post-analytical causes (Table 2), whilst a lot to lot variability or improvement of kits over time should also be considered. 

5. Conclusion 

Strict adherence to SOPs according to current suggestions [25], is necessary in order to reduce variability of CSF biomarker determination and increase diagnostic performance in the era of biomarker-supported AD diagnosis. 

6. Acknowledgments

This research has been co-financed by the European Union (European Regional Development Fund - ERDF) and Greek national funds through the Operational Program “Competitiveness and Entrepreneurship” of the National Strategic Reference Framework (NSRF) -Research Funding Program: Joint Programming Neurodegenerative Disease, “Biomarkers for Alzheimer’s disease and Parkinson’s disease”.


 

 

1st period

2nd period

3rd period

P value

Oct 2000-Nov 2009

Jan 2010-May 2012

Jun 2012-Dec 2016

τT 300 pg/ml, n

26

14

11

 

median (quartiles)

295 (270 - 324)

294 (279 -306)

297 (294 - 299)

0.980

Min - max

171 - 380

260 - 338

259- 306

 

CV

0.163a

0.069

0.045

<0.0001

Measurement error*

0.092 (0.056 -0.148)b

0.042 (0.014 - 0.091)

0.010 (0.003 - 0.019)

0.0127

42 500 pg/ml, n

20

13

12

 

median (quartiles)

389 (358 - 445) c

452 (431 - 471) d

514 (499 - 546)

<0.0001

Min - max

289 - 582

389 - 512

452 - 581

 

CV

0.171e

0.072

0.066

0.0069

Measurement error*

0.221 (0.153 - 0.285) f

0.097 (0.057 - 0.138)

0.033 (0.008 - 0.092)

0.0146

τP-181 62.5 pg/ml, n

17

13

12

 

median (quartiles)

63.3 (59.5 - 65.5)

62.1 (60.7 - 62.8)

62.0 (61.0 - 62.9)

0.603

Min - max

47.8-68.7

59.9 - 69.8

52.9- 65.6

 

CV

0.086g

0.039

0.051

0.017

Measurement error *

0.040 (0.020 - 0.097)h

0.008 (0.005 - 0.034)

0.01 (0.008-0.044)

0.031

τT: Total Tau Protein. Aβ42: Amyloid-Beta Peptide with 42 amino acids. τP-181: phospho-tau protein phosphorylated at threonine-181. Quartiles: 25th-75th percentile. CV: coefficient of variation. *Absolute value [median (quartiles)]. Kruskal-Wallis test followed by Bonferroni-corrected Mann-Whitney tests. Bartlett’s test for equality of variances followed by Bonferroni-corrected F-tests. aP=0.0075 vs 2nd period and 0.0006 vs 3rd period. bP=0.053 vs 2nd period and 0.0009 vs 3rd period. cP=0.015 vs 2nd period and 0.0001 vs 3rd period. dP=0.0003 vs 3rd period.  eP=0.033 vs 2nd period and 0.074 vs 3rd period.  f P=0.003 vs 2nd period and 0.0003 vs 3rd period. gP=0.03 vs 2nd period. hP=0.051 vs 2nd period.

 

Table 1: Results of the 3 internal standard determinations for CSF biomarkers during the 3 studied periods.

 

2002 - 2009

2010 - June 2012

After June 2012

τT

 

 

Point to point sigmoid curve

Sigmoid (4-parameter logistic) curve fitting

Sigmoid (4-parameter logistic) curve fitting

Single channel pipettes

Single channel pipettes

Multichannel pipettes

Room temperature 18-26°C

Stable room temperature 25±2°C

Stable room temperature 25±2°C

Aβ42

 

Point to point sigmoid curve

Sigmoid (4-parameter logistic) curve fitting

Sigmoid (4-parameter logistic) curve fitting

Single channel pipettes

Single channel pipettes

Multichannel pipettes

Rare use of uncoated polypropylene plate

Always uncoated polypropylene plate

Always uncoated polypropylene plate

Room temperature 18-26°C

Stable room temperature 25±2°C

Stable room temperature 25±2°C

τP-181

 

Sigmoid (4-parameter logistic) curve fitting

Sigmoid (4-parameter logistic) curve fitting

Sigmoid (4-parameter logistic) curve fitting

Single channel pipettes

Single channel pipettes

Multichannel pipettes

Room temperature 18-26°C

Stable room temperature 25±2°C

Stable room temperature 25±2°C

τT: Total Tau Protein. Aβ42: Amyloid-Beta Peptide with 42 amino acids. τP-181: Phospho-Tau Protein Phosphorylated At Threonine-181.

 

Table 2: Analytical parameters in our laboratory changed after the end of 2009.

 

1.   Molinuevo JL, Blennow K, Dubois B, Engelborghs S, Lewczuk P, et al. (2014) The clinical use of cerebrospinal fluid biomarker testing for Alzheimer's disease diagnosis: a consensus paper from the Alzheimer's Biomarkers Standardization Initiative. Alzheimer’s Dement 10: 808-817.

2.     Blennow K, Dubois B, Fagan AM, Lewczuk P, de Leon MJ, et al. (2015) Clinical utility of cerebrospinal fluid biomarkers in the diagnosis of early Alzheimer’s disease. Alzheimer’s Dement 11: 58-69.

3.     Kapaki E, Paraskevas GP, Zalonis I, Zournas C (2003) CSF tau protein and beta-amyloid (1-42) in Alzheimer's disease diagnosis: discrimination from normal ageing and other dementias in the Greek population. Eur J Neurol 10: 119-128.

4.   Kapaki EN, Paraskevas GP, Tzerakis NG, Sfagos C, Seretis A, et al. (2007) Cerebrospinal fluid tau, phospho-tau181 and beta-amyloid1-42 in idiopathic normal pressure hydrocephalus: a discrimination from Alzheimer's disease. Eur J Neurol 14: 168-173.

5.    Wallin A, Kapaki E, Boban M, Engelborghs S, Hermann DM, et al. (2017) Biochemical markers in vascular cognitive impairment associated with subcortical small vessel disease - A consensus report. BMC Neurol 17: 102.

6.   Paraskevas GP, Kasselimis D, Kourtidou E, Constantinides V, Bougea A, et al. (2017) Cerebrospinal Fluid Biomarkers as a Diagnostic Tool of the Underlying Pathology of Primary Progressive Aphasia. J Alzheimer’s Dis 55: 1453-1461.

7.     Bourbouli M, Rentzos M, Bougea A, Zouvelou V, Constantinides VC, et al. (2017) Cerebrospinal Fluid TAR DNA-Binding Protein 43 Combined with Tau Proteins as a Candidate Biomarker for Amyotrophic Lateral Sclerosis and Frontotemporal Dementia Spectrum Disorders. Dement Geriatr Cogn Disord 44: 144-152.

8.    McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR Jr, et al. (2011) The diagnosis of dementia due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimer’s Dement 7: 263-269.

9.     Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, et al. (2011) The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimer’s Dement 7: 270-279.

10.  Dubois B, Feldman HH, Jacova C, Hampel H, Molinuevo JL, et al. (2014) Advancing research diagnostic criteria for Alzheimer's disease: the IWG-2 criteria. Lancet Neurol 13: 614-629.

11.  Dubois B, Hampel H, Feldman HH, Scheltens P, Aisen P, et al. (2016) Preclinical Alzheimer's disease: Definition, natural history, and diagnostic criteria. Alzheimer’s Dement 12: 292-323.

12.  Isaac M, Vamvakas S, Abadie E, Jonsson B, Gispen C, Pani L (2011) Qualification opinion of novel methodologies in the predementia stage of Alzheimer's disease: cerebro-spinal-fluid related biomarkers for drugs affecting amyloid burden-regulatory considerations by European Medicines Agency focusing in improving benefit/risk in regulatory trials. Eur Neuropsychopharmacol 21: 781-788.

13. Cavedo E, Lista S, Khachaturian Z, Aisen P, Amouyel P, et al. (2014) The road ahead to cure Alzheimer’s disease: development of biological markers and Neuroimaging methods for prevention trials across all stages and target populations. J Prev Alzheimers Dis 1: 181-202.

14. Teunissen CE, Verwey NA, Kester MI, van Uffelen K, Blankenstein MA (2010) Standardization of Assay Procedures for Analysis of the CSF Biomarkers Amyloid β (1-42), Tau, and Phosphorylated Tau in Alzheimer's Disease: Report of an International Workshop. Int J Alzheimer’s Dis pii 635053.

15. Mattsson N, Andreasson U, Persson S, Arai H, Batish SD, et al. (2011) The Alzheimer's Association external quality control program for cerebrospinal fluid biomarkers. Alzheimer’s Dement 7: 386-395.

16.  Kang JH, Korecka M, Toledo JB, Trojanowski JQ, Shaw LM (2013) Clinical utility and analytical challenges in measurement of cerebrospinal fluid amyloid-β (1-42) and τ proteins as Alzheimer disease biomarkers. Clin Chem 59: 903-916.

17.  Mattsson N, Andreasson U, Persson S, Carrillo MC, Collins S, et al. (2013) Alzheimer's Association QC Program Work Group. CSF biomarker variability in the Alzheimer’s Association quality control program. Alzheimer’s Dement 9: 251-261.

18.  Vos SJ, Visser PJ, Verhey F, Aalten P, Knol D, et al. (2014) Variability of CSF Alzheimer's disease biomarkers: implications for clinical practice. PLoS One 9: e100784.

19.  Niemantsverdriet E, Goossens J, Struyfs H, Martin JJ, Goeman J, et al. (2016) Diagnostic Impact of Cerebrospinal Fluid Biomarker (Pre-)Analytical Variability in Alzheimer's Disease. J Alzheimer’s Dis 51: 97-106.

20.  http://www.neurodegenerationresearch.eu/fileadmin/Project_Fact_Sheets/PDFs/Biomarkers/BIOMARKAPD_Fact_Sheet_Template.pdf

21.  Le Bastard N, De Deyn PP, Engelborghs S (2015) Importance and impact of preanalytical variables on Alzheimer disease biomarker concentrations in cerebrospinal fluid. Clin Chem 61: 734-743.

22. del Campo M, Mollenhauer B, Bertolotto A, Engelborghs S, Hampel H, et al(2012) Recommendations to standardize preanalytical confounding factors in Alzheimer's and Parkinson's disease cerebrospinal fluid biomarkers: an update. Biomark Med 6: 419-430.

23.  Paraskevas GP, Kapaki E, Papageorgiou SG, Kalfakis N, Andreadou E, et al. (2009) CSF biomarker profile and diagnostic value in vascular dementia. Eur J Neurol 16: 205-211.

24.  Verwey NA, van der Flier WM, Blennow K, Clark C, Sokolow S, et al. (2009) A worldwide multicentre comparison of assays for cerebrospinal fluid biomarkers in Alzheimer’s disease. Ann Clin Biochem 46: 235-240.

25.  Lewczuk P, Riederer P, O’Bryant SE, Verbeek MM, Dubois B, et al. (2018) Cerebrospinal fluid and blood biomarkers for neurodegenerative dementias: An update of the Consensus of the Task Force on Biological Markers in Psychiatry of the World Federation of Societies of Biological Psychiatry. World J Biol Psychiatry 19: 244-328.

Copyright and Licensing: This is an Open Access Journal Article Published Under Attribution-Share Alike CC BY-SA: Creative Commons Attribution-Share Alike 4.0 International License. With this license readers can share, distribute, download, even commercially, as long as the original source is properly cited. Read More.

   

share article