Food & Nutrition Journal (ISSN: 2575-7091)

Article / Research Article

"When Non-Significant Results Can Mask Significant Results-A Procedure for UsingDose-Related Analyses"

Gilbert R Kaats1*, Mrvichin NA1, Nugent SD2, Preuss HG3

1Integrative Health Technologies,Inc.,San Antonio, USA

2Mannatech Inc.,Coppell,USA

3Georgetown University, Washington DC, USA

 *Corresponding author:Gilbert R Kaats, Integrative Health Technologies, Inc., San Antonio, USA. Tel: +12108244200; +12108612400; Fax: +12103906142; Email: grk@ihtresearch.com

Received Date: 27 July, 2017; Accepted Date: 09 September, 2017; Published Date: 15 September, 2017

1.    Abstract

 Background: Non-significant within-group and between-group comparisons of baseline-ending changes in clinical trials and pilot studies can mask positive changes if dose-related compliance procedures are not applied.

 Objective:  To provide examples of three studies in which between- and within-group analyses of changes from baseline suggested minimal or no efficacy in contrast to dose-related procedures that suggested otherwise.

 Methods Data were obtained from three unpublished clinical or pilot studies in which between-group and within-group analyses suggested the absence of, or minimal, efficacy.  Data from these three studies were re-analyzed using a dose-related procedure based on study participants’ compliance with product-usage requirements.  Using compliance ratings, data were sub-grouped by quartiles or the median of the amount of product usage.

 Results: In all three studies within-groups t-tests suggested there were minimal or non-significant changes from baseline and no significant differences between the active and placebo or control groups.  However, using product consumption procedures revealed significant differences between upper and lower quartiles and between above and below the medians of product usage.  No similar differences were found in either the placebo or control groups when the compliance procedures were applied suggesting the findings was not a reflection of subjects’ motivation.

 Conclusions: Incorporating the dose-related procedures into study protocols can lead to an increased understanding of treatment plans or interventions. 

 2.    Keywords: Compliance; Dose-related

1.    Introduction

 In many pilot and clinical trials, the significance levels of within and between-group analyses are used as the basis of decisions regarding the safety and efficacy of the product being tested. However, notwithstanding these outcomes, additional within-group dose-related analyses are often available, but overlooked. In some cases, these analyses can increase the confidence in product’s efficacy, particularly when using small study groups. Dose-related within-group analyses can also provide useful information with regard to the accuracy of pre-established dose levels. Conversely, in instances where between- and within-group analyses have shown negative results, additional dose-related analyses may suggest that the failure of the product to demonstrate efficacy was the result of poorsubject compliance as opposed to a lack of efficacy. A number of studies have suggested that subjects often overstate their product usage for a variety of reasons, not the least of which is a desire to please the researcher or concern that they will not receive the incentive fees for participation in the study if they fail to take the product as prescribed. Thus, the inaccuracy of subject self-reports can undermine the validity of within and between group comparisons. In studies in the US, poor compliance has been associated with increased health care costs and risk of hospitalization[1], particularly when the underlying problem is difficult if not impossible for the subject or patient to receive timely feedback on progress or lack of progress.  After three decades of study [2], concluded that in spite of continuing efforts, “…no substantial new insights have arisen from quantitativeresearch of compliance and methods for measuring remain inadequately addressed.” [3]are more hopeful concluding that “Even with high rates of noncompliance, experimental data can yield useful and something accurate information on the effects of a treatment on the population” and offer statistical formulas for adjusting for non-compliance.

 This study reports re-analyses of three studies in which the between- and within-group analyses suggested weak or no significant efficacy and dose-related analyses suggested otherwise and the method used to enhance subject compliance with product usage reporting. In all three studies, the procedures described below followed the guidelines set forth earlier [4] to improve candid end-of-study product usage reports.

 2.     Method

 During the process of giving informed consent, subjects were explained that while they would receive a fee for their participation, the fee was not an “Incentive fee” but rather a “Recording fee” for timely recording their actual product usage and discomfort they might attribute to the product.  As such, subjects were advised, that while we encouraged taking the product as prescribed, at least equally important is their candid recording of how much or how little product they actually consumed. Throughout the three studies reviewed below, subjects were reminded of this distinction between “Incentive” and “Recording” fees and that their fees would be paid at the end of the study irrespective of the amounts of product usage and that proportional fees would be paid even if the subject withdrew from the study. At the conclusion of the study comparisons were made of changes from baseline in body composition measures: Scale Weight (SW), Fat Mass (FM) and Fat-free Mass (FFM) and a Body Composition Change Index (BCCI). The BCCI is based on the assumption that reductions in FM and gains in FFM signified positive changes and were scored accordingly. Conversely, increases in FM and decreases in FFM were assumed to be negative changes and were scored accordingly.  The BCCI is the net sum of these positive and negative outcomes. Additional information on the BCCI is provided elsewhere. Since there were no changes in height in these short-time studies, changes in the BMI were identical to changes in scale weight (Correlating 0.998) and are not reported.  Using the subjects’ product usage reports, within-group comparisons were made between subjects above and below the median of product usage and in quartiles. The three studies shown below. All three studies were approved by the Solutions Institutional Review Board, 16609Cantrell Road Ste 15a, Little Rock, AR., 72223and were conducted in accordance with the provisions of the World Medical Association’s Declaration of Helsinki.

 3.    Results 

Study group 1

 Subjects in this group consumed a dietary supplement designed to facilitate positive changes in body composition over a 60-day study period.  Using subjects’ product usage reports was for the first study, “Compliant” subjects where those who were above the median self-reported compliance with product usage requirements.  “Partially Compliant” for those who were below the median of product usage. Using the number of capsules subjects reported during the study, subjects were classified either as “Compliant” or “Partially Compliant” based whether or not their product usage was above or below the median product usage report for the study cohort.

 Comparisons between the two groups are shown in(Table1) below.  As shown on the “Total Caps” line, there was a statistically significant (p<0.001) between subjects above the median (n=9) and those below the median (n=11).  As the comparisons between the partially compliant and compliant reveals, there were no differences between the two groups with regard to overall weight loss. However, the Compliant group had a three-fold greater reduction in % fat (P=0.03) and FM (p=0.04). Although the Partially Compliant group lost FFM (-0.1 lbs.), the Compliant group gained FFM (+1.2 lbs.), although this difference failed to reach statistical significance (p=0.27).  Similarly, the compliant group’s BCI was 6 times greater than the Partially Compliant group (+0.5 lbs.  vs. + 3.0 lbs.).  However, the difference failed to reach statistical significance (p=0.27).

4.    Conclusion

 Poor adherence to anti-osteoporotic therapy significantly increases the risk of morality, possibly due to an increased risk of infection. Efforts should be made to improve adherence.

 This study investigated the effects of 1st-year adherence to anti-osteoporotic treatment on the risk of mortality in patients with magnetic resonance imaging-proven acute osteoporotic vertebral fractures after vertebra plasty. Poor adherence to anti-osteoporotic therapy significantly increases the risk of morality, possibly due to an increased risk of infection. Efforts should be made to improve adherence.Poor adherence to medications was significantly associated with an increase in the rate of infection (HR: 4.56; 95% CI: 1.12-18.52), which was the most common cause of death.

 More recently [5] attempted to assess the effects of whole grain consumption on changes in body composition finding that the data did not allow for valid conclusions about the relationship because of poor compliance. About the role of whole grains and body weight and body composition. However, they do point out the need for precise biomarkers of food consumption that would allow for post hoc validation of compliance. Extra weight should be placed on these measurable “On ensuring compliance to the recommended diet beyond self-report during the study.”

 The aim of this study, therefore, was to determine this study reports re-analyses of three studies in which the between- and within-group analyses suggested weak or no significant efficacy and dose-related analyses suggested otherwise and the method used to enhance subject compliance with product usage reporting.   In all three studies, the procedures described below followed the guidelines set forth earlier [4] to improve candid end-of-study product usage reports.


Figure 1: Comparison of changes in body composition above and below the median of amount of product consumed during study period.


Citation: Kaats GR, Mrvichin NA, Nugent SD, Preuss HG (2017) When Non-Significant Results Can Mask Significant Results - A Procedure for Using Dose-Related Analyses. Food Nutr J: FDNJ-144. DOI: 10.29011/2575-7091.100044.