31.5.18

Psychosocial Effects of Parent-Child Book Reading Interventions: A Meta-analysis.

Xie QW, Chan CHY, Ji Q, Chan CLW.

CONTEXT: Parent-child book reading (PCBR) is effective at improving young
children's language, literacy, brain, and cognitive development. The psychosocial
effects of PCBR interventions are unclear.
OBJECTIVE: To systematically review and synthesize the effects of PCBR
interventions on psychosocial functioning of children and parents.
DATA SOURCES: We searched ERIC, PsycINFO, Medline, Embase, PubMed, Applied Social
Sciences Index and Abstracts, Social Services Abstracts, Sociological Abstracts, 
Family and Society Studies Worldwide, and Social Work Abstracts. We hand searched
references of previous literature reviews.
STUDY SELECTION: Randomized controlled trials.
DATA EXTRACTION: By using a standardized coding scheme, data were extracted
regarding sample, intervention, and study characteristics.
RESULTS: We included 19 interventions (3264 families). PCBR interventions
improved the psychosocial functioning of children and parents compared with
controls (standardized mean difference: 0.185; 95% confidence interval: 0.077 to 
0.293). The assumption of homogeneity was rejected (Q = 40.010; P < .01). Two
moderator variables contributed to between-group variance: method of data
collection (observation less than interview; Qb = 7.497; P < .01) and rater
(reported by others less than self-reported; Qb = 21.368; P < .01). There was no 
significant difference between effects of PCBR interventions on psychosocial
outcomes of parents or children (Qb = 0.376; P = .540).
LIMITATIONS: The ratio of moderating variables to the included studies limited
interpretation of the findings.
CONCLUSIONS: PCBR interventions are positively and significantly beneficial to
the psychosocial functioning of both children and parents.

24.5.18

Early intensive behavioral intervention (EIBI) for young children with autism spectrum disorders (ASD).

Reichow B, Hume K, Barton EE, et al. 

BACKGROUND: The rising prevalence of autism spectrum disorders (ASD) increases the need for evidence-based behavioral treatments to lessen the impact of symptoms on children's functioning. At present, there are no curative or psychopharmacological therapies to effectively treat all symptoms of the disorders. Early intensive behavioral intervention (EIBI) is a treatment based on the principles of applied behavior analysis. Delivered for multiple years at an intensity of 20 to 40 hours per week, it is one of the more well-established treatments for ASD. This is an update of a Cochrane review last published in 2012.

OBJECTIVES: To systematically review the evidence for the effectiveness of EIBI in increasing functional behaviors and skills, decreasing autism severity, and improving intelligence and communication skills for young children with ASD.

SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, 12 additional electronic databases and two trials registers in August 2017. We also checked references and contacted study authors to identify additional studies.

SELECTION CRITERIA: Randomized control trials (RCTs), quasi-RCTs, and controlled clinical trials (CCTs) in which EIBI was compared to a no-treatment or treatment-as-usual control condition. Participants must have been less than six years of age at treatment onset and assigned to their study condition prior to commencing treatment.

DATA COLLECTION AND ANALYSIS: We used standard methodological procedures expected by Cochrane.We synthesized the results of the five studies using a random-effects model of meta-analysis, with a mean difference (MD) effect size for outcomes assessed on identical scales, and a standardized mean difference (SMD) effect size (Hedges' g) with small sample correction for outcomes measured on different scales. We rated the quality of the evidence using the GRADE approach.

MAIN RESULTS: We included five studies (one RCT and four CCTs) with a total of 219 children: 116 children in the EIBI groups and 103 children in the generic, special education services groups. The age of the children ranged between 30.2 months and 42.5 months. Three of the five studies were conducted in the USA and two in the UK, with a treatment duration of 24 months to 36 months. All studies used a treatment-as-usual comparison group.Primary outcomesThere is low quality-evidence at post-treatment that EIBI improves adaptive behaviour (MD 9.58 (assessed using Vineland Adaptive Behavior Scale (VABS) Composite; normative mean = 100, normative SD = 15), 95% confidence interval (CI) 5.57 to 13.60, P < 0.0001; 5 studies, 202 participants), and reduces autism symptom severity (SMD -0.34, 95% CI -0.79 to 0.11, P = 0.14; 2 studies, 81 participants; lower values indicate positive effects) compared to treatment as usual.No adverse effects were reported across studies.Secondary outcomesThere is low-quality evidence at post-treatment that EIBI improves IQ (MD 15.44 (assessed using standardized IQ tests; scale 0 to 100, normative SD = 15), 95% CI 9.29 to 21.59, P < 0.001; 5 studies, 202 participants); expressive (SMD 0.51, 95% CI 0.12 to 0.90, P = 0.01; 4 studies, 165 participants) and receptive (SMD 0.55, 95% CI 0.23 to 0.87, P = 0.001; 4 studies, 164 participants) language skills; and problem behaviour (SMD -0.58, 95% CI -1.24 to 0.07, P = 0.08; 2 studies, 67 participants) compared to treatment as usual.

AUTHORS' CONCLUSIONS: There is weak evidence that EIBI may be an effective behavioral treatment for some children with ASD; the strength of the evidence in this review is limited because it mostly comes from small studies that are not of the optimum design. Due to the inclusion of non-randomized studies, there is a high risk of bias and we rated the overall quality of evidence as 'low' or 'very low' using the GRADE system, meaning further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.It is important that providers of EIBI are aware of the current evidence and use clinical decision-making guidelines, such as seeking the family's input and drawing upon prior clinical experience, when making recommendations to clients on the use EIBI. Additional studies using rigorous research designs are needed to make stronger conclusions about the effects of EIBI for children with ASD.

2.5.18

The effect of acute and chronic exercise on cognitive function and academic performance in adolescents: A systematic review.

Li JW, O'Connor H, O'Dwyer N, Orr R.


OBJECTIVES: To investigate whether exercise, proposed to enhance neuroplasticity  and potentially cognitive function (CF) and academic performance (AP), may be beneficial during adolescence when important developmental changes occur.
DESIGN: Systematic review evaluating the impact of acute or chronic exercise on CF and AP in adolescents (13-18 years).
METHODS: Nine databases (AMED, AusportMed, CINAHL, COCHRANE, Embase, Medline, Scopus, SPORTdiscus, Web of Science) were searched from earliest records to 31st  October 2016, using keywords related to exercise, CF, AP and adolescents. Eligible studies included controlled trials examining the effect of any exercise  intervention on CF, AP or both. Effect size (ES) (Hedges g) were calculated where possible.
RESULTS: Ten papers (11 studies) were reviewed. Cognitive domains included: executive function (n=4), memory (n=4), attention/concentration (n=2), visuo-motor speed (n=1), logical sequencing (n=1) and psychometric aptitude (n=1). All papers, nine of 10 being acute studies, reported at least one parameter showing a significant effect of exercise in improving CF and AP. However, the CF parameters displayed substantial heterogeneity, with only 37% favouring acute and chronic exercise. Where ES could be calculated, 52% of the acute CF parameters favoured rest. Memory was the domain most consistently improved by exercise. Academic performance demonstrated a significant improvement with exercise in one of two acute studies and the only chronic study (p≤0.001).
CONCLUSIONS: The evidence for the effect of exercise on CF and AP in adolescents  is equivocal and limited in quantity and quality. Well-designed research is therefore warranted to determine the benefits of exercise in enhancing CF and AP  and reducing sedentary behaviour.

25.4.18

Challenges in Developing U.S. Preventive Services Task Force Child Health Recommendations.

Kemper AR, Krist AH, Tseng CW, Gillman MW, Mabry-Hernandez IR, Silverstein M, Chou R,
Lozano P, Calonge BN, Wolff TA, Grossman DC.

Am J Prev Med. 2018 Jan;54(1S1):S63-S69. doi: 10.1016/j.amepre.2017.08.023.

The U.S. Preventive Services Task Force (USPSTF) uses an objective evidence-based
approach to develop recommendations. As part of this process, the USPSTF also
identifies important research gaps in scientific evidence. In March 2016, the
USPSTF convened an expert panel to discuss its portfolio of child and adolescent 
recommendations and identify unique methodologic issues when evaluating evidence 
regarding children and adolescents. The panel identified key domains of
challenges, including measuring patient-centered health outcomes; identifying
intermediate outcomes predictive of important health outcomes; evaluating the
long time horizon needed to assess the balance of benefits and harms;
understanding trajectories of growth and development that result in unique
windows of time when expected benefits or harms of a preventive service can vary;
and considering the perspectives of other individuals who might be affected by
the delivery of a preventive service to a child or adolescent. Although the
expert panel expressed an interest in being able to make more recommendations for
or against preventive services for children and adolescents, it also reinforced
the importance of ensuring recommendations were based on sound and sufficient
evidence to ensure greatest benefit and minimize unnecessary harms. Accordingly, 
the need to highlight areas with insufficient evidence is as important as making 
recommendations. Having identified these key challenges, the USPSTF and other
organizations issuing guidelines have an opportunity to advance their methods of 
evidence synthesis and identified evidence gaps represent important opportunities
for researchers and policy makers.

26.3.18

Interventions promoting exclusive breastfeeding up to six months after birth: A systematic review and meta-analysis of randomized controlled trials.

Kim SK, Park S, Oh J, Kim J, Ahn S.


BACKGROUND: The World Health Organization (WHO) recommends that mothers practice 
exclusive breastfeeding (EBF) of their infants for 6 months. Various
breastfeeding support interventions have been developed to encourage mothers to
maintain breastfeeding practices. Research aim: This study aims to review how
effectively breastfeeding support interventions enable mothers to practice EBF
for 6 months and to suggest the best intervention strategies.
METHODS: Six databases were searched, including MEDLINE, EMBASE, Cochrane,
CINAHL, PsycINFO, and KoreaMed. The authors independently extracted data from
journals written in English or Korean and published between January 2000 and
August 2017. Randomized controlled trials (RCTs) reporting EBF until 6 months
were screened.
RESULTS: A total of 27 RCTs were reviewed, and 36,051 mothers were included. The 
effectiveness of breastfeeding support interventions to promote EBF for 6 months 
was significant (odds ratio [OR] = 2.77; 95% confidence interval [CI]:
1.81-3.76). A further subgroup analysis of intervention effects shows that a baby
friendly hospital initiative (BFHI) intervention (OR = 5.21; 95% CI: 2.15-12.61),
a combined intervention (OR = 3.56; 95% CI: 1.74-7.26), a professional provider
led intervention (OR = 2.76; 95% CI: 1.76-4.33), having a protocol available for 
the provider training program (OR = 2.87; 95% CI: 1.89-4.37) and implementation
during both the prenatal and postnatal periods (OR = 3.32; 95% CI: 1.83-6.03)
increased the rate of EBF for 6 months.
CONCLUSION: We suggest considering a multicomponent intervention as the primary
strategy and implementing BFHI interventions within hospitals. Evidence indicates
that intervention effectiveness increases when a protocol is available for
provider training, when interventions are conducted from the pre- to postnatal
period, when the hospital and community are connected, and when healthcare
professionals are involved.

23.3.18

The Proposal to Lower P Value Thresholds to .005

John P. A. Ioannidis
JAMA. Published online March 22, 2018. doi:10.1001/jama.2018.1536
P values and accompanying methods of statistical significance testing are creating challenges in biomedical science and other disciplines. The vast majority (96%) of articles that report P values in the abstract, full text, or both include some values of .05 or less.1 However, many of the claims that these reports highlight are likely false.2 Recognizing the major importance of the statistical significance conundrum, the American Statistical Association (ASA) published3 a statement on P values in 2016. The status quo is widely believed to be problematic, but how exactly to fix the problem is far more contentious. The contributors to the ASA statement also wrote 20 independent, accompanying commentaries focusing on different aspects and prioritizing different solutions. Another large coalition of 72 methodologists recently proposed4 a specific, simple move: lowering the routine P value threshold for claiming statistical significance from .05 to .005 for new discoveries. The proposal met with strong endorsement in some circles and concerns in others.
P values are misinterpreted, overtrusted, and misused. The language of the ASA statement enables the dissection of these 3 problems. Multiple misinterpretations of P values exist, but the most common one is that they represent the “probability that the studied hypothesis is true.”3 A P value of .02 (2%) is wrongly considered to mean that the null hypothesis (eg, the drug is as effective as placebo) is 2% likely to be true and the alternative (eg, the drug is more effective than placebo) is 98% likely to be correct. Overtrust ensues when it is forgotten that “proper inference requires full reporting and transparency.”3 Better-looking (smaller) P values alone do not guarantee full reporting and transparency. In fact, smaller P values may hint to selective reporting and nontransparency. The most common misuse of the P value is to make “scientific conclusions and business or policy decisions” based on “whether a P value passes a specific threshold” even though “a P value, or statistical significance, does not measure the size of an effect or the importance of a result,” and “by itself, a P value does not provide a good measure of evidence.”3
These 3 major problems mean that passing a statistical significance threshold (traditionally P = .05) is wrongly equated with a finding or an outcome (eg, an association or a treatment effect) being true, valid, and worth acting on. These misconceptions affect researchers, journals, readers, and users of research articles, and even media and the public who consume scientific information. Most claims supported with P values slightly below .05 are probably false (ie, the claimed associations and treatment effects do not exist). Even among those claims that are true, few are worth acting on in medicine and health care.
Lowering the threshold for claiming statistical significance is an old idea. Several scientific fields have carefully considered how low a P value should be for a research finding to have a sufficiently high chance of being true. For example, adoption of genome-wide significance thresholds (P < 5 × 10−8) in population genomics has made discovered associations highly replicable and these associations also appear consistently when tested in new populations. The human genome is very complex, but the extent of multiplicity of significance testing involved is known, the analyses are systematic and transparent, and a requirement for P < 5 × 10−8 can be cogently arrived at.
However, for most other types of biomedical research, the multiplicity involved is unclear and the analyses are nonsystematic and nontransparent. For most observational exploratory research that lacks preregistered protocols and analysis plans, it is unclear how many analyses were performed and what various analytic paths were explored. Hidden multiplicity, nonsystematic exploration, and selective reporting may affect even experimental research and randomized trials. Even though it is now more common to have a preexisting protocol and statistical analysis plan and preregistration of the trial posted on a public database, there are still substantial degrees of freedom regarding how to analyze data and outcomes and what exactly to present. In addition, many studies in contemporary clinical investigation focus on smaller benefits or risks; therefore, the risk of various biases affecting the results increases.
Moving the P value threshold from .05 to .005 will shift about one-third of the statistically significant results of past biomedical literature to the category of just “suggestive.”1 This shift is essential for those who believe (perhaps crudely) in black and white, significant or nonsignificant categorizations. For the vast majority of past observational research, this recategorization would be welcome. For example, mendelian randomization studies show that only few past claims from observational studies with P < .05 represent causal relationships.5 Thus, the proposed reduction in the level for declaring statistical significance may dismiss mostly noise with relatively little loss of valuable information. For randomized trials, the proportion of true effects that emerge with P values in the window from .005 to .05 will be higher, perhaps the majority in several fields. However, most findings would not represent treatment effects that are large enough for outcomes that are serious enough to make them worthy of further action. Thus, the reduction in the P value threshold may largely do more good than harm, despite also removing an occasional true and useful treatment effect from the coveted significance zone. Regardless, the need for also focusing on the magnitude of all treatment effects and their uncertainty (such as with confidence intervals) cannot be overstated.
Lowering the threshold of statistical significance is a temporizing measure. It would work as a dam that could help gain time and prevent drowning by a flood of statistical significance, while promoting better, more-durable solutions.6 These solutions may involve abandoning statistical significance thresholds or P values entirely. If any thresholds are to continue in use, even lower thresholds are probably preferable for most observational research. Comprehensive reviews (termed umbrella reviews) that have evaluated multiple systematic reviews of observational studies propose a P < 10−6 threshold.5 In addition, falsification end-point methods (ie, using such Pvalue thresholds that almost all well-established null associations will not be able to pass them) also point to very low P values.7 With the advent of big data, statistical significance will increasingly mean very little because extremely low P values are routinely obtained for signals that are too small to be useful even if true.
Adopting lower P value thresholds may help promote a reformed research agenda with fewer, larger, and more carefully conceived and designed studies with sufficient power to pass these more demanding thresholds. However, collateral harms may also emerge. Bias may escalate rather than decrease if researchers and other interested parties (eg, for-profit sponsors) try to find ways to make the results have lower P values. Selected study end points may become even less clinically relevant because it is easier to reach lower P values with weak surrogate end points than with hard clinical outcomes. Moreover, results that pass a lower P value threshold may be limited by greater regression to the mean and new discoveries may have even more exaggerated effect sizes than before.
Because the proposed threshold of P < .005 is imperfect, other more difficult but more durable alternative solutions should also be contemplated (Table). These solutions vary based on how quickly and easily they can be adopted. They can target the use and interpretation of the past biomedical literature accumulated to date or the design and deployment of the new literature that will accumulate in the future. The situation is dire for the past literature because there is no perfect remedy after the fact. In the long-term, the scientific workforce will need to be more properly trained in using the best fit for purpose statistical inference tools and biases will need to be addressed preemptively rather than retrospectively. However, these may continue to be largely unachievable goals.

Various Proposed Solutions for Improving Statistical Inference on a Large Scale
Data are becoming more complex. If time for rigorous training in methods and statistics for researchers and for research users remains limited, subpar medical statistics and concomitant misinterpretations may continue. Nevertheless, hopefully several fields will adopt better standards for P values, will decrease their dependence onP values, and enhance the adoption of other useful inferential tools (eg, Bayesian statistics) when appropriate. The rapidity and extent of these changes is unpredictable. Low adoption in the past may cause some pessimism. However, a fresh start and a rapid acceleration of adoption of better practices is always possible. Incentives from major journals and funders as well as radical changes in training curricula may be necessary to achieve more widespread and effective shifts.
References
1. Chavalarias  D, Wallach  JD, Li  AH, Ioannidis  JP.  Evolution of reporting P values in the biomedical literature, 1990-2015.  JAMA. 2016;315(11):1141-1148.
2. Ioannidis  JP.  Why most published research findings are false.  PLoS Med. 2005;2(8):e124.
3. Wasserstein  RL, Lazar  NA.  The ASA’s statement on P-values: context, process, and purpose.  Am Stat. 2016;70(2):129-133.
4. Benjamin  DJ, Berger  JO, Johnson  VE,  et al.  Redefine statistical significance.  Nat Hum Behav. 2018;2:6-10.
5. Li  X, Meng  X, Timofeeva  M,  et al.  Serum uric acid levels and multiple health outcomes.  BMJ. 2017;357:j2376.
6. Resnick  B. What a nerdy debate about P values shows about science-and how to fix it.https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005. Accessed February 1, 2018.
7. Prasad  V, Jena  AB.  Prespecified falsification end points.  JAMA. 2013;309(3):241-242.

16.3.18

Do clinicians want recommendations? A multi-center study comparing evidence summaries with and without GRADE recommendations

Neumann I, Alonso-Coello P, Vandvik PO, Agoritsas T, Mas G, Akl EA, et al.
Journal of Clinical Epidemiology , Article in press.

Abstract


Background

Evidence-based clinical practice guidelines provide recommendations to assist clinicians in decision-making and to reduce the gap between best current research evidence and clinical practice. However, some argue that providing pre-appraised evidence summaries alone, rather than recommendations, is more appropriate.

Objectives

To evaluate clinicians’ preferences, understanding of the evidence and intended course of action in response to evidence summaries with and without recommendations.

Methods

We included practicing clinicians attending educational sessions across 10 countries. Clinicians were randomized to receive relevant clinical scenarios supported by research evidence of low or very-low certainty, and accompanied by either strong or weak recommendations developed with the GRADE system. Within each group, participants were further randomized to receive the recommendation plus the corresponding evidence summary or the evidence summary alone. We evaluated participants’ preferences and understanding for the presentation strategy as well as their intended course of action.

Results

189/219 (86%) and 201/248 (81%) participants preferred having recommendations accompanying evidence summaries for both strong and weak recommendations, respectively. Across all scenarios less than half of participants correctly interpreted information provided in the evidences summaries (e.g. estimates of effect, certainty in the research evidence). Presence of a recommendation resulted in a more appropriate intended course of action for two scenarios involving strong recommendations.

Discussion

Evidence summaries alone are not enough to impact clinicians’ course of action. Clinicians clearly prefer having recommendations accompanying evidence summaries in the context of low or very-low certainty of evidence (Trial registration NCT02006017).