{'fraction_non_alphanumeric': 0.02615339406406112, 'fraction_numerical': 0.007640317367029092, 'mean_word_length': 5.257352941176471, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Measurement and methodology in data synthesis research: Commentary on Roizen et al. ()

Roizen, Fillmore, Chikritzhs, and Stockwell (2013) rightly point out that we currently lack understanding of the reasons why alcohol abstainers have a higher risk of cognitive decline and dementia than light to moderate drinkers. They argue that former drinking bias may influence the results for ‘alcohol abstainers’ due to what has been described as ‘misclassification’ (Mertens, Moos, & Brennan, 1996). ‘Abstainers’ include former problem drinkers as well as lifetime abstainers. Hence it is possible that the increased risk of dementia in abstainers is due to risk associated with former problem drinkers, rather than the relative benefit of alcohol consumption per se. Misclassification has been investigated in other areas of research linking alcohol consumption to health outcomes but has not necessarily led to the conclusion that abstainers with no history of alcohol drinking are as healthy as drinkers. Research on risk of mortality in relation to alcohol consumption has shown that over 60% of ‘abstainers’ were former problem drinkers. However, when analysed separately, there was only a slight difference in mortality risk between types of abstainers (Mertens et al., 1996). Other research on cardiovascular disease (CVD) has shown that inclusion of ‘sick quitters’ (abstainers who abstain due to poor health) in the abstainer groups does not explain the finding of lower rates of CVD among drinkers (Marmot & Brunner, 1991). Roizen et al. have not acknowledged methodological issues in this field relating to dementia research and meta-analysis. For example, it is essential for any valid meta-analysis of a risk factor for dementia to exclude studies that include participants with dementia at baseline and when this does not occur the study results are not reliable. However, the authors have not critically evaluated studies on the basis of this fundamental issue which is as important is the inclusion of former drinkers in the abstainer category. Poor measurement of either the exposure or outcome measure, or poor research design will each lead to inaccurate findings. The choice of studies included in meta-analyses is an important aspect contributing to the quality of the meta-analysis and will influence the outcome of the results. Meta-analyses require very detailed evaluation of measures, wording of items and evaluation of ‘quantities’ according to different national guidelines. The process of harmonisation requires categories to be collapsed in order to pool studies and is by nature often a compromise to enable data to be synthesised. Anstey, Mack, and Cherbuin (2009) included a long section in the discussion outlining study limitations which were not acknowledged in the critique by Roizen, see also Anstey (2008). The three meta-analyses referred to in this commentary yielded similar conclusions, and hence are an important attempt to draw together disparate literature and draw some conclusions about their inter-relationship. Re-iterating the widely acknowledged measurement issues in this field, most of which were addressed in each of these reviews, (albeit imperfectly) does not advance the field. Re-analysis of datasets to evaluate potential methodological biases may resolve concerns about issues such as misclassification, and is a more constructive future direction.