{'fraction_non_alphanumeric': 0.039077348384495766, 'fraction_numerical': 0.017134135806024305, 'mean_word_length': 4.881944444444445, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Relative validity of a short food frequency questionnaire assessing adherence to the Norwegian dietary guidelines among colorectal cancer patients

Background The Norwegian food-based dietary guidelines (FBDG) aim at reducing the risk of developing chronic diseases and promote overall health. We studied the effect of the Norwegian FBDG in colorectal cancer (CRC) patients. There is a need for a time-efficient dietary assessment tool measuring adherence to these guidelines in patients treated for dietary dependent cancer, such as CRC patients. Objective To evaluate a new short food frequency questionnaire (NORDIET-FFQ), developed to estimate adherence to the Norwegian FBDG among CRC patients. Design Eighty-one CRC patients from both study groups in the Norwegian Dietary Guidelines and Colorectal Cancer Survival study, an ongoing dietary intervention, completed both the short 63-item NORDIET-FFQ and a 7-day weighed food record. Results The NORDIET-FFQ was on group level able to estimate intakes of fruits, vegetables, unsalted nuts, fish, fatty fish, high fat dairy products, unprocessed meat, processed meat, red meat, water, sugar-rich beverages, alcoholic drinks, and sugar- and fat-rich foods. Ranking of individuals according to intake was good (r = 0.31–0.74) for fruits and vegetables, fruits, unsalted nuts, whole grain products, sugar-rich cereals, fish, fatty fish, dairy products, red meat, water, sugar-rich beverages, alcoholic beverages, and sugar- and fat-rich foods. The NORDIET-FFQ was able to identify the individuals who did not fulfil the recommendations of fruits, vegetables, unsalted nuts, whole grains, low-fat dairy products, processed meat, water, alcoholic beverages, and sugar- and fat-rich foods (sensitivity: 67–93%). Conclusions The NORDIET-FFQ showed good ability in to estimate intakes of plant-based foods, fish, dairy products, meat, and energy-dense foods; adequate ranking of individuals according to intake of most recommendations except for unprocessed meat, processed meat, and vegetables; and importantly a good ability to identify those patients in need of dietary counselling for foods that are known to modulate the risk of CRC. Trial registration National Institutes of Health ClinicalTrials.gov; Identifier: NCT01570010.


2
(page number not for citation purpose)

Subjects and study design
Men and women aged 50-80 years old, with non-metastatic CRC (International classification of diseases (ICD)- 10 18-20), staged I-III according to the TNM (tumour node metastases) staging system (28), and participating in the CRC-NORDIET study (7) were invited to take part in the present validation study. The participants in the validation study were recruited from both intervention (n = 48) and control groups (n = 33) at the follow-up visit 6 months after baseline of the study, from January 2014 to October 2015. About 15% of the participants received chemotherapy post-surgery, of which the mean time from last chemotherapy injection to the validation study start was 155 days. Hence, none of the participants included in the validation study underwent adjuvant treatment during the time frame covered by the dietary assessment tools. All seasons during a year were included. The participants completed the self-administered NORDIET-FFQ at the study centre and received a digital scale and weighed record (WR), to be completed at home within 2 weeks (Fig. 1).

Ethics and approvals
The CRC-NORDIET study is being carried out in accordance with the Helsinki Declaration and informed consent was obtained from all participants. The study was approved by the Regional Committees for Medical and Health Research Ethics (REC Protocol Approval 2011/836) and by the data protection officials at Oslo University Hospital, Oslo, Norway and Akershus University Hospital, Lørenskog, Norway. The study is registered on the National Institutes of Health ClinicalTrials.gov (Identifier: NCT01570010).

Characteristics of the participants
Anthropometric measurements (weight, height, and hipand waist circumference) were measured as previously described (7). Level of education, smoking status, and tumour location were retrieved from other questionnaires within the clinical trial and from medical records.
Colorectal cancer (CRC) is the third most common cancer in Norway and second most common cause of cancer death (6). Low intake of whole grains, foods containing dietary fibre, dairy products and high intake of red and processed meat, alcoholic drinks as well as increased body fat have been associated with higher risk of developing CRC (4). All of these risk factors are included in the Norwegian FBDG, but the national guidelines have a much broader perspective than only these risk factors related to CRC.
Little is known about the effect of diet on disease outcomes and survival in CRC patients. We have therefore initiated a large, long-term randomised controlled trial (RCT) (CRC-NORDIET) to study the effect of diet in CRC patients post-surgery (7). Instead of only focusing on the dietary factors associated with risk of CRC, participants in the CRC-NORDIET study are instructed to follow a dietary pattern that is consistent with the Norwegian FBDG, since CRC patients have increased risk of lifestyle-related co-and multimorbidities.
It is therefore of interest to assess to what extent CRC patients in the CRC-NORDIET study comply with the Norwegian FBDG, both those foods that are causally related to CRC as well as those foods that are related to other lifestyle-related co-and multimorbidities.
Dietary intervention studies as well as nutrition education and counselling would benefit from a user friendly, short dietary assessment tool. In nutritional research, a variety of comprehensive dietary assessment tools are used, including food frequency questionnaires (FFQs), 24-h dietary recall, and food records (8). The FFQ is an established assessment method often used when investigating the effects of diet on disease outcomes in populations or groups of dietary interventions (9,10). Since most FFQs aim at capturing total habitual diet and therefore often include 200-300 questions, they are time-consuming for the respondents to complete, and data handling may be complex for the researcher (8,(11)(12)(13)(14)(15)(16)(17). Short FFQs are less time-consuming for both the patient and the researcher. Short FFQs designed to cover a recent time period (i.e. 1-2 months) have been shown to be useful for identifying dietary changes in individuals and in intervention studies (13,(17)(18)(19)(20) and may also be applicable to dietary counselling of patients in a clinical setting (11).
In recent years, a number of short FFQs have been developed to monitor adherence to food recommendations (11,(21)(22)(23)(24)(25)(26)(27); however, none of these assess adherence to a national FBDG. As part of the ongoing CRC-NOR-DIET study (7), we developed a short semi-quantitative FFQ (NORDIET-FFQ), designed to estimate the adherence to the Norwegian FBDG. The objective of the present study was to validate the ability of the NORDI-ET-FFQ to assess adherence to the Norwegian FBDG in CRC patients.  recommendations of the Norwegian FBDG as defined in Table 1. The main recommendations are listed in Table 1 together with the quantitative limits required to fulfil the Norwegian FBDG. The last two columns of Table 1 list all the specific questions in the NORDIET-FFQ that are included in calculation of adherence to the recommendations. Whole grain intake was estimated from whole grain products using a whole grain factor (30, 31). The lower range of recommended fish intake was used in the definition of daily intake (i.e. 300 g/week or 43 g/d).

The translation of qualitative Norwegian FBDG into quantitative limits
In order to measure adherence to the qualitative recommendations of the Norwegian FBDG we had to translate these qualitative recommendations into quantitative values and lower limits of intake ( Table 2). The quantitative values used in the present paper are listed in the footnotes of Table 2. The last two columns of Table 2 list all the specific questions in the NORDIET-FFQ that are included in calculating adherence to the recommendations. Qualitative recommendations that included the terms 'limit' or 'reduce' were quantitatively defined as the highest acceptable amount of daily intake. For example, the recommendation regarding processed meat reads, 'Limit the intake of processed meat'. In this case we a priori set the quantitative limit required to fulfil the Norwegian FBDG to ≤20 g/d. This is equivalent to a fast-food meal or dinner with processed meat once a week. Moreover, the term 'preference' included in the recommendation of unprocessed meat (i.e. 'Preference should be given to the consumption of unprocessed meat') was defined as meaning that intake of unprocessed meat should exceed the intake of processed meat.
For recommendations where a daily intake of specific food items was specified, this was defined as the minimum amount that should be consumed daily with some modifications. For example, a moderate intake of unsalted nuts (about 140 g/week or 20 g/d) is recommended. However, because nuts are protein and energy dense, high intakes may lead to weight gain. Therefore, the recommendation of unsalted nuts was defined as a daily intake of at least 20 g or more among normal weight [body mass index (BMI) < 25] individuals and between 20 and 30 g/d among overweight and obese individuals (BMI ≥ 25). In the case of dairy products, the recommendation states that 'Low-fat dairy products should be included in your daily diet'. The recommended daily amount was defined as at least half a portion of low-fat dairy products per day. Water is recommended as the primary choice of beverage; however, there is no quantitative daily recommendation of water intake in the Norwegian FBDG. Therefore, we determined that, to fulfil this recommendation, at least 25% of the daily beverage intake should be water.

NORDIET-FFQ
The NORDIET-FFQ is a short semi-quantitative 63-item FFQ designed to assess dietary intake (in grams per day) over the previous 1-2 months. It takes on average 15 min to complete. The questions in the NORDIET-FFQ correspond to the food groups relevant for the Norwegian FBDG (3). The NORDIET-FFQ is not designed for estimation of total energy or nutrient intake.
The NORDIET-FFQ included questions of both frequency (how often the food item was consumed) and amount of the food items. The 63 questions cover the following food groups: fruit, berries, nuts, vegetables, cereals, beverages, cakes, sweet candy, breads and spreads, oils, margarine and butter, dairy products, fish, meat, rice, pasta, and dietary supplements. The questionnaires were checked for completeness by the researchers, so that incomplete answers could be corrected. Data from the NOR-DIET-FFQ were scanned and the image files transformed into data files using Cardiff Teleform 2006 Software (6.0) (Datascan, Oslo, Norway). The software proofread the answers in the NORDIET-FFQ pending approval by the handler. When values were missing the following rules were used: (1) when frequency was reported but amount was missing, the lowest amount was registered; (2) when amount was reported but the frequency was missing, the lowest frequency above 0 was registered; (3) when both frequency and amount were missing, they remained as missing values; (4) if the amount of food was reported and the frequency was reported as zero, the amount was removed.
The food composition database and nutrient calculation system developed at the Department of Nutrition, University of Oslo (KBS, version 4.7, 2010, AE-10), was used for the calculations of food and beverage intake (29).

Seven-day weighed food record
All participants were provided with a WR and a digital scale (5 kg kitchen scale, Clas Ohlson Model CFC2025, Oslo, Norway) and were instructed how to weigh and record all foods and beverages consumed during a period of 7 consecutive days. The participants returned the completed WRs to the study centre at Department of Nutrition, University of Oslo, by postal mail. Dietary data were retrieved from the WR and manually coded and imported into the food database AE-10 and KBS software system (KBS, 2010). The manual coding and import of data were done by two researchers (HBH and SFØ), in accordance with a protocol developed at the Department of Nutrition, University of Oslo. All of the 81 WRs were included in the analyses.

Dietary recommendations
Quantitative Norwegian FBDG The questions about food and beverage intake in the NOR-DIET-FFQ were grouped according to the quantitative 4 (page number not for citation purpose)  Boiled rice and pasta contain 70% water and 30% cereal. Whole grain factor used in calculation of whole grain intake from rice and pasta: • Brown rice = 0.30 • White rice = 0 • Whole grain pasta = 0.30 • White pasta = 0 e Not including shellfish, mussels, or roe.  Acceptable amount of intake equal to one portion per week. f Defined as dairy products (not cheese and milk) containing less than 20% fat and dairy products labelled as light or reduced fat or containing less than 950-1,150 kJb energy. g Defined as cheese containing less than 17% fat, cheese labelled as light or reduced fat, or containing less than 950-1,150 kJ energy. h Should include at least half portion per day (1 portion = 1 glass of lean milk = 200 g). i Whole milk with more than 3.5% fat. j Defined as dairy products (not cheese and milk) containing more than 20% fat and/or energy content more than 950-1,150 kJ. k Defined as cheese containing more than 17% fat, cheese not labelled as light/reduced fat, or containing more than 950-1,150 kJ energy.

Sample size
A sample size of 40 men and 40 women allows the detection of differences of one portion of fruit or vegetable (one portion = 100 g) between test and reference methods, assuming a standard deviation of 1.6 portion (or 160 g) (32, 33) with a significance level of 5% and power of 80%. Moreover, a sample size of 38 men and 38 women was needed to detect a correlation coefficient of 0.5 or higher, with a significance level of 5% and power of 90% (34).

Statistical analysis
Data were analysed using IBM SPSS Statistics, version 22. All p-values were two-sided with a significance level of 5%. All data were checked for normal distribution by evaluating histograms, normal Q-Q plots, and the Kolmogorov-Smirnov test (p > 0.05).
All subject characteristics were normally distributed and are presented as means with 95% confidence interval. The categorical data are presented as frequencies and 7 (page number not for citation purpose) Spearman's rank order correlation (rho) was calculated to explore the strength of the relationship between the continuous variables from the two different methods. We used the levels of agreement between two methods as defined by Hankin et al. (35), of which a correlation below 0.3 is poor, between 0.3 and 0.49 is fair, and above 0.5 is satisfactory. Kappa correlation was used to explore the strength of the relationship between the categorical variables of 'Oil, margarine, and butter' from the two different methods. Bland-Altman plots were used to explore bias such as over-or under-reporting (estimated by mean differences), limits of agreement (mean difference ± 1.96 SD), and presence of outliers in the data (36,37).
To evaluate the participants' adherence to the dietary recommendations as described in Tables 1 and 2, we calculated the NORDIET-FFQs sensitivity and specificity compared with the WR. Sensitivity was defined as the percentage of subjects who reported not fulfilling the recommendations for both the NORDIET-FFQ and WR assessments divided by the number of patients not fulfilling the recommendations according to the WR only. Specificity was defined as the percentage of subjects who reported fulfilling the recommendations for both the NORDIET-FFQ and WR assessments divided by the number of subjects fulfilling the recommendations according to the WR only. Sensitivity and specificity above 60% was defined as good.

Results
Eighty-one participants accepted the invitation (92% participation rate, Fig. 1). General characteristics of the study population are presented in Table 3. Daily mean energy intakes estimated from the WR were 8.9 and 7.6 MJ for men and women, respectively.

Intakes of food and beverages from the NORDIET-FFQ and WR
Median intakes of food and beverages estimated from the NORDIET-FFQ and the WR are presented in Table 4. Overall, the NORDIET-FFQ was able to estimate intake of the main food groups in the Norwegian FBDG and that are associated with cancer risk, except for whole grain products, water, and red and processed meat. Correlation coefficients between intakes estimated from the NORDI-ET-FFQ and WR are presented in Table 5. Correlation coefficients ranged from 0.12 for unprocessed meat to 0.74 for 8 (page number not for citation purpose) Includes low-fat dairy products (containing less than 20% fat), reduced-fat cheese (less than 17% fat), and lean milk (less than 1.5% fat).
f Includes high-fat dairy products (containing more than 20% fat), high-fat cheese (more than 17% fat), and whole milk (more than 3.5% fat).
g Total meat intake = unprocessed + processed meat.

9
(page number not for citation purpose) Includes low-fat dairy products (containing less than 20% fat), reduced-fat (less than 17% fat) and lean milk (less than 1.5% fat). d Includes high-fat dairy products (containing more than 20% fat), high-fat cheese (more than 17% fat), and whole milk (more than 3.5% fat). *Correlation is significant at the 0.01 level (two-tailed).
alcoholic beverages. However, most food groups showed fair and satisfactory correlations, with the exception of poor correlations for unprocessed meat, processed meat, and vegetables. Bland-Altman analyses are presented in Fig. 2 and Supplementary file 3. The majority of the plots (i.e. differences between methods on the y-axis against the mean value of methods on the x-axis) were within the 95% limit of agreement for each food group. At the upper level of intake of foods, there was a wider scatter of difference. Sensitivity and specificity analyses are presented in Table 6. The NORDIET-FFQ was able to identify individuals in need of dietary counselling for most of the guidelines, with the exception of red meat, unprocessed meat, fish, and sugar-rich beverages. Estimated median intakes of the individual food items (ungrouped) are presented in Supplementary file 1, with associated correlations coefficients between test and reference method in Supplementary file 2.

Fruits, berries, vegetables, and nuts
Median intakes of the food groups 'Fruits, berries, and vegetables', 'Fruits and berries', and 'Vegetables' did not differ significantly between the methods, for all participants in total or when divided by gender (Table 4). However, the limits of agreement were wide and under-reporting increased at high intakes ( Fig. 2a and b, Supplementary file 3A). The Spearman's rho was fair for the first two food groups, but poor for 'Vegetables' ( Table 5). The questions about fruits and vegetables in the NORDIET-FFQ showed sensitivity in the range of 84-87% and a low specificity ranging from 14 to 50% (Table 6). The median intake of unsalted nuts did not differ significantly between the NORDIET-FFQ and WR for men only. Moreover, with regard to differences between the two methods the limits of agreements were 20 g above and below the mean difference (Fig. 2c). Spearman's rho was satisfactory (Table 5), the sensitivity was good, and specificity was low ( Table 6).

Whole grain products
There was significant difference in median intakes of whole grain products and cereals with high content of fat and sugar between the two methods but with satisfactory Spearman's rho (Tables 4 and 5). The NORDIET-FFQ tended to report higher intakes in the category of 'Whole grain products' on the group level, with almost half of the recommended daily intake, which increased with higher intakes in both women and men (Fig. 2d). However, the NORDIET-FFQ was able to identify individuals not fulfilling the dietary recommendation for whole grain products among men and women, respectively (Table 6).

10
(page number not for citation purpose) The under-reporting of intake of the category 'Cereals with high content of fat and sugar' showed a trend towards increasing differences between methods with higher intakes (Fig. 2e). The questionnaire was only able to identify individuals fulfilling the recommendations of 'At least half of the grains should be whole grains' and 'Reduce cereals with a high content of fat, salt, and sugar' ( Table 6).

Fish
Median intakes of fish did not differ significantly between the two assessment methods on group level. Estimated intakes of fatty fish were significantly different for women but not for men ( Table 4). The differences in intakes of fish and fatty fish between the methods did not show any trend in the distribution but were scattered above and below the 11 (page number not for citation purpose) red meat showed a fair Spearman's rho, whereas unprocessed meat and processed meat showed a poor Spearman's rho ( Table 4). The limits of agreements were almost within 140 g ( Fig. 2h and i, Supplementary file 3D). The NORDIET-FFQ was only able to identify individuals in need of dietary counselling for intakes of processed meat (Table 6). However, the NORDIET-FFQ was able to identify those who followed the recommendations for intakes of red meat and unprocessed meat (specificity, 90 and 82%) but not for processed meat (specificity of 37%) ( Table 6).

Oil, margarine, and butter
The NORDIET-FFQ was able to identify participants who fulfilled the recommendations for intakes of dietary fat in their diets but not those who did not fulfil the recommendations (Table 6). Moreover, the measure of agreement between the methods was poor (kappa coefficient = 0.29, p <0.006).

Water and other beverages
Median intakes of water were significantly different for women and the total population and the Spearman's rho was fair (Tables 4 and 5). The underestimation of water from the NORDIET-FFQ increased with higher intakes mean differences (Fig. 2f and Supplementary file 3B). The Spearman's rho was fair, sensitivity was low, and the specificity was high for both food groups (Tables 4 through 6).

Dairy products
The median intake of low-fat dairy products was significantly different between the two methods, but the Spearman's rho was satisfactory (Tables 4 and 5). The differences between the methods were evenly distributed above and below the mean difference, which showed a mean under-reporting of 26 g/d on the group level. The limits of agreement were mostly within the amount of two glasses of milk (Fig. 2). The median intake of high fat dairy products was not significantly different between the methods, and the Spearman's rho was fair for the total population, poor for men, and satisfactory for women (Tables 4 and 5). Both sensitivity and specificity were high for low fat dairy products, but the sensitivity was lower for high fat dairy products (Table 6).

Meat
The NORDIET-FFQ was able to estimate intakes of unprocessed meat, red meat (only women), and processed meat (only women) on the group level (Table 4). Moreover, Table 6. Sensitivity and specificity of the NORDIET-FFQ to detect participants not complying or complying with the Norwegian FBDG relative to WR

12
(page number not for citation purpose) Moreover, stratifying by gender increased the estimation of intakes for additional food groups. Gender differences in dietary assessment methods have been reported in other studies as well (8,14,41). Lee and co-workers (41) emphasise the importance of including different portion sizes for men and women in FFQs, due to their findings that when gender is not considered, greater inaccuracy in dietary intake assessment is found in women compared to in men. In the present study, portions sizes were equal for men and women and they reported intakes of nuts, fish, water, meat, and sugar-and fat-rich foods differently on the NORDIET-FFQ.
Short FFQs have been shown to be able to classify individuals according to intakes of food groups and nutrients (12,15,20,23,36,37). In the present study, the NORDIET-FFQ showed fair to satisfactory agreement with respect to ranking individuals by their dietary intake compared to WR. The Spearman's rho ranged from 0.12 to 0.74, with most correlations categorised as fair or satisfactory and statistically significant (p < 0.01) (Tables 5 and  Supplementary file 2). This is consistent with other similar studies showing a correlation coefficient for foods and nutrients ranging from 0.3 to 0.7 (13,15,17,42). Carlsen and co-workers found that correlations between intakes of fruit and vegetable from a long FFQ and WR ranged from 0.31 to 0.58 (43). Importantly, in the present study, a fair or satisfactory correlation was documented for the foods that are shown to be associated with risk of CRC, such as fruits and vegetables, whole grains, dairy products, red and processed meat, and alcoholic beverages (2)(3)(4)44). However, three food groups showed poor correlations for both genders and in total, that is vegetables, unprocessed meat, and processed meat. Classifying individuals according to intake of food groups may be advantageous in clinical intervention studies as a measure of intervention effects, as well as in a clinical setting to identify patients 'at risk' (23). Based on the high number of fair and satisfactory correlations observed in this study the NORDIET-FFQ's ability to rank participants according to food intake is fair and comparable to other short FFQs (11,17,20,26,(45)(46)(47)(48).
Correlation coefficients measures associations between a questionnaire and its reference method but are unable to detect systematic errors that may be of clinical importance (37).
When assessing the agreements between two methods, Bland-Altman plots are recommended (34,36,37). Overall, the Bland-Altman plots in the present study showed wide limits of agreement for most of the food groups and a trend towards increased over-or under-reporting with higher mean intakes. However, the mean differences were smaller than 20% between the methods for most of the food groups and the limits of agreement were almost within a daily portion for fruits, vegetables, nuts, fish, meat, and dairy products. Systematic and random errors (Fig. 2j). Median intakes of beverages with added sugar were not significantly different between the methods and the Spearman's rho was fair for all participants in total, and satisfactory and poor among men and women, respectively (Tables 4 and 5, Fig. 2k). There were no significant difference between the methods in median intakes of alcoholic drinks in total and the Spearman's rho was satisfactory (Tables 4 and 5). The sensitivity analyses of beverages ranged from 52% ('Reduce sugar rich beverages') to 97% ('No alcoholic intake'), and the specificity ranged from 49% ('Water is recommended as the primary choice of drink') to 89% ('Reduce sugar-rich beverages') ( Table 6).
Sugar-and fat-rich foods The NORDIET-FFQ was able to estimate median intakes and to rank individual intakes of sugar-and fat-rich foods on the group level (Tables 4 and 5, Supplementary file 3E). The NORDIET-FFQ was also able to identify those not fulfilling the recommendation and moderately those who fulfilled the recommendation (Table 6).

Discussion
In the present study, we evaluated the NORDIET-FFQ's ability to assess adherence to the Norwegian FBDG in CRC patients. The NORDIET-FFQ was able to estimate intakes of the main dietary guidelines, such as fruits, vegetables, fish, meat, high-fat dairy products, beverages, and energy-dense foods. Only three food groups showed significant differences between methods for both genders, of which 'Low-fat dairy products' and 'Cereals with high content of sugar and fat' were under-reported and the 'Whole grain products' was over-reported in the NORDI-ET-FFQ compared to the WR. We speculate that the under-reporting of unhealthy cereals and the over-reporting of healthy whole grain products may be a result of social desirability bias, as seen in other studies, where presumed unhealthy foods are under-reported and healthy foods are over-reported (38)(39)(40). The underreporting of 'Low fat dairy products' may also be explained by participants having difficulty interpreting the dairy questions and not knowing if the dairy products they use are high-or lowfat products. There is a wide range of dairy products on the market with varying fat content and a valid intake estimate of this food category relies heavily on participants' knowledge of fat content in the products they consume. Thus, this may reduce the ability of the NORDIET-FFQ to estimate intakes of low-fat products. The estimate of the category 'Low fat dairy products' was an aggregation of the following three entries: 'Low-fat milk', 'Cheese with low fat content', and 'Dairy products with low-fat content'. Each of these entries gives valid estimates of intakes. Thus, the single questions regarding dairy foods gave better estimates when used separately.

13
(page number not for citation purpose) and should not be dependent on differences in diet among the participants. Both groups were equally exposed to the dietary assessment methods, anthropometric measurements, and biological samplings, as well as the direct contact with the researchers. However, we cannot rule out that the intervention group might have been more conscious about the registration of food and beverages than the participants in the non-intervention group.
FFQs are associated with limitations such as fixed food list, memory, and perception of portion sizes (8). However, WRs control these errors due to their independency of memory and direct measurement of food intakes (8,16). In order to be used as a standard reference method in assessing the validity of questionnaires, WRs should cover a sufficient number of days to represent average dietary intakes (8). In the present study this was taken into account, since all the WRs recorded dietary intakes over 7 consecutive days.
As the aim of the NORDIET-FFQ was to measure adherence to the dietary guidelines, an aggregation of the single foods was needed. Hence, using the aggregated questions conforming to the Norwegian FBDG in the NORDIET-FFQ resulted in less detailed information of food intake. However, short FFQs containing aggregated questions have been shown to capture nearly as much information on dietary intakes as long FFQs (8,16).

Conclusion
The NORDIET-FFQ was able, on a group level, to estimate intakes of most food groups related to the Norwegian FBDG, such as fruits, vegetables, nuts, fish, dairy products, meat, beverages, and sugar-and fat-rich foods. Moreover, the NORDIET-FFQ was able to rank individual intakes and to identify those individuals in need of dietary counselling for foods that are shown to be associated with risk of CRC, such as fruit and vegetables, whole grains, red meat, alcoholic beverages, and dairy products. The NORDIET-FFQ was not able to rank individual intakes of processed meat, unprocessed meat, and vegetables. Overall, the NORDIET-FFQ gives valid estimates of dietary intake according to the Norwegian FBDG.

Conflict of interest and funding
All other authors declare that they have no competing interests. This project has received funding from the Research Council of Norway, Throne Holst Foundation of Nutrition Research, Norwegian Cancer Society, and South Eastern Norway Regional Health Authority.
can more easily be revealed by Bland-Altman plots, as the data points are less compressed compared to a scatter plot. For instance, in the present study 'Whole grain products' showed a satisfactory correlation, but a systematic error was revealed by the Bland-Altman plot, showing increasing over-reporting with increasing intakes. Moreover, 'Fish' showed a fair correlation and the distribution of differences against the mean value of the two methods did not show any clear trend. Intakes in the category of 'Vegetables' showed poor correlation and an increased under-reporting with higher intakes, indicating poor ability of the NORDIET-FFQ to measure intakes of vegetables. This may be due to the difficulty of reporting portions of vegetables, since these foods are often included in dishes, compared to fruits, which are more often eaten raw in one unit (43). However, the questionnaire was good in measuring intakes of vegetables on the group level and according to the recommendations. Thus the NORDIET-FFQ gave a fairly good estimate of intake for several of the food groups on the group level compared to the WR standard.
The NORDIET-FFQ was able to detect individuals not fulfilling the Norwegian FBDG (i.e. sensitivity) for 10 out of 20 recommendations, covering most of the food groups shown to be associated with CRC risk (i.e. fruits and vegetables, whole grains, processed meat, alcohol intake, dairy products) (1)(2)(3)(4)44). Moreover, the NORDIET-FFQ was able to identify individuals who fulfilled 13 out of 20 recommendations (i.e. specificity) ( Table 6).

Strengths and limitations
Attenuation of agreement between two methods can occur due to different time period covered by the methods. In the present study we compared a retrospective method (NOR-DIET-FFQ) with a prospective method (WR). A limitation of the study may be that the NORDIET-FFQ asked for dietary intake over the previous 1-2 months, whereas the WR recorded dietary intakes for the subsequent week. However, the short time period covered by this study design, approximately 2.5 months, would probably limit the error between NORDIET-FFQ and WR recording (8,14). Timing and sequence for the test and reference method is important in validation studies, of which the test method should be administrated prior to the reference method (14). In the present study, the participants completed the FFQ prior to the WR and thereby avoided any learning effects from the reference method. Seasonal effects of dietary intakes were not expected since dietary intakes were collected from all seasons during a year. Moreover, variation in reporting of dietary intakes may be attributed to the comparison between closed and open ended methods (14). Study participants were recruited from both arms of the intervention. The purpose of an evaluation study is to compare estimates of intake of test and reference method. Hence, the diet of the participants should be assessed alike