{'fraction_non_alphanumeric': 0.03401288313159199, 'fraction_numerical': 0.01879379910809089, 'mean_word_length': 5.01938645078824, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Effect of sample size and P-value filtering techniques on the detection of transcriptional changes induced in rat neuroblastoma (NG108) cells by mefloquine

Background There is no known biochemical basis for the adverse neurological events attributed to mefloquine. Identification of genes modulated by toxic agents using microarrays may provide sufficient information to generate hypotheses regarding their mode of action. However, this utility may be compromised if sample sizes are too low or the filtering methods used to identify differentially expressed genes are inappropriate. Methods The transcriptional changes induced in rat neuroblastoma cells by a physiological dose of mefloquine (10 micro-molar) were investigated using Affymetrix arrays. A large sample size was used (total of 16 arrays). Genes were ranked by P-value (t-test). RT-PCR was used to confirm (or reject) the expression changes of several of the genes with the lowest P-values. Different P-value filtering methods were compared in terms of their ability to detect these differentially expressed genes. A retrospective power analysis was then performed to determine whether the use of lower sample sizes might also have detected those genes with altered transcription. Results Based on RT-PCR, mefloquine upregulated cJun, IkappaB and GADD153. Reverse Holm-Bonferroni P-value filtering was superior to other methods in terms of maximizing detection of differentially expressed genes but not those with unaltered expression. Reduction of total microarray sample size (< 10) impaired the capacity to detect differentially expressed genes. Conclusions Adequate sample sizes and appropriate selection of P-value filtering methods are essential for the reliable detection of differentially expressed genes. The changes in gene expression induced by mefloquine suggest that the ER might be a neuronal target of the drug.


Background
Mefloquine (Lariam) is a prophylactic antimalarial that is also used for malaria chemotherapy. Adverse central nervous system (CNS) events have been associated with its use. Severe CNS events requiring hospitalization occur in 1:10,000 and 1:200-1200 patients taking mefloquine for chemoprophylaxis and treatment, respectively [1]. Milder CNS events (e.g. dizziness, headache and insomnia) are a more frequent occurrence, occurring in up to 25% of those receiving chemoprophylactic doses and 90% of patients receiving therapeutic doses [1]. Higher blood levels of mefloquine are reached under prophylactic as compared to therapeutic regimens [1,2]. The relative incidence of adverse effects is, therefore, probably dose-related, although the concomitant effect of malaria during treatment cannot be dismissed. It is likely, then, that the neurological events associated with mefloquine have a biochemical basis. In this study, an attempt was made to deduce a possible mechanism of action for mefloquine in rat neuronal cells using Affymetrix rat toxicology arrays.
Microarray analysis offers the unique potential to identify unknown targets of toxic agents, as transcriptional responses of the entire genome can be measured in parallel [3]. Ideally, one should be able to identify new targets quickly, confidently, and without recourse to alternative methods. Appropriate selection of a method for filtering gene expression data is therefore critical to this process. One of the first definitions to emerge was the arbitrary designation of a particular level of -usually two-fold up or down regulation -gene expression as representing 'significance' [4,5]. Such arbitrary definitions emerged from the observation that fold-regulation of genes between control cultures with identical cell populations seldom varies by more than this level (discussed by Ideker et al. [6]).
However, arbitrary designations cannot be considered 'significant' in the traditional, statistical sense unless experimental variance is taken into consideration. An evolving method of analysis is to define significant changes in gene expression in terms of a particular P-value after performing appropriate statistical tests that take into account the variability of gene expression data and sample size [6][7][8][9][10]. However, care must be taken to use appropriate statistical tests, P-value thresholds for significance, and sufficient n, otherwise, variance-based methods, as with less rigorous fold-change approaches, will generate high error rates. Recent studies have discussed the utility of the 'Z score', the parametric t-test, and the nonparametric Wilcoxon rank sum test for expression profiling [9,10]. However, the effects of inadequate sample size and P-value correction methods are only beginning to be addressed [11].
Due to restrictions on the type and availability of biological samples and the prohibitive cost of arrays, many array studies have resorted to the use of extremely low sample sizes (for a recent example see Lang et al. [12]). This is potentially problematic because the power of statistical tests decreases with sample size. There is also the multiplicity problem [13]. As the number of hypotheses being tested increases so does the number of type I errors (false conclusions of significance). This is of great concern in microarray studies given the number of statistical comparisons being made (i.e. one test per gene on an array). Therefore, P-value correction is essential in expression profiling to control an appropriate type 1 error rate, although undue conservatism may result in the failure to detect transcrip-tional changes for some genes that might indeed be verifiable by other means. As shown in this study, adoption of an experimental design that incorporates an adequate sample size and appropriate selection of a P-value filtering method is critical if genes with altered transcription are to be efficiently and effectively identified.

Reagents and media
Mefloquine was obtained from Walter Reed Army Institute of Research chemical repository. Dulbecco's Modified Eagle Medium (DMEM), hypoxanthine-aminopterin-thymidine (HAT) medium supplement, foetal calf serum (FCS) and gentamycin were purchased from Gibco BRL (Rockville, Maryland). RNA-STAT was obtained from Tel-Test (Friendswood, Texas).
Cell maintenance NG108-15 (mouse neuroblastoma-rat glioma hybrid) cells were maintained in 75-cm 2 tissue culture flasks in DMEM supplemented with HAT, 10% FCS and gentamycin (50 µg/ml), in a humidified 6.0% CO 2 incubator at 37°C. For the microarray studies, 175 cm 2 tissue culture flasks were seeded with 4.6 million NG108 cells in 49.6 ml culture medium 24 h prior to the experiments. For cytotoxicity studies, 25 cm 2 tissue culture flasks were seeded with 0.66 million NG108 cells in 10 ml culture medium 24 h prior to the experiment.

Cytotoxicity of mefloquine in NG108 and primary rat neuronal cell cultures
The cytotoxicity of mefloquine was assessed using 25 cm 2 tissue culture flasks. After overnight incubation of NG108 cells, culture media were replaced with fresh DMEM containing mefloquine (2.5-40 µM) or 1% DMSO (control). After incubation of the flasks for 2 h, the cells were washed twice, and then resuspended, in 5 ml phosphate buffered saline. Total numbers of viable cells at each mefloquine concentration were determined using trypan blue exclusion as previously described [14]. Viability (%) was calculated using the following formula: Viability (%) = # viable cells in treated culture/# viable cells in control culture * 100. Data shown represent the mean (%) viability (± SEM) for three experiments. The cytotoxicity of mefloquine to primary embryonic rat neurons was assessed in 24 well tissue culture plates using the MTT assay as previously described [15]. Data represent mean (%) viability (± SEM) for eight replicate experiments. Fifty percent inhibitory concentrations (IC 50 s) were calculated using Prism software.

Design of microarray experiments, cell harvesting and total RNA extraction
On the day of the experiment, the media was removed from the seeded flasks and replaced with 70 ml DMEM supplemented with either 0.25% DMSO (controls) or 10 µM mefloquine in 0.25% DMSO (treated). The cells were treated with mefloquine or DMSO for 2 h. This treatment regime reduces the viability of NG108 cells by approximately 35% (Figure 1). After incubation, media was removed and replaced with 15.0 ml RNA STAT (Tel-Test, Friendswood Texas). Total RNA was then extracted according to the manufacturer's instructions. Eight pairs of RNA samples (eight control and eight treated samples) were collected on different occasions.

CDNA synthesis, in vitro transcription and fluorescent labeling, hybridization, staining and scanning of gene chips, and assay monitoring
Detailed procedures for preparation of cDNA and fluorescently labeled cRNA, hybridization, staining, and scanning of gene chips and assay monitoring are outlined by Vahey et al. [16]. The platform chosen for global expression profile was the Rat Tox U34 Array (Affymetrix, Santa Clara, California), which contains probes for EST clusters and genes linked to a variety of toxic endpoints (total of 1031 probe sets including controls). RNA (10 µg) extracted from each individual flask was hybridized to a single gene chip (i.e. a total of 16 chips were used).

Gene expression data analysis
Affymetrix analysis software (version 4) was used to generate average difference (AD) values for each gene for each treatment (Affymetrix, Santa Clara, California). AD values represent the difference in mean fluorescence between positive and mismatch probe cells for each gene. All genes with mean AD levels < 100 in either mefloquine or DMSO-treated cultures were excluded. This procedure eliminated most of the genes called absent by the Affymetrix software. For simplicity, an AD value is hereafter referred to as the expression value of a gene. The expression values for each gene chip were imported directly into Parteck Pro 2000. No additional data normalization or scaling methods were employed (as these procedures were performed previously by the Affymetrix software). No additional filtering of data was conducted on the basis of either number of reporting probe cells or the present/absent calls generated by the Affymetrix analysis software. Paired t-tests (two-sided, df = 7) were performed to compare the expression levels of each of the remaining genes (695 of 1031) in DMSO and mefloquine-treated cultures. The genes were then rank ordered in terms of their unadjusted P-values. This general approach to the analysis of Affymetrix expression data is outlined in Partek technical literature. Fold-changes (FC) in expression were = mean mefloquine expression level/mean DMSO expression level.

Figure 1
Dose response curves for mefloquine against rat neuroblastoma (NG108) and primary embryonic rat neuronal cells.

Expression data generated for this study is available from GEO (Accession #GSE39 for a summary of the experiment and GSM1654-1669 for individual treatments)
RT-PCR validation PCR following reverse transcription was performed for semi-quantitative determination of steady-state expression of RNA for tubulin (negative control) and six of the eleven lowest P-value ranked transcripts (see RT-PCR conditions in Table 1). For each sample, 2 µg RNA was transcribed in a 20 µl reaction using the Invitrogen SuperScript First-Strand Synthesis System oligo-(dT) primer method according to the manufacturer's instructions. cDNA was RNase-treated (E. coli RNAse H at 37°C for 20 min) and diluted to 400 µl in molecular biology grade water prior to PCR. For each gene, 5 µl of cDNA was incorporated into a 25 µl PCR mixture containing 8.8 mM TrisHCL, 44 mM KCl, 1.3 mM MgCl 2 , 0.00088 % gelatin, 0.284 mM each deoxynucleoside triphosphate, 1 U of Taq DNA polymerase (Applied Biosystems, Foster City, CA) and 1.78 µM of each primer. After an initial denaturation of 94°C for 5 min, cycles consisted of 30 sec denaturation at 94°C, 30 sec of primer annealing at a gene specific annealing temperature and 60 sec of primer extension at 72°C, followed by a final elongation step of 72°C for 5 min. Cycle numbers were chosen such that amplification was in the linear range of detection. PCR products (20 µl) were electrophoresed on ethidium bromide-stained 1.2% agarose gels, revealed by UV illumination and analyzed densitometrically. Gene specific primer sequences, cycle times and annealing temperatures are outlined in Table 1. For each transcript, raw densitometry data was subjected to paired t-testing (two-sided, df = 7) to determine whether differences in expression existed in mefloquine and DMSO-treated cultures (P < 0.05).

Designation of a list of genes with altered expression
The mefloquine data set was used to compare several Pvalue correction and gene expression filtering methods. The methods were compared in terms of their ability (or failure) to detect genes defined as having (truly) altered expression. Genes with altered changes in expression were those with an array P < 0.003. This threshold was selected because of the good correlation of microarray and RT-PCR P-values and because it represents the highest unadjusted P-Value for which an associated mefloquine-induced expression change was confirmed by RT-PCR.

Descriptions of different gene expression filtering methods
Genes were filtered on the basis of their unadjusted P-values according to several different methods: (i) The normal P < 0.05 threshold (i.e. P < 0.05 for significance), (ii) the modified Bonferroni's step-down procedure of Holm or (iii) the Holm-Bonferroni procedure applied in reverse with initial P-values of 0.05 or 0.01. Applying the Holm step-down procedure [13], the P-value threshold for significance for each genes is determined on the basis of its rank according to the following formula: P = 0.05/(total number of t-tests or genes in array + 1 -gene rank). Therefore, the gene ranked 1 (i.e. having the lowest P-value) in an expression set for which 695 statistical tests are to be conducted, the threshold P-value is 0.05/695. For the lowest ranked gene (i.e. with the highest P-value) the threshold is 0.05/1. This method was also applied in reverse, utilizing starting P-values of 0.05 and 0.01, according to the following formula: P = (0.05 or 0.01)/gene rank. This approach is hereafter referred to as the reverse Holm procedure. For comparison, the expression data were also filtered using an ad hoc fold-change method. For the foldchange method, the expression ratios were calculated for each gene and a two-fold change was used as the criterion for significance.

Power analysis using mefloquine data set
In general, when using statistical tests (in this case a paired t-test), the required sample size to detect a particular change (e.g. treatment versus control) depends on the magnitude of the difference, variability of the data, the required statistical power and the acceptable type 1 error. The mefloquine data set was used to assess the power and implications of sample size in terms of the minimum detectable average fold-change in expression of the six genes defined as being truly upregulated by the drug. First, a publicly available power/sample size calculator ( [17], available at http://www.mc.vanderbilt.edu/prevmed/ps) was used to determine the power of the paired test (twosided, df = 7). Then, based on this level of power and the same critical values, absolute change in expression was calculated at different sample sizes. The critical values for the calculations were as follows. The detectable difference (δ) was the absolute, average changes in the expression values of the control group. The standard deviation (σ) was the square root of the average variance of differences in expression of the six genes in individual pairs of DMSO and mefloquine-treated cultures. The type 1 error (twosided) was set at α = 0.003, which corresponded to the maximum P-value for which expression changes were confirmed by RT-PCR. Absolute changes in expression at different sample sizes are presented as minimum detectable fold-changes in expression using the following formula: Minimum detectable fold-change = (average expression value of the six upregulated genes in control cultures + size of the detectable difference)/average expression value of the six upregulated genes in control cultures.

Results
NG108 and primary embryonic rat neuronal cells were similarly susceptible to mefloquine. The IC 50 s of the drug against NG108 cells and primary neurons were 12 and 6.6 µM respectively and the overall shapes of the dose response curves were similar ( Figure 1).
The eleven genes with the lowest (array) P-values are presented in Table 2. The upregulation by mefloquine of transcripts encoding the transcription factors cJun and IkappaB (IkB) and the ER stress protein GADD153 were confirmed by RT-PCR (Table 2, Figure 2). All the genes listed in Table 2 had P-values for the microarray expression data substantially less than 0.05. However, expression changes were only confirmed for those genes with array P-values < 0.003. Also, there was a good correlation between P-values for the microarray and RT-PCR expression data (r = 0.97. P < 0.002, Figure 3). Therefore, the seven genes with array P-values < 0.003 were considered to have expression truly altered by mefloquine ( Table 2).
The fold-change and P-value filtering methods were compared in terms of their ability to detect (not detect) genes differentially modulated by mefloquine ( Table 3). The fold-change and P < 0.05 filtering methods detected all seven differentially expressed genes, but also identified many other genes which, given their relatively high unadjusted P-values, are unlikely to be differentially expressed

Figure 2
Semi-quantitative RT-PCR of tubulin, pJunB, EST cJun (AI175959), IkappaB and GADD153 mRNA from DMSO or mefloquinetreated NG108 rat neuronal cell cultures. cDNA was synthesized using oligo-(dT) primers and amplified using gene specific primers. Results are expressed relative to mean optical density for DMSO samples for each gene. (* P value < 0.05, two-sided paired t-test, df = 7).
(i.e. are probably false positives). The Holm-Bonferroni procedure failed to detect any differentially expressed genes. Other conservative filtering methods described in the literature [13] performed similarly poorly (e.g. the Sidak and Hochberg procedures). The reverse Holm procedures detected more significant genes than the Holm-Bonferroni method and generated fewer false positive results than the P < 0.05 method. However, the more conservative reverse Holm method (initial P < 0.01) failed to identify two significant genes.
The effect of sample size on the ease of detection of mefloquine-induced transcriptional changes was then examined. The power of a paired t-test capable of detecting the average change in expression of the six genes upregulated by mefloquine was found to be 50%. The effect of sample size on the minimum fold-change detectable using a test of such power is illustrated in Figure 4. The minimum detectable fold-change increased with decreasing sample size. Reduction of total sample size from 16 to 12 modestly increased the minimum detectable fold-change from 3.0 to 3.8. At a total sample size lower than 10, the minimum detectable fold-change exceeded the maximum change in expression induced by mefloquine.

Discussion
The detection of differentially expressed genes depends critically on having adequate replication (sample size) and on the selection of appropriate filtering methods (in the case of this study a P-value filtering method). In terms
of their ability to detect genes differentially modulated by mefloquine, the different P-value filtering methods can be ranked in terms of their conservativeness as follows: Holm-Bonferroni > reverse Holm (initial P < 0.01) > reverse Holm (initial P < 0.05) > no correction (i.e. P < 0.05). These results are not surprising if one considers the distribution of observed (unadjusted) microarray P values and that expected (uniform if independent tests) under the complete null hypothesis that all genes are unaltered ( Figure 5). Application of the reverse Holm procedures generates relatively few false positive results compared to no P-value correction, but detects genes more efficiently than if the Holm-Bonferroni procedure is applied.
This, of course, begs the question as to whether it is appropriate to now conclude that the reverse Holm procedure is the most appropriate P-value filtering method. The answer, of course, depends on the goal of the proposed study. If, for example, one wished to be certain that a gene is differentially expressed, or does not wish to resort to laborious conventional techniques to confirm expression changes, the only appropriate filtering method is the Bonferroni (or other related) procedure, as the chance of generating false positive results using such methods is negligible. However, if one wishes to identify all differentially expressed genes, and possesses the resources necessary to confirm all expression changes using traditional approaches, the P < 0.05 threshold (or a foldchange method) might be the most appropriate. However for many studies that do not fall at either of these extremes, application of a reverse Holm procedure might be the most appropriate choice.
The choice of an appropriate sample size is also critical if (true) differentially expressed genes are to have a high probability of being detected. A power analysis was conducted to determine whether the transcriptional changes induced by mefloquine might have been detectable using lower sample sizes. There was a modest increase in the minimum detectable fold-change of an average gene when the sample size was reduced to twelve from sixteen. In theory, then, a modest reduction in sample size may  still have allowed many significant genes to be detected. However at sample sizes below ten, the minimum detectable fold-change exceeded the maximum transcriptional modulation of any gene by mefloquine, implying that such changes would be extremely difficult if not impossible to detect. These observations may be directly relevant for planning future studies in which transcriptional changes of similar magnitude, and data-sets with similar variance characteristics, are expected. However, such a power analysis might be considered conservative for studies in which larger magnitude fold-changes are expected.
Appropriate experimental design is also necessary to ensure that any changes in gene expression observed are rel-evant to the problem being investigated. The choice of cell system, toxicant concentration and exposure time must all be carefully considered. In the present study, an immortal (NG108) cell line was selected as these cells are easier to maintain in vitro and large amounts of RNA can be routinely isolated. Cell lines do not always respond in the same manner that untransformed cells would in an in vivo context; therefore their use in experimental model systems may not always be appropriate. However, this does not appear to be the case for mefloquine, since NG108 cells and primary embryonic rat neurons appear similarly susceptible to the drug (Figure 1). The mefloquine concentration (10 µM) was selected on the basis of its physiological relevance [18][19][20] and ability to elicit a

Figure 4
Power analysis of sample size versus minimum detectable fold-change for mefloquine-modulated genes. Calculations were performed with PS software (Dupont and Plummer, 1990), using the following critical values: Power = 0.5, α = 0.003, σ = 1686 (based on an average control expression value of 1315), with total sample size ranging from 4-16. Reduction of sample size from 16 to 12 resulted in a modest increase in the minimum detectable fold-change. Reduction of total sample size to less than 10 results in elevation of the minimum detectable fold-change above the maximum transcriptional modulation induced by mefloquine.
measurable physiological response without inducing maximum cell death (Figure 1). A short exposure time (2 h) was selected for two reasons. Firstly, shorter toxicant exposure time ensured that any changes observed in specific mRNA transcript levels were due to the direct cellular effects of mefloquine, rather than secondary effects caused by changing culture conditions (since drug-treated NG108 cells divide less rapidly than DMSO-treated cells). Secondly, a short in vitro exposure time is appropriate since the adverse neurological effects of mefloquine in vivo occur within 24-48 hours of the first 1-2 doses administered [21,22].
Mefloquine induced changes in the expression of three genes, GADD153, IkB and cJun. CJun is a transcription factor upregulated in response to many forms of neurological injury [23], thus its modulation by mefloquine under conditions of cellular stress is unsurprising and does not imply a specific mechanism of action. However, this is not the case for GADD153 and IkB. Two highly conserved responses are observed under conditions of endoplasmic reticulum (ER) stress; the ER overload and unfolded protein responses [24,25]. The unfolded protein response is characterized by generalized suppression of protein synthesis and the specific induction of ER-resident proteins and GADD153 [24]. The transcription factor nuclear factor kB is activated during the ER overload response, leading to the downstream induction of pro-inflammatory proteins [25]. Therefore, the transcriptional modulation of GADD153 and IkB by mefloquine suggests that the ER might be a target of the drug.
In neurons, GADD153 is selectively upregulated under conditions of endoplasmic reticulum (ER) stress arising from depletion of calcium stores [26]. Here, the upregulation of GADD153 was observed after mefloquine treat-

Figure 5
P-value distributions for mefloquine and theoretical null data sets and threshold trend-lines for the various P-value filtering methods. The section of the null P-value distribution below the intersections with the various P-value threshold lines represents the relative likelihood of generating false positive results. The section of the mefloquine P-value distribution below the intersections with the various P-value threshold lines represents the relative likelihood of detecting differentially expressed genes. The Holm-Bonferroni procedure fails to detect genes significantly altered by mefloquine, but has a low false positive rate. The use of the traditional P < 0.05 filter will detect all differentially expressed genes but will inevitably be associated with a high false positive rate. The two reverse Holm procedures detected most differentially expressed genes and are associated with much lower false positive error rates than the traditional P < 0.05 filter. ment in NG108 cells (10 µM for 2 h). In preliminary experiments utilizing primary rat neurons, we have also observed an upregulation of GADD153 after mefloquine treatment (unpublished data). Mefloquine has been found to alter calcium flux, into and out of, isolated skeletal muscle and brain microsomes, via an inhibitory effect of the compound on the ER calcium pump and calcium release channels (IC 50 of 42-43 uM, [27,28]). Plasma mefloquine concentrations (therapeutic dosing) may reach 21 µM [18] and the drug crosses the bloodbrain barrier, accumulating to concentrations in excess of 50 and 90 µM in the brains of humans and rats respectively [19,20]. Therefore, these biochemical effects occur at concentrations within a relevant physiological range. Collectively, these observations suggest that mefloquine disrupts neuronal function through a combination of disrupted calcium homeostasis and ER stress. This hypothesis is currently under investigation in this laboratory.

Conclusions
Adequate sample sizes and appropriate selection of P-value filtering methods are essential for the efficient and effective detection of differentially expressed genes. Mefloquine induced changes in the expression of genes encoding cJun, IkB and the ER stress response protein GADD153. The upregulation of GADD153 by mefloquine suggests that the drug might affect the function of the ER in neurons, perhaps by disruption of calcium homeostasis.