{'fraction_non_alphanumeric': 0.022082018927444796, 'fraction_numerical': 0.01498422712933754, 'mean_word_length': 5.609375, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A modification of canonical variates analysis to handle highly collinear multivariate data

A modification of the standard Canonical Variates Analysis (CVA) method to cope with collinear high‐dimensional data is developed. The method utilizes Partial Least Squares regression as an engine for solving an eigenvector problem involving singular covariance matrices. Three data sets are analyzed to demonstrate the properties of the method: a two‐group problem with near infrared spectroscopic data consisting of 60 samples and 376 variables, a multi‐group problem with fluorescence spectroscopic data (1023 variables) consisting of 83 samples from six groups and a three‐group problem with physical‐chemical data (10 variables) consisting of 41 samples from three groups. It is demonstrated that the modified CVA method forces the discriminative information into the first canonical variates as expected. The weight vectors found in the modified CVA method possess the same properties as weight vectors of the standard CVA method. By combination of the suggested method with, for example, Linear Discriminant Analysis (LDA) as a classifier, an operational tool for classification and discrimination of collinear data is obtained. Copyright © 2007 John Wiley & Sons, Ltd.