{'fraction_non_alphanumeric': 0.024051309460181722, 'fraction_numerical': 0.008017103153393906, 'mean_word_length': 4.813664596273292, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Chemometrics and a New Web Site

An interesting exchange occurred recently on the International Chemometrics Society's list server. A scientist from a government agency in the United States posted a query about normalising a matrix by row prior to use in principal components analysis (PCA). He had a very large matrix of test data with 50,000 rows and 60 columns, where the rows represented samples and the columns represented variables. After performing PCA on the original matrix, he found that the first factor accounted for 89.7% of the variance in his matrix and additional factors accounted for little of the remainder. The conclusion was that the variables were "homogeneous" and had a high degree of commonality. Anyone of the variables might be sufficient to test a sample. After performing PCA on a column-normalised matrix, he found almost identical results. By column-normalised, he meant that each column was mean-centred and variance-scaled, resulting in columns that had a mean of zero and variance of unity. He then performed the unusual procedure of performing PCA on a row-normalised matrix and found unusual results. The first factor accounted for only 10.5% of the variance in his matrix, and thirteen additional factors were required to account for just 50% of the variance in his matrix. Questions were asked about the validity of row-normalisation and the interpretation of the results. One scientist responded by pointing out the role of mean-centring; explaining that when using original data, the points furthest from the origin of the data space have the greatest leverage in PCA, and when using mean-centred data, the points furthest from the mean have the greatest leverage in PCA. The practical difference between these two approaches is generally negligible. (Author's note: the first factor in PCA of original data often is similar to the