{'fraction_non_alphanumeric': 0.03485568643659158, 'fraction_numerical': 0.011848177998208996, 'mean_word_length': 4.7134986225895315, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Painting Analysis Using Wavelets and Probabilistic Topic Models

In this paper, computer-based techniques for stylistic analysis of paintings are applied to the five panels of the 14th century Peruzzi Altarpiece by Giotto di Bondone. Features are extracted by combining a dual-tree complex wavelet transform with a hidden Markov tree (HMT) model. Hierarchical clustering is used to identify stylistic keywords in image patches, and keyword frequencies are calculated for sub-images that each contains many patches. A generative hierarchical Bayesian model learns stylistic patterns of keywords; these patterns are then used to characterize the styles of the sub-images; this in turn, permits to discriminate between paintings. Results suggest that such unsupervised probabilistic topic models can be useful to distill characteristic elements of style.


INTRODUCTION
In recent years wavelet methods have contributed to art history through their application to forgery detection [1], linking of underdrawing and overpainting [2], and uncovering elements of style [3,4]. The wavelet transform is a powerful tool which captures local features of an image at different resolutions; the characteristics of wavelet coefficients at different scales can be analyzed via hidden Markov tree models. When applied to brushwork, the combination of these feature extraction methods and machine learning algorithms is able to separate high-resolution digital scans of paintings by Vincent van Gogh from paintings by other artists (see Johnson et al. [5]). Given the features extracted from the wavelet coefficients of the scanned images, several different classifiers can be usefully applied (see [2,3] for examples).
In this paper we seek to uncover the intrinsic styles in a dataset provided by the North Carolina Museum of Art, consisting of high-resolution scans of five panels attributed to Giotto di Bondone. In contrast to prior work [2], we lack side information such as underdrawings of different styles. As in [1,2], we extract features from the dataset by combining a dual-tree complex wavelet transform [6] with hidden Markov trees [7]. In the absence of a training set (such as a set of overpainting patches labeled by underdrawing styles in [2]), we cannot apply supervised learning algorithms. Our main contribution is to use probabilistic topic models to model different styles present in each painting, where a style corresponds to a topic in the bag of words model. This produces a meaningful discrimination between the five paintings based on the style distribution in each painting.
This paper is organized as follows. Section 2 reviews some background of wavelet-based HMT models applied to feature representation. Section 3 introduces our post analysis strategy via topic models. Results and discussion are provided in Section 4; conclusions and future work are in Section 5.

Color Representation
We first introduce the HSL color representation, commonly used in computer graphics, and easily computed from RGB; it gives a perceptually more accurate representation of color relationships, superior to RGB. HSL [8] describes colors by their hue (coded by an angle on the color wheel, from 0 to 360), saturation (ranging from 0 to 1) and lightness (again from 0 to 1); this color space is often represented as a doublecone. This double-cone can be expressed in Cartesian coordinates (XYZ), computed from HSL values by setting

Wavelet Transforms
Wavelet transforms allow us to analyze images at different resolutions. By their finite support, wavelets are able to capture information specific to a particular location. Multiresolution analysis then provides information specific to both location and resolution or scale. The clustering and persistence properties of wavelet transforms provide us with statistical models to characterize the dependencies between wavelet coefficients [9]; see also Section 2.3.

Algorithm 1 Feature Extraction
In the dual-tree complex wavelet transform (CWT) [6], two separate discrete wavelet transforms (DWT) decompose signals into real and imaginary parts. The magnitude of each complex wavelet coefficient is nearly shift invariant, rendering CWT insensitive to small image shifts. The dual-tree CWT has six subbands and provides greater orientation selectivity than DWT. It represents local differences in terms of six basic directions, making it possible to capture brushstroke information specific to location, scale and orientation.

Hidden Markov Trees
The clustering and persistence properties of wavelet coefficients suggest the use of an HMT model to capture the intrinsic statistical structure of wavelet coefficients of an image; this provides an efficient dimensionality reduction technique that is robust to noise induced by the scanning process [7].
At each level, the wavelet coefficients are modeled as a mixture of two Gaussian distributions: one, with large variance, corresponds to the wavelet component with high energy; the other, with small variance, corresponds to smooth components. Each HMT model thus has three parameters: • T , state transition probability between successive scales • σ S , variance of the narrow Gaussian distribution • σ L , variance of the wide Gaussian distribution We divide paintings into patches of size 64 × 64; features are extracted independently for each. HMT parameters are estimated by iterative Expectation Maximization (EM) method (see details in [9]). The feature vector of each patch has 6 × (2 × 6 + 2 × 4) = 120 entries: 2 variances for each of 6 scales in each of 6 subbands, and 2 (non-redundant) transition probabilities for each of 4 finest scales. (See Algorithm 1.)

TOPIC MODELING BASED STYLE ANALYSIS
The "signature features" making up the feature vectors can be viewed as primitive building blocks of the style of an artist;

Algorithm 2 Keyword Labeling
Input: All feature vectors Θ = θ i,j

5,ni
i,j=1 . Initialize: Set T = 1, ∀i, j, c 1 i,j = 1. Repeat until T = 11: depending on the style of each artist, these signatures appear with different proportions [3,5]. Usage patterns representing these proportions can be learned from the data. In a departure from previous work, we propose to learn these usage patterns, using Latent Dirichlet Allocation (LDA), a probabilistic model for discrete bag of words data. The choice of stylistic elements in the creation of a painting is here viewed as similar to the choice of words in the process of creating a literary text. Learned topics, or stylistic patterns, are weights over the stylistic words; each painting is represented by weighted combinations of these patterns. This approach is similar to the bag of words model for object recognition in [10] which represents images with different proportions of object recognition feature patterns.
Since there are only five paintings, we divide each panel into sub-images with no overlap (these sub-images are larger than the small patches we used for feature extraction in Section 2). We consider each sub-image as an independent representation of the style of that panel's artist. We begin with definitions and show the mathematical formulation afterwards.
• The basic unit of a sub-image is a single patch ω, characterized by keywords indexed by {1, 2, . . . , T }. (We will explain below how to generate these keywords.) A patch associated to the uth keyword (u = 1, 2, . . . , T ) is represented by a T −dimensional vector with all but the uth entry zero, and ω u = 1. Two patches with the same keyword have the same drawing style. • A sub-image is composed of a sequence of N patches (equivalent to one article in the LDA model in [11]). • An album is a collection of sub-images, corresponding to a "corpus" in [11].
To generate the keywords, proceed as follows: cluster all patches with a divisive hierarchical clustering method, constituting a fixed vocabulary of 2 10 = 1024 keywords in which patch P i,j is assigned a label c 11 i,j ∈ {1, 2, . . . , 1024}. This  Fig. 1. Graphical model for the generative "painting process". Only one node is observed (shaded): the patches in sub-images. Boxes denote repeated processes. The different plates: N is the collection of patches, K is the stylistic patterns, SI is the collection of sub-images.
is a form of vector quantization; it represents each sub-image as a bag of words, see Algorithm 2. The binary structure of the labeling procedure ensures that labels that share dominant digit in their binary expansions are similar. "Stylistic patterns" correspond to distribution over "keywords"; our approach assumes that two sub-images having a similar distribution over keywords are similar in style.
The Dirichlet priors that control each sub-image/pattern distribution and pattern/keyword distribution have parameters α and β. These parameters are sampled once when generating a collection of images. If there is a collection of K stylistic patterns, then the K-dimensional Dirichlet random variable π, describing the pattern proportion for one sub-image, is sampled once per sub-image. The probability density function of π is given by To generate a patch in a sub-image, the "artistic process" first chooses a stylistic pattern t n based on π; this is a Kdimensional unit vector with t j n = 1 if the jth pattern is selected. The pattern-keyword distribution φ is a matrix of size T × K, with column j representing the keyword distribution in pattern j. Then the process produces a patch ω n ∼ p(ω n |t n , φ j ); note that φ l,j = p(ω l n = 1|t j n = 1). Given α and β, the joint distribution of this generative model can be written as A graphical description and dependencies of the generative model is shown in Figure 1. The only observed variable is the patch, the hidden variables are sub-image/pattern distributions, patterns and pattern assignments. One needs to infer the hidden variables based on the observed variables, i.e., our goal is to compute π and φ from ω. In package [12] for variational Bayesian (VB) implementation of LDA.

RESULTS AND DISCUSSION
We divide the scans of the 5 panels into 221 non-overlapping sub-images, each of size 480 × 480 pixels. In the feature extraction step we divide each sub-image into 64 × 64 small patches with 32 pixels overlap, so there are ((480 − 64)/32 + 1) 2 = 196 patches in one sub-image. Each patch is associated with one keyword label; we observe the statistical keyword distribution in each sub-image. The number of stylistic patterns, K, is set to be 20.
We obtain two main parameters from LDA inference: the distribution function over the keywords for each pattern, and  the weights over patterns in each sub-image. The keyword distributions for some patterns are in Figure 2; each pattern is a sparse combination of the keywords in the vocabulary. Some patterns are highly weighted over keywords with labels with the same dominant binary digits (for example, patterns 16 and 18 are both concentrated on keywords indexed from 128 to 255); these patterns are thus similar (see Section 3) .
The pattern distribution for each of the five panels is obtained by adding the pattern weights of all its sub-images and normalizing the weights. Figure 3 shows, e.g., that patterns 6 and 8 are more heavily represented in panel 5 than in the other four. Figure 4(a) visualizes this in terms of the original panels: each sub-image in each panel is shown with a brightness proportional to the sum of the weights of patterns 6 and 8 in that sub-image. Figure 4(b) does the same for patterns 16, 18, 19 and 20, which are clearly more heavily represented in panel 2. Figure 4(c) illustrates the differences between the panels differently. Each sub-image is characterized by a 20dimensional pattern distribution vector π. We use the t-SNE algorithm of [13] to visualize all sub-images by projecting the pattern distribution matrix onto an optimal two-dimensional plane. Most of the blue dots (sub-images of panel 5) are located in one ellipse and a large percentage of the pink dots (panel 2) in another, disjoint ellipse; the sub-images in other three paintings are widely distributed in the plane. Once again panels 2 and 5 (the Virgin Mary and St. Francis) stand out.

CONCLUSIONS
In this paper, we motivated and introduced a successful application of probabilistic topic modeling in painting analysis.
The results are preliminary but promising. This project started when two of us (D. S. and W. B.) asked the other authors whether the techniques of [2] could be used to quan-tify stylistic differences art historians perceive among the five panels of the Peruzzi Altarpiece. It is known that the panels were painted on wood from one single plank, that they all were prepared in the same way, that the underdrawings are similar; a recent study by the Getty Institute also showed similarities between pigments in the red, pink, blue, brown hair, and flesh paints. Yet the same study also found differences making especially the panel depicting St. Francis (panel 5 in our numbering) stand out. It is intriguing that this painting likewise stood out most in our analysis, as being "stylistically" more different. On the other hand, the Getty study found more reason to single out the panel of John the Baptist (panel 4) than the Virgin Mary (panel 2).
Future work includes increasing the adaptivity of our model in the area of painting analysis, restricting it to subsets of the panels (only faces, or hands, or draped fabric, or hair). We intend to check, in particular, to what extent specific elements can be separated by our methods; concentrating on hair, for instance, the thin parallel brush work in the mustache/beard in Christ (panel 3) and John the Baptist (panel 4) seems, to the art historian's eye, possibly from a different hand than other parts of the altarpiece, and it would be interesting to see whether this is mirrored by a more adapted image analysis of the type described in this paper. Detailed images of the same five panels with other imaging modalities, such as X-ray radiography and infrared reflectography, are also available; we would like to fold the results of all these techniques together in a more ambitious, multispectral analysis. Finally, there have been several very interesting articles in the art history literature which attempt to establish similarities and differences in techniques between those paintings thought to be by Giotto and others in the workshop or followers; in this framework, it would be useful to extend our work to this larger body of paintings.