{'fraction_non_alphanumeric': 0.0396410355736329, 'fraction_numerical': 0.007586028794628446, 'mean_word_length': 4.963233878729548, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Few-shot Visual Reasoning with Meta-analogical Contrastive Learning

While humans can solve a visual puzzle that requires logical reasoning by observing only few samples, it would require training over large amount of data for state-of-the-art deep reasoning models to obtain similar performance on the same task. In this work, we propose to solve such a few-shot (or low-shot) visual reasoning problem, by resorting to analogical reasoning, which is a unique human ability to identify structural or relational similarity between two sets. Specifically, given training and test sets that contain the same type of visual reasoning problems, we extract the structural relationships between elements in both domains, and enforce them to be as similar as possible with analogical learning. We repeatedly apply this process with slightly modified queries of the same problem under the assumption that it does not affect the relationship between a training and a test sample. This allows to learn the relational similarity between the two samples in an effective manner even with a single pair of samples. We validate our method on RAVEN dataset, on which it outperforms state-of-the-art method, with larger gains when the training data is scarce. We further meta-learn our analogical contrastive learning model over the same tasks with diverse attributes, and show that it generalizes to the same visual reasoning problem with unseen attributes.


Introduction
The visual reasoning task proposed in recent works [1,2] often involves visual puzzles such as Raven Progressive Matrices (RPM), where the goal is to find an implicit rule among the given images, and predict the correct image piece that will complete the puzzle (see Figure 1b). Since one should identify a common relational similarity among the visual instances with diverse attributes (shape, size, and color), solving such a visual reasoning problem requires reasoning skills which might help to take a step further toward general artificial intelligence.
Recently, researchers in the machine learning community have proposed specific models for visual reasoning using deep learning [1,[3][4][5]. Deep neural networks are powerful and effective predictors for a wide spectrum of tasks such as classification and language modelling, and has yielded impressive performance on them. However, while humans can solve these logical and abstract reasoning problems by observing only few samples, the state-of-the-art deep reasoning models still require large amount of training data to achieve similar performance on the same task (see Figure 1c).
We hypothesize that such sample-efficiency comes from the flexibility of human intelligence, which can generalize well across different problems by identifying the same pattern in the two problem-pairs  Figure 1: Analogy can be explained with (a) simple proportional analogies (e.g., word analogies) and (b) visuospatial analogies (e.g., Raven's progressive matrices [12]). (c) the state-of-the-art deep reasoning network [3] shows dramatically degrading performance if the number of training samples is decreasing.
with analogical reasoning. For instance, given any two pairs of relationships, "A is to B" and "C is to D", one can form an analogy from the morphological parallelism between the two, e.g., as "A is to B as C is to D" or "A : B :: C : D" (Figure 1a). With such an analogical relationship established between the pairs, one will be able to predict any single entity given the rest of the entities. For visual reasoning problems, we could also identify a visuospatial analogy [6] between two sets of visual entities. In general, any pair of sets from which we could identify certain relational or structural similarity between the two sets can suffice as an analogy. Analogical reasoning has been extensively investigated to understand human reasoning process in the field of cognitive science [7] and artificial ingelligence [8,9].
However, the existing deep visual reasoning models do not have ability to perform such analogical reasoning. To overcome this limitation, we propose a deep representation learning method that exploits such analogical relationship between two sets of instances, with the special focus on the visual reasoning problem. This will allow the model to learn representations that capture relational similarities between the entities, in a sample-efficient manner.
Specifically, we train a reasoning network to learn the abstract concepts using several slightly modified/augmented problems manipulated from their original ones. As the visual reasoning problem includes designed (yet implicitly embedded) rules, we replace a randomly selected panel with a noise panel of the instances within each set. We then ensure that the relation between the two differently perturbed set should have the same relation, by enforcing their representation to be as similar as possible with contrastive learning [10,11]. Furthermore, for a cross-domain problem (with different source and target domains) given training and test sets that contain the same type of visual reasoning problems, we extract the structural relationships between elements in both domains and further maximize similarity between two relations via contrastive learning.
The contribution of our work is as follows: • We propose a simple but novel analogical learning method that can capture the relational similarity between two different sets of examples. • We validate our model on the visual reasoning task with RPM dataset, and show that it significantly outperforms existing baselines, with a larger margin under limited training samples or/and for test examples with unseen attributes. • We propose a way to manipulate the structured set of training samples and use it to improve the generalization performance within an analogy point of view.
We also believe that our proposed method has potential to be generalized for other applications.

Analogy and analogical reasoning
Analogies, which are defined on a pair of sets with the same structures or relation among the elements, are widely used as important heuristics to make new discoveries [13], as they help obtain new insights and formulate solutions via analogical reasoning. Analogical reasoning, which reasons about the new problem by drawing analogies between two problems, has been treated as an important topic for AI research.
In general, if there are common characteristics between two objects, an analogical argument can be used as follows [13]: "1) S is similar to T in certain (known) respects. 2) S has some further feature Z. 3) Therefore, T also has the feature Z, or some feature Z* similar to Z. 1) and 2) are premises. 3) is the conclusion of the argument." An analogy is a one-to-one mapping between S and T regarding objects, properties, relations and functions [13]. Not all of the items in S are required to correspond to that of T. Hence, in practice, the analogy only identifies selective shared items.

Visual reasoning on Raven Progressive Matrices (RPM)
The task of RPM can be formally defined using an ordered set (tuple) of context panels arranged spatially in 3×3 matrix with a missing cell at bottom-right corner, where a reasoner is a function to find an answer y from an unordered set of choices C = {c j } 8 j=1 to fill the missing cell then make complete analogy matrix [3].
RPM is a row-wise and column-wise invariant to permutation [12,14,15]. Thus, the commutative property between symmetric components is satisfied if an affine transform based rule is applied [14] : f c • g r :: g r • f c ∀c, r (e.g., f 1 • g 2 :: g 2 • f 1 ) where f c denotes column-wise transformation and g r denotes a row-wise transformation. The matrix can be written formally as, While this example is a simplified form, RPM problem can embed a variety of transformation (rule) and attributes [1,3]. A task or a problem type of RPM can be categorized based on rules and attributes applied.
RPM problem and deep learning. Recent advanced deep learning based reasoning models focus on the measuring relationships between the context and answer panels. The output score is calculated for those pairwise relation embedding. The final answer is chosen based on the answer panel where the highest score is shown. Calculating all possible relations using a number of combinations of context panels to extract relational representation has been conducted in Relation Network [16] and Wild Relation Network (WReN) [1]. Another way is to use a contrast effect by comparing the answer panel centered representation to the general representations which is resulted by aggregating all the context features [17,3].
Most research works have been examined on Raven's Progressive Matrix problem with large number of training samples which is more than 40 thousands (RAVEN dataset [2]) or 1 million (Procedurally Generated Matrices (PGM) [17]) and much more than validation/test set. It is brought into a question about generalization for unseen tasks.
Visual analogy learning. Apart from RPM problem, there are some analogy based researches for a different purpose. Making analogy images by vector subtraction and addition in an embedding space has been investigated [18]. After accurately recognizing a visual relationship between images and then a transformed query image is generated accordingly. Disentangled representation for abstract reasoning has also been investigated in [4]. The authors investigated if a disentangled representation captures the salient factors of variation in samples. For visual recognition, an analogy-preserving visual-semantic embedding model that could both detect analogies has been developed [19]. These works provide good insights to understand embedding learning for a reasoning problem.

Meta-analogical contrastive learning
One of the many reasons for the sample-inefficiency of existing deep reasoning networks may be their inefficient use of information from each single problem. To address this limitation, we note that we can extract multiple relations from a single problem. Basically, we can generate more visual reasoning problems from a given sample, by considering one of the context element as the answer, and the rest of the context elements as the context. We can then enforce this self-supervised problem to have the same relational representation as the original problem, as although we do not explicitly know the actual rule used to generate them, we know that they have the same relation. We expect this analogical learning to discover implicit relations within a problem which helps the model to better adapt to the new problem instances from the same reasoning task.  pairs between support and queries can be generated. This is conducted in a contextual episode module in Figure 3. (b) Different types of analogy learning are derived using different representations such as score, latent variable, and denoised image. Those are used in learning for "inference analogy", "variational analogy", and "generative analogy", respectively.

Analogical queries derived from support
From the above RPM, we assume that there is a missing sample in the context panel matrix apart from an empty answer cell. As this missing value can be estimated by neighbor panels if analogy rules (represented with transformation functions) are estimated (that is missing value immutation). As similar to inferring an answer panel using context panels, we can generate multiple validation cases by dividing the given 8 context panels by 7 sub-context panels and 1 dummy answer panel. We expect this is a simple but effective way to obtain relational embedding.
Multiple subsets derived from the original context panels (X) include a set of support (S) with the original context panels and query with manipulated context panels (Q i ) per episode. Hence, multiple analogical relation can be defined as follows: S : Q 1 :: S : Q 2 :: ... :: where Q i is a query with a structured noise. An example of generating a set of support and query by inserting a noise panel into context panels is explained in Figure 2a. Similar to an imputation process of missing data, implicit rules of a context set can be found by calculating a combination of those analogy relation equations.
Let X S and {X Qi } k i , ∀k denote the support and the query in an image level, and S and {Q i } k i , ∀k denote the support and the query in embedding space which is transformed by encoder (e.g. CNN in Figure 3), respectively hereinafter.

Analogical learning strategy for a pair of contexts
Even though those multiple analogies have similar relational properties, there is no guarantee to have similar embedding for the same output. So we propose to add analogy learning to make the embedding from support and queries similar each other. Analogy between two cases can be measured with a similarity (kernel) function. Instead of the traditional gram matrix based kernel function (e.g. Gaussian kernels), we use a transformation based analogy kernel (A) for effective representation learning via end-to-end training processes.
In this paper, we use network based transformations which include a task inference encoder e(·), a variational encoder v(·), and a denoising decoder d(·) for a non-linear mapping of a kernel function. To define the kernel function, these transformations are combined with entropy based measures such as binary cross entropy (BCE) and Kullback-Leibler divergence (KL) so that it can be directly applied to cast inference as an optimization problem. Formally, three analogy kernels are defined for measuring 1) similarity between task inference scores (via e(·)) measured by BCE, 2) similarity between variational latent variables (via v(·)) measured by KL, and 3) similarity between generative context panels (d(·)) and ground truth context panel including correct choice panel measured by BCE. As depicted in Figure 2b, e(·) and v(·) is defined by MLP and d(·) by transposed CNN. While these functions can be used jointly, we use it individually to reduce complexity of overall learning. We denote individual analogy kernels as follows.
Analogy via task inferences. Analogy measure or similarity of task inference scores can be defined formally as follows: where p(S) and q(Q i ) are modelled using the encoder e(·; θ) for S and Q i respectively; and e(·; θ) denotes an inference network.
Analogy via generative contextual images. Finally, analogy between reconstructed images of the support and the query is defined by measuring their similarity to the original context panels X S as follows, where X S denotes the original context panel images of support S, q(S) and q(Q i ) are modelled using d(·; π), and d(·; π) denotes a decoder network parametrized with π.
These three analogy scores {A Inf , A V ar , A Gen } constitute directly an individual loss function for analogy learning. A loss for analogy learning uses one of those losses, hence, where Contrastive estimation. Since the candidate choice set is an unordered list, a score for each choice panel should be independently calculated rather than associated between them. For determining of the final choice, thus, we use Noise Contrastive Estimation (NCE) [20]. NCE is a general learning framework that estimates parameters by taking noise samples to infer the final answer instead of cross entropy which contains a normailzation factor. This framework has been used in many previous works related to detect contextual answers such as neural language modelling [21,22], image captioning model [11], and visual reasoning [3]. We train our models using noise-contrastive estimation loss as following [3] which has proved a good performance in RPM problem.
where L denotes NCE [3], and f (·; φ) denotes neural networks parameterized by φ, X denotes the original context input, X S and X Q denote support and query panels respectively which are derived from X, C denotes candidate answer set, and an answer label.
Overall network architecture is depicted in Figure 3. Our framework can use any existing neural networks to extract analogy embedding.

Meta-analogical contrastive learning
In the RPM problem, there are two kinds of mapping from a source domain to a target domain. First, in one problem set, 'context panels' are the source domain and 'subset of context panels and candidate answer panel' are the target domain as shown in Section 3.1. Then, between problems, domain mapping is found using a task category estimated based on rules and attributes. Source and target problems are sampled using a domain episode module (see Figure 4). Then analogy learning can be applied according to task similarity between problems. As similar to equation 3.2, we apply analogical contrastive learning between embedding vectors of S and Q from the source problem and target problem, E S ∈ {S, Q} S and E T ∈ {S, Q} T , respectively. where which is the analogy measure for a domain and d(·, ·) ∈ R denotes a soft task similarity between task scores inferred by e(·; θ), (e.g., e(·; θ) − e(·; θ) ). Hence, a final loss for analogy reasoning network is written as follows: L reasoning (X, C, y) = L analogy + L contrastive + L nce .

Experimental setup
We experiment our proposed method on publicly available RPM dataset, RAVEN [2]. For all experiments, we train network models on the training set using ADAM optimizer. For each epoch,  we evaluate the models on the validation set. Using the model shown the best validation accuracy, we evaluate the methods on the test set to report test results. All input images are resized to 80×80 size. We experiment using NVIDIA Tesla V100. For the compared method, we use a publicly available implementation to reproduce results reported in [1,3]. More details on our experimental settings are provided in supplementary material.

Results
Very small (limited) training data. As mentioned in the introduction section, one of important tasks in analogy learning is to find an efficient learning framework which can learn something with a limited amount of samples. The state-of-the-art method for RPM dataset, CoPINet [3], achieved human level performance (86.43 % > 84.41%) using more than 20,000 observed problems. In this experiment, using less than 1.6 % of all training images (42,000 in total) which is much smaller than the number of samples conducted in [3], we examine generalization performance of the methods.
As shown in Table 1, our proposed method (+Analogy, +Inf/+Var/+Gen) achieves a significantly better performance than the state-of-the-art method, CoPINet. '+Analogy' indicates a case of analogy learning without a kernel process (+Inf/+Var/+Gen). As generative analogy learning requires decoder network (more memory), we use smaller minibatch size (2) while other compared methods used (32).
Effect of analogical contrastive learning. In this experiment, we show generalization performance of the proposed method using the full set of training data, by reporting the test accuracy (%) on the RAVEN dataset. As shown in Table 2, our proposed analogical learning strategy yields performance improvement over conventional learning methods. Particularly, with LSTM and CNN as baselines (LSTM+Analogy and CNN+Analogy) we achieve significantly enhanced performance over them with more than 50 % relative improvements. For this experiment, we do not apply NCE to examine the effect of analogical learning only. Although CoPINet has already outperformed human on this task, leaving little room for improvement, our method obtains significantly improved performance over it.  Cross domains for unseen visual experience. Flexible intelligence can be measured by testing an unobserved problem. In this experiment, we compare the proposed method with CoPINet in terms of unobserved attributes cross training and test sets. We use training samples that contain the same type of visual reasoning problems along a training set and validation/test sets but different shapes are included differently from train ("triangle", "square", and "hexagon") to val/test ("pentagon" and "circle"). Table 3, our proposed analogy learning method shows significantly improved performance (more than 50% relative improvement) compared to the baseline method. In this experiment, we use center configuration of RAVEN which is the most simple and distinctive configuration. When the smaller minibatch size (2) is used, it showed a better generalization performance. This might be because each problem is dissimilar each other. We used large number epochs (400) so that training loss converges. We also provide an ablation study about meta-analogical contrastive learning using pairwise samples on cross-domains (across different configurations) in the supplementary material.

Conclusion
In this paper, we proposed a framework to learn the representations for abstract relations among the elements in a given context, by solving analogical reasoning problems. We propose to exploit the analogy, the relational similarities between two given reasoning problems. Specifically, we focused on the analogy between the set representing the original problem instance and its perturbation, as well as to another problem instance from the same task class, and then enforced the representations of the pair of instances to be as similar as possible. We further propose a generative analogy which generates a sample that preserves the analogical relationship between two problem instances in the given pair. Moreover, to allow the model to solve for unseen tasks, we further meta-learn our contrastive learning framework over pairs of samples with different attributes. We first validated our method on both visual reasoning on the full RAVEN dataset against relevant baselines including the state-of-the-art method (CoPINet), which it largely outperforms. We further validated our analogical contrastive learning framework on a challenging few-shot visual reasoning problem, on which it yields even larger improvements over CoPINet. The results of few-shot visual reasoning experiment on unseen tasks is even more impressive, as our method outperforms CoPINet by 15% to 26.25%. While we mostly focused on visual reasoning task, our proposed analogical learning strategy may be useful for other application areas where similar relational structures could be found.

Statement of Broader Impact
We investigated a low-shot visual reasoning problem requires reasoning skills which might help to take a step further toward general artificial intelligence in this paper. We introduce a new visual analogical learning technique. By defining multiple analogy problems with analogical contrastive learning, our method shows significantly improved generalization performance using a limited number of training samples. Furthermore, the method shows a good generalization over cross domains with different attributes. Thus, our contribution can be applied in a wide range of application in machine learning, from natural language processing to computer vision, under reasoning concepts.
Reasoning may have some ethical issues, because it is a research topic more about general intelligence which is related to human thinking. In future, for example, one can use reasoning frameworks to show the superiority of a group of people over other people. However, we do not think our work has ethical implications, because we experimented on simple and generated visual reasoning dataset (Raven's Progressive Matrices used for IQ-test) which does not include any parts to overwhelm or replace human's ability on general intelligence in the real-world. This, thus, cannot be used to prove the correctness of the reasoning problem.

A Experimental results
In this section, we examine our proposed meta-analogical contrastive learning using pairwise input samples from cross-domains. Firstly, we show an effect of contrastive learning based on relation between problems in addition to analogical learning. Then, we examine if our analogical learning is effective on analogical sampling based learning framework such as existing few-shot meta learning framework. Additionally, we show generalization performance of our proposed method across different visual domains.
As mentioned in the experimental section in main text, we use training samples that contain a similar type of visual reasoning problems along a training set and validation/test sets, while different shapes are included from train to val/test.
Meta contrastive learning of pairwise inputs over cross domains. For contrastive learning on a domain level (between problems), we use estimated task scores to calculate their analogical relations. Instead of hard binary categorization ({0, 1}) indicating positive or negative pairs in general contrastive metric learning [10], we use a soft (task) similarity metric between problems. The soft task similarity between task scores Eq. (7) in main text, d(e(E S ), e(E T )) ∈ R, can be defined in more detail as follows, where D := || e(E S ) ||e(E S )|| − e(E T ) ||e(E T )|| || denotes a normalized distance between score vectors, e(·) denotes the score vector of an inference encoder, E S is a sample from a source domain, E T from a target domain, CReLU denotes the Rectified-linear-unit (ReLU) with clamping defined as min(max(0, x), 1) ∈ [0, 1], and {p, n} ∈ [0, 0.5] is a constraint term for positive and negative relation which aims to make samples having small distance (< p) closer and samples having large distance (> n) farther, respectively. Table 4, our proposed method with meta-analogical and contrastive learning (MetaContrast) using pairwise problems (two inputs) shows improved performance compared to that of using a pointwise problem (single input). Table 4: Test accuracy (%) at the best validation accuracy is shown, where different visual shape attributes are used among train and valid/test sets. The improved accuracy by MetaContrast (second line at each row) is indicated using an underline compared to their counterparts (first line at each row, also shown in Table 3  Analogy sampling based meta learning. In this section, we experimented analogy sampling based reasoning using the given RPM problem definition (called problem category or task, as mentioned above) from RPM dataset generation. The meta information from RPM dataset includes discrete codes composed of attributes and rules. With the given problem category (task), a subset for learning can be sampled (via domain episode module in Figure 4 in main text). In each subset, a training sample and a test sample can have analogy relation dependent on their problem categories. In this experiment, we apply conventional meta (few-shot) learning (e.g. Model-Agnostic Meta-Learning (MAML)) [26], which consists of K-shot and N -class training samples for classification task. Here, by replacing class with task, K-shot and N -task reasoning framework can be defined. With this setting, we examined our proposed method using analogical sampling which contains different attributes (shape) but similar rule for meta-training (source) and meta-test (target) sets. This analogy sampling resembles hard task label based contrastive learning.

As shown in
Here, we show analogical learning with the existing meta learning framework for fast adaptation from the source domain to the target domain. As similar to above experiments, CoPINet is used to baseline model in MAML. We use a public PyTorch based MAML code 1 . We experimented this method on the same dataset used in the experiment above while meta-test using training and test samples is different from test settings in the above. Experiments are conducted with settings: 1-shot training and 1-shot test, number of epochs = 200, batchsize = 16, and number of batches = 100 without hyperparameter optimization.
As shown in Table 5, our proposed analogy learning with analogy sampling (+Analogy Learning in the table) shows significant performance improvement in terms of test accuracy at the best validation accuracy compared to analogy sampling based meta learning (Analogy Sampling in the table). Hence, this result brought a new direction of analogy sampling based analogy learning via the meta-learning framework.
Generalization across different configurations. We experimented our proposed method across different configurations on RAVEN dataset including: 2×2Grid ("distribute four", corresponding folder name in the dataset), 3×3Grid ("distribute nine"), O-IC ("in center single out center single"), O-IG ("in distribute four out center single"), L-R ("left center single right center single"), and U-D ("up center single down center single"). At each configuration, while shapes can be shared, overall visual arrangements are different each other. As similar to above experiment, we use training samples with three shapes ("triangle", "square", and "hexagon") and val/test sample with two different shapes ("pentagon" and "circle") to training for a generalization purpose.
To show generalization reasoning performance of source domains on target domains, our model is trained using samples from "center" configuration (training) and then tested samples from other six configurations. As shown in Table 6, our proposed method ("+Analogy") shows significantly improved generalization performance on cross domain problems.

B Experimental settings
Implementation. For reasoning experiments, we used PyTorch code based implementation to reproduce the baseline methods (LSTM [23], CNN [24], and vanilla ResNet [25]) and strong baselines (WReN [1], ResNet+DRT [2], and CoPINet) based on publicly available PyTorch code 2 . For dataset generation, we used publicly available RAVEN PyTorch code 3 in the experiment "Cross domains for unseen visual experience".
Hyper-parameter settings. All models were trained using the Adam optimiser, with exponential decay rate parameters β 1 = 0.9, β 2 = 0.999, and = 1e−8. We fixed the random seed ('12345', following open codes of the baseline method), and learning rate 1e−4. An input size of image panels are resized from 80 × 80 to 160 × 160. Images are normalized into [−1, 1]. Any other transformation or data augmentation techniques are not used.
Model details. Here, we provide details for all our models. where • MLP consists of an inner-product layer, MLP 1 ∈ R (d×(a·r)) , MLP 2 ∈ R (r×64) , • d denotes a dimensionality of a (feature) vector before a classifier, • a denotes the predefined number of attributes, r number of rules, (a = 10, r = 6) • reshape makes the vector size from (batchsize × (a · r)) to (batchsize · a × r), • and sum make summation along a to make a feature vector as (batchsize × r).
2) Variational context latent. We provide an architecture of the variational encoder in detail as follow, v(·) := [MLP µ , MLP σ ] where MLP consists of an inner-product layer, MLP µ ∈ R (d×p) for the mean latent vector, MLP σ ∈ R (d×p) for the standard-deviation latent vector, d denotes a feature dimension before a classifier, and p = 256.