{'fraction_non_alphanumeric': 0.03122466960352423, 'fraction_numerical': 0.016, 'mean_word_length': 4.655969703009767, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
From Disjoint Sets to Parallel Data to Train Seq2Seq Models for Sentiment Transfer

We present a method for creating parallel data to train Seq2Seq neural networks for sentiment transfer. Most systems for this task, which can be viewed as monolingual machine translation (MT), have relied on unsupervised methods, such as Generative Adversarial Networks (GANs)-inspired approaches, for coping with the lack of parallel corpora. Given that the literature shows that Seq2Seq methods have been consistently outperforming unsupervised methods in MT-related tasks, in this work we exploit the use of semantic similarity computation for converting non-parallel data onto a parallel corpus. That allows us to train a transformer neural network for the sentiment transfer task, and compare its performance against unsupervised approaches. With experiments conducted on two well-known public datasets, i.e. Yelp and Amazon, we demonstrate that the proposed methodology outperforms existing unsupervised methods very consistently in fluency, and presents competitive results in terms of sentiment conversion and content preservation. We believe that this works opens up an opportunity for seq2seq neural networks to be better exploited in problems for which they have not been applied owing to the lack of parallel training data.


Introduction
Sentiment transfer can be considered as a subset of the style transfer task, the main goal of which is to convert a text that presents a style τ 1 to another style τ 2 , while keeping its original meaning µ. Given the increasing number of applications that currently make use of natural language user interfaces, style transfer can be useful in many realworld applications, for instance, chatbot personality transformation for fitting chatbot language to a specific public, bias removal (such as gender and racial bias), offensive and hate speech-language filtering, and thus forth.
Previous studies on style transfer focused mostly on unsupervised methods, for instance Generative Adversarial Networks (GANs), owing to the lack of parallel corpora. However, given that style transfer can be viewed as a monolingual machine translation (MT) task, and that seq2seq models such as the transformer have shown to outperform unsupervised methods in multi-lingual MT when a sufficiently large parallel corpus is available (Lample et al., 2018;Artetxe et al., 2019;Subramanian et al., 2018), in our opinion it is expected that seq2seq would outperform unsupervised approaches if parallel data is available for style transfer. However, to the best of our knowledge, a parallel corpus for style transfer currently does not exist. But considering that semantic similarity metrics are becoming more and more effective (Schwenk and Douze, 2017;Wu et al., 2018), and that considerably large non-parallel data exist for some style transfer tasks, for instance, sentiment transfer and the Yelp and Amazon review datasets, one could take advantage of such metrics to build parallel corpora (Shen et al., 2017;Li et al., 2018).
Given these standpoints, we propose and evaluate an approach to create parallel training data from non-parallel sets of data, on sentiment transfer datasets as a use-case for style transfer 1 , and compare the resulting transferred outputs of a Transformer Seq2Seq neural network (Vaswani et al., 2017) against those of state-of-the-art unsupervised methods. Considering the Yelp and Amazon data sets for sentiment transfer, we take advantage of semantic similarity using Universal Sentence Encoders (USE) (Cer et al., 2018) to represent sentences and the euclidean distance, which is scalable to large sets of data. Our results show that our proposed method can generate more fluently-written texts than unsupervised approaches, and that is well balanced in terms of sentiment conversion and content preservation.
The remainder of this work is organized as follows: Section 2 introduces the related work; Section 3 details the methodology for building parallel corpora, the used seq2seq model, and the evaluation metrics; Section 4 presents the experiments and results. Finally, the paper is concluded with some final remarks.

Related Work
Several methods have been proposed for converting one text to another, which is usually referred to as machine translation (MT). In recent years, great progress has been made with deep learning for multi-lingual MT (Nguyen Le et al., 2017), where a text in a given input language needs to be converted to another text in the desired output language. Much of the progress made in that field is owned to the possibility of mining large corpora of parallel sentences from the web (Uszkoreit et al., 2010;Morishita et al., 2019).
Mono-lingual MT (Ghosh et al., 2017;Shen et al., 2017;dos Santos et al., 2018) also has emerged in recent years, given the potential set of applications, such as the conversion of offensive language to non-offensive (dos Santos et al., 2018) and the generation of customizable affective text (Ghosh et al., 2017). In this case, the input text should be converted to another one in the same language, keeping its main content, but being transformed in some aspects such as language style, tone, or sentiment. Differently from multi-lingual MT, mono-lingual MT generally suffers from the lack of parallel corpora (that is, different versions of the same text rephrased in different tones) to train end-to-end deep learning methods. Efforts to create corpora have been made only on limited domains, such as formality transfer for informal texts (Rao and Tetreault, 2018). As a consequence, both corpora and approaches proposed for the task are generally non-parallel, and unsupervised systems have emerged, mostly making use of text generation models and adversarial samples (Ghosh et al., 2017;Shen et al., 2017;dos Santos et al., 2018;Li et al., 2018;Zhang et al., 2018;Luo et al., 2019).
Recent work in multi-lingual MT has shown that supervised methods tend to achieve better results than unsupervised approaches when the number of parallel sentences is larger than 100,000 (Lample et al., 2018). Considering that the non-parallel data used by unsupervised methods for mono-lingual MT tasks, in special sentiment transfer, large nonparallel set of samples are available, and that textual semantic similarity and representation methods are evolving considerably (Kusner et al., 2015;Wu et al., 2018;Cer et al., 2018;Turc et al., 2019), one could build a sufficiently large corpus of parallel data to train Seq2Seq models for mono-lingual MT.
For this reason, the main contribution of this work is to present an investigation of training Seq2Seq neural networks for sentiment transfer, considering as training data parallel corpora generated from non-parallel disjoint sets, by making use of state-of-the-art semantic representations.

Methodology
In this section, we first describe the proposed method for building a style transfer parallel corpora, followed by the seq2seq Transformer neural network and selected metrics for the evaluation methodology.

Parallel corpora building method
Consider two disjoint sets of textual data X 1 = {x 1 0 , . . . , x 1 N } and X 2 = {x 2 0 , . . . , x 2 M }, with N and M samples, related to two distinct styles τ 1 and τ 2 , respectively. The task of creating a parallel corpus consists of creating a third set, namelȳ X 1,2 = {. . . , (x 1 i , x 2 j ), . . .}, where 1 ≤ i ≤ N , 1 ≤ j ≤ M , and x 1 i has been found to be semantically similar to x 2 j according to a similarity metric Ψ.
In this work, we implement the aforementioned idea in the following manner. We initializeX 1,2 as an empty set. Then, by iterating in the samples of one set, we compute the similarity of each sample against all samples on the other set, adding a new pair inX 1,2 comprising the current sample in the iteration and its corresponding most similar one from the other set. More formally, for each x 1 i ∈ X 1 , we compute the semantic similarity ψ i,j to each x 2 j ∈ X 2 , resulting in the set Since one cannot rely on the assumption that each pair (x 1 i , x 2 j ) are actually parallel samples, a post-filtering is applied onX 1,2 considering two thresholds, i.e. θ min and θ max . While the first aims at reducing the effect of noise that can be presented in the input data, such as samples that are too similar, the second is used to eliminate pairs with not enough similarity between the samples.
To compute the semantic similarity, we take into account Universal Sentence Encoders (USE) sentence embeddings (Cer et al., 2018). Such an approach consist of a Transformer Neural Network (Vaswani et al., 2017), trained on varied sources of data. That approach has been designed not only to serve as a baseline model to take advantage of transfer learning when little data is available, but also as a means to encode textual information, i.e., sentences, into real-valued N -dimensional embedding vector.
Thus, after pre-processing, normalizing, and tokenizing all samples in X 1 and X 2 , we compute the USE embedding vector for each of these samples, resulting in sets As a consequence, to compute the set of similarities Ψ 1 i , we compute the Euclidean distance 2 between the sentence embedding vectors in V 1 and V 2 . Note that, this method can be costly in terms of processing time. Nevertheless, it can be easily scaled up to large sets of data using fast K-nearest neighbor methods.

Seq2Seq Transformer Neural Network
For this work, we use the Transformer Neural Network (Vaswani et al., 2017) as our seq2seq model. The Transformer consists of an Encoder-Decoder architecture, but instead of relying on recurrent neural networks such as in (Luong et al., 2015), it is based on stacked attention layers. That makes the architecture less complex and faster to be trained, and a by-product of that is that it has been consistently outperforming recurrent models in many machine translation tasks (Lakew et al., 2018).
Briefly speaking, the Transformer is based solely on attention mechanisms, not relying on recurrence and convolutions at all. Given the sequential nature of texts, positional features are encoded jointly with word embeddings. By stacking multiple attention layers in both the encoder and the decoder, combined with multi-head attention, the Transformer is able not only to achieve better results but also has a more computationally efficient architecture for training.
For this research, we make use of a publicly available implementation of the Transformer, based on the Pytorch framework for Deep Learning 3 .
We have defined an architecture with the following meta parameters: 6 attention layers, 8 attention heads, word embeddings with 512 dimensions, batch size of 64, and dropout rate 0.1. This network was trained for 50 epochs with the Adam optimizer.
Based on the work described in (Lakew et al., 2018), we make use of an approach to which we refer as shared training. This approach consists of training a seq2seq model for multiple tasks at once, where the task is specified by an special token included in the begining of the input. In this case, since style transfer can be done from style τ 1 to τ 2 and our corpora building method takes that order into account, for converting to the other way around (from τ 2 to τ 1 ), we would need to invert the pairs inX 1,2 to create the setX 2,1 and train a second model. We shared training, we concatenate both setsX 1,2 andX 2,1 , and include in each sample x 1 i ∈X 1,2 , a special token "from1to2". Similarly, for each x 2 i ∈X 2,1 , the token "from2to1" is included.

Evaluation Metrics
We considered the following aspects to evaluate the performance of our style transfer method: 1. Style conversion: if it converts the input text to the desired style; 2. Content preservation: if it preserves nonstylistic parts of the input sentence; 3. Fluency: if the method generates sentences with appropriate language fluency, i.e., grammatically, syntactically, and semantically wellformed sentences.
These aspects are implemented with the following metrics.

Style Transfer Accuracy (STAcc)
The STAcc metric is used to measure style conversion rate. Basically, it consists of computing the ratio of generated samples that have been successfully converted to the target style.
In detail, let X test be the test set and |X test | the number of samples in that set. Also, consider that the number of correctly converted examples is represented by C, where 0 ≥ C ≥ |X test |, this metric can be computed as: The computation above is relatively simple, and accuracy is a well-known metric. Therefore, finding the value for STAcc is trivial once C has been found. However, finding a value for C is the main issue for the metric, since it depends on evaluating the set of generated outputs how many of them were converted successfully. That could be done either by manual inspection or by considering some automated method, such as a text classifier.
For the automated process, we take into account an approach that has been used by Shen et al. (2017) and Li et al. (2018), the TextCNN text classifier (Kim, 2014) 4 . This classifier simply takes as input a text, and provides as output the sentiment label, i.e. either positive or negative. Further details about how we train the classifier are provided in Section 4.

BLEU score
We consider the BLEU score to assess the similarity between ground-truth candidate sentences and the generated sentence (Papineni et al., 2002), which can present indications regarding content preservation.
BLEU provides a score ranging between 0 and 1, which is computed counting matching n-grams in the candidate sentence to n-grams in the generated sentence 5 . Since we are comparing a set of examples, the mean BLEU score of the samples against ground-truth candidates represents the overall score on the test set.
The ground-truth is represented by manually created references, which are provided in the datasets considered in this work.

Perplexity
This measure has been often used to measure the fluency of machine-generated text, i.e. how wellformed are the sentences generated by a given algorithm. In such case, lower perplexity means better fluency.
For this work, we use the language modeling toolkit SRILM (Stolcke, 2002), which computes the perplexity of the generated sentences in the test set, having the language model been computed from the training set, e.g. setX 1,2 .

Experimental Evaluation
In this section we present the experiments that have been conducted to evaluate the proposed methodology. To take advantage of the reproducibility 6 and being able to compare our results with previous works, we consider two sentiment transfer data sets, i.e., the Yelp dataset (Shen et al., 2017) and the Amazon dataset (He and McAuley, 2016), along with the publicly available results made available by (Li et al., 2018). By evaluating our trained method on the same data sets by those authors, we can directly compare our results with theirs.

Data sets
Both data sets consist of positive and negative sentences extracted from restaurant reviews and product reviews posted on Yelp and Amazon, respectively. To generate the sentiment dataset, we considered for both types of reviews that high-star reviews (i.e., rating above three) are positive and those below are negative, and final corpora contain the individual sentences of the respective reviews. It is worth mentioning that we make use of the same data used by previous works, without introducing any extra processing that could affect the results.
The Yelp dataset is slightly smaller than the Amazon one. The former is composed of a training set of 177,218 negative and 266,041 positive samples, and the validation set and the test set contain 4,000 and 1,000 samples, equally distributed in the two sentiment classes. The Amazon dataset is composed of a training set with 277,228 negative samples, and the same number of positive ones. The Amazon validation set contains 1,015 and 985, respectively negative and positive examples, while the test set has 1,000 equally-distributed samples.

Parallel Corpora Creation
We applied the proposed parallel corpora creation method (see Section 3.1) in two different scenarios: 1. All: With θ min set to 0, and θ max set to infinity, the parallel data is found and no postfiltering is applied.
2. Filtered: With θ min set to 0.3, and θ max set to 1.0, pairs that are too similar or not similar enough have been discarded. Peptides signal the dermal system to produce more collagen.
No complaints from her lips to my ears. Considering that we conduct shared training of positive to negative and negative to positive, as we mentioned in Section 3.2, the actual number of samples is doubled (354,436 and 275,232 training pairs for Yelp, and 554,456 and 441,290 for Amazon), which far exceeds the required number of 100,000 samples for training seq2seq models (Lample et al., 2018). NEGATIVE POSITIVE Not even the best fried chicken in Charlotte.
The best fried chicken in Charlotte! The food was ok.
The food was good. Food was ok, the service was horrible.
Service was bad but the the food was good. The macaroni salad is so bad.
The macaroni salad is good and I usually dont like macaroni salad. They start you off with chips and salsa.
They give you chips and salsa to start.  Since it is not feasible to manually inspect the full training sets, we conducted on inspection of a subset of examples of each dataset for a qualitative analysis of the data generated. We observed that the Yelp corpus presented stronger relationship in terms of the main subject and opposite sentiments, such as the first three examples in Table 2. Remarkably, there are examples such as the third one, which presents two main subjects, i.e., food and service, with different sentiments, even though for food the change in sentiment was more subtle. As we can see, some examples may not be too aligned in terms of subjects, such as the fourth one, and samples where the sentiment may not be very clear due to the lack of proper contexts, such as the fifth one. The data on Amazon seems to be more dependent on context than Yelp data, such as the last two examples in Table 3. Even though there are pairs that are quite different in sentiment, such as the first two examples, some pairs present also very subtle sentiment contrast, such as the third one. Such differences, compared with Yelp, might indicate that it may be harder to train the seq2seq method with this dataset. Additional evidence is presented in Figure 1, which contains a cumulative distribution function (CDF) of the similarities of the pairs in the training partitions of each data set. As can be observed, the pairs created for the Amazon data set present slightly lower similarity values, which may impact negatively the training process.

Experimental Evaluation
With the parallel training sets described in the previous sections, we have trained two versions of the seq2seq method as described in Section 3.2: Seq2Seq all and Seq2Seq f iltered , which use as training set the parallel corpora created with the All and Filtered scenarios, respectively.
The goal is to compare the previously-mentioned methods against five unsupervised methods: StyleEmbedding (Fu et al., 2017); CrossAligned (Shen et al., 2017); MultiDecoder (Fu et al., 2017); DeleteAndRetrive (Li et al., 2018); and Re-trieveOnly (Li et al., 2018). The first three unsupervised methods are similar in the sense that an encoder is learned for representing the input sentence, then a decoder is used in different ways to generate the output sentence, with the aid of discriminator classifiers, such as a GAN. The last two consists of using markers that are style-specific, so that these markers can be replaced to transfer from one style to another. While RetriveOnly is somewhat a simpler method, which retrieves an output based on finding the target marker, DeleteAndRetrive makes use of a Recurrent Neural Network (RNN)-based decoder to generate the output sentence.
We present a quantitative evaluation using the metrics described in Section 3.3. For computing the STAcc metric, we have trained a TextCNN classifier on the training partitions of each corresponding data set, by considering the default metaparameters provided by the implementation. This method achieved 96.1% accuracy on Yelp's test set and 79.9% on Amazon's. The classifier achieves higher accuracy on the Yelp data set, which might be another indication that the Amazon data set might be more challenging than Yelp.
Since we are comparing different methods on different metrics, it is not trivial to select a winning approach among all. We are taking into account the average ranking to make such comparison clearer, with the assumption that all metrics have the same weight. In other words, since there are three different evaluation metrics, i.e., STAcc, BLEU, and Perplexity, and seven different methods (the two proposed seq2seq and five from the literature), the methods are ranked from 1 to 7 in each metric, where 1 is the best and 7 is the worst. A final raking is then computed by considering the average ranking position of each approach across the four metrics. In that case, lower values are better.
The main results are presented in Table 4. Considering the Avg. Ranking evaluation, the proposed Seq2Seq all is the top performer on Yelp data, reaching an average ranking of 2.67 on both, and ranks second on Amazon, with an average ranking of 3.33. Overall, we observe that both Seq2Seq all and Seq2Seq f iltered consistently present good fluency, ranking as the top performers in Perplexity for both datasets. And they tend to be balanced in terms of style conversion and content preservation. That might be a good aspect since the proposed method does not cover too much of one aspect with the penalty of hurting the other one. As observed with RetrieveOnly and StyleEmbedding, each method presents the best result in either STAcc or BLEU, but also present the worst result in the other metric.
On Amazon data, as somewhat expected from the analysis of the training sets, the seq2seq methods have not performed as well as on Yelp data. Seq2Seq all was the second-best on Avg. Ranking, presenting the best value for Perplexity but was ranked only third on STAcc and sixth on BLEU. Seq2Seq f iltered performed slightly better than Seq2Seq all in BLEU, being the fourthbest, but was worse in STAcc and Perplexity. It is worth mentioning that CrossAligned was the top performer, reaching the best STAcc values, beating Seq2Seq all by 0.10 points. Nevertheless, Seq2Seq all presented a similar performance with the CrossAligned method in terms of BLEU score (i.e., 0.20 vs. 0.21) and better Perplexity score (i.e., 8.01 vs 17.02).
Surprisingly though, this analysis showed that filtering examples from the corpora might not result in better performance since Seq2Seq f iltered was outperformed by Seq2Seq all . This indicates that the method has coped well with the noise presented in the original data. But surely further investigation should be done.
To complement this analysis, we present some  Had to returned one entree because too cold.

StyleEmbedding
Had to returned one entree because too cold. CrossAligned Had to get our burgers and very nice.

MultiDecoder
Had to take however happy hour, great cold. DeleteAndRetrieve Had to returned one entree because it was well worth it! RetrieveOnly One spicy with lots of mexican oregano and one more mild one. Seq2Seq all I ordered right away and my food was ready in minutes. generated samples from both data sets, to illustrate the performance of Seq2Seq all compared with the other methods. Table 5 shows that on Yelp data, the proposed method can successfully convert the sentiment (positive to negative) and maintain the non-stylistic content terms while for the negative to positive conversion the sentiment was converted with a slight change in content, but which seems to make sense in that context. The other methods, in contrast, seem not to deal well with the inputs.
In the second conversion (negative to positive), DeleteAndRetrieve can change the sentiment and keep the original meaning but generating a quite awkward sentence. The examples generated using Amazon data, shown in Table 6, present similar results. Seq2Seq all is also able to successfully convert the sentence sentiment and preserve the original content. The other methods struggle in the task, either by not properly converting the sentiment or keeping content, or generating some awkward or ungrammatical sentences, such as CrossAligned in the first example and DeleteAndRetrieve in the second one.

Manual Evaluation
In addition to the quantitative analysis, we have also conducted a manual inspection of results presented by the Seq2Seq all and CrossAligned, which performed best on Amazon, and RetrieveOnly, which presented good STAcc values on both Yelp and Amazon. The main goal of the manual evaluation is to understand whether some of the results presented in the previous section with automated metrics are confirmed. This evaluation has been conducted as follows. We asked 4 volunteers to label a random sample of the test sets either from Amazon and Yelp data sets. We asked each volunteer to rank the methods according to three criteria: if the sentiment (polarity) was the opposite from the input sentence, if the sentence had maintained the original meaning of the input sentence, and if the output sentence was Amazon -Positive to negative Input Its actually a little better than the filter that came with the unit.

StyleEmbedding
Also have a little good piece of something with it with the only. CrossAligned There is a little of the game, but the sound is too.

MultiDecoder
Still the toy for a very stick, this the second of this item. DeleteAndRetrieve Its actually worse than a little better than the filter that came with the unit.

RetrieveOnly
Make its head bigger than its body or smaller than its neck. Seq2Seq all The filter is not the best filter I have ever used. Amazon -Negative to positive Input Ridiculous! I had trouble getting it on with zero bubbles.

StyleEmbedding
Would prefer my friend that had to follow the after one. CrossAligned So far I have been using it for years and now.

MultiDecoder
Beautiful I have to replace it with after using the first. DeleteAndRetrieve They are easy to use, I had trouble getting it on with zero bubbles. RetrieveOnly Had a little trouble getting bubbles out, but otherwise was worth the purchase. Seq2Seq all I have had no issues with it getting bubbles. grammatically correct. Figure 2 shows the manual evaluation results. We computed the average of the ranks given by the volunteers. The lower the value the higher ranked the method was classified by the volunteers. Note that for the Yelp data set, on average, for the three criteria the Seq2Seq method was better classified than the other two methods. For the Amazon data set, we also have the same conclusion except for the Meaning criteria, in which the RetrieveOnly method better maintained the original meaning of the sentence on average.

Conclusion
In this paper we proposed and evaluated an approach to create parallel data sets for training seq2seq neural networks for style transfer. We demonstrate that in the sentiment transfer use-case the seq2seq model can be a viable alternative approach to unsupervised methods, achieving the best performance in the Yelp dataset and showing a promising performance on Amazon.
In our opinion, the research presented in this paper shows that the lack of parallel data is not a definitive factor for not using seq2seq methods in text generation tasks. With proper care, a wellperforming model, such as Transformer, can be applied for such cases.
However, we aware that better investigation should be conducted on several fronts. Among them, we can cite better investigation on the parallel set creation method, e.g. considering other similarity metrics. In addition, better evaluation of filtering samples should also be carried out, in special to improve the results with Amazon. Also, to better fine-tune the Seq2Seq neural network is also something that needs to be done, since that be also present a positive impact on the results, but this paper lacks a proper investigation in this specific aspect.