{'fraction_non_alphanumeric': 0.03180821059229082, 'fraction_numerical': 0.009166405515512378, 'mean_word_length': 4.6646449704142015, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Usability and Aesthetics: Better Together for Automated Repair of Web Pages

With the recent explosive growth of mobile devices such as smartphones or tablets, guaranteeing consistent web appearance across all environments has become a significant problem. This happens simply because it is hard to keep track of the web appearance on different sizes and types of devices that render the web pages. Therefore, fixing the inconsistent appearance of web pages can be difficult, and the cost incurred can be huge, e.g., poor user experience and financial loss due to it. Recently, automated web repair techniques have been proposed to automatically resolve inconsistent web page appearance, focusing on improving usability. However, generated patches tend to disrupt the webpage's layout, rendering the repaired webpage aesthetically unpleasing, e.g., distorted images or misalignment of components. In this paper, we propose an automated repair approach for web pages based on meta-heuristic algorithms that can assure both usability and aesthetics. The key novelty that empowers our approach is a novel fitness function that allows us to optimistically evolve buggy web pages to find the best solution that optimizes both usability and aesthetics at the same time. Empirical evaluations show that our approach is able to successfully resolve mobile-friendly problems in 94% of the evaluation subjects, significantly outperforming state-of-the-art baseline techniques in terms of both usability and aesthetics.


I. INTRODUCTION
In our modern world, the World Wide Web (WWW) has become one of the most popular sources of information [2], [5]. This information is often accessible through web pages via numerous types of devices such as smartphones and tablets and via various web browsers such as Chrome, Safari, Firefox, Internet Explorer, and many more. The variety of devices and browsers, on the one hand, allows various ways to access information, but on the other hand, poses several profound challenges, among which guaranteeing consistent web appearance across all environments is one of the most important problems.
In practice, developers often have to ensure that web pages' appearance is amenable to various environments. Specifically, there are two popular types of web appearance issues that † Corresponding author developers usually concern with, including usability and aesthetics of web pages across sizes and types of different devices. Usability issues include bugs related to font size, which affect readability, the margin of touchable elements, the presence of navigation or content that overflows the device's viewport, etc. Aesthetic issues concern relative proportions and positioning of elements on the page. To ensure the absence of these issues, developers often have to test web pages and fix any discovered errors manually. This common practice, however, is very timeconsuming and very expensive due to the enormous amount of environments that need to be considered. For example, these environments may refer to multiple browsers such as Chrome, Firefox, or multiple devices such as smartphones, tablets. Automated techniques that can help developers cope with automatic testing and repairing bugs on web pages are thus of tremendous value. Recent years have seen a pragmatic progress on automated web repair techniques proposed to resolve usability and aesthetic issues on web pages [14], [15]. These approaches generally follow three stages: bugs discovery, localization, and repair. The first stage, i.e., bugs discovery, can be performed via readily available tools such as Google Mobile-Friendly Test Tool (GMFT) [3], Google PageSpeed Insights Tool (PSIT) [4] or Bing [1]. The latter two stages, i.e., localization and repair, are often more challenging with less tool support. Techniques like MFix [14] achieve this often via a genetic programming approach that gradually generates repair candidates that improve usability over time. Although proposed repair algorithms can help to improve web pages' appearance, since they mainly focus on usability while letting aesthetics be best-effort, they may not provide a patching solution that can guarantee good usability and aesthetics simultaneously.
Machine-patched pages may suffer from a poor aesthetic and result in an undesirable layout. For example, Figure 1 depicts a repaired page by state-of-the-art MFix [14] across four different runs. From the figure, we can clearly see the instability and imbalance of MFix. The first two patches have layout distortion as some symbols and text are overlapped. Compared to the original page, the third patch does not change, meaning it does not manipulate the original page to reflect developers' intent. The fourth patch is even more severely problematic as it has cluttered navigation.
In this work, we focus on automated repair of web pages, considering both usability and aesthetics. To assess usability, we use PSIT [4], which rates how friendly a web page is based on a scale from 0 to 100; the higher the score, the more usable the web page is. To assess aesthetics, we follow a common metric proposed in [14], which calculates the alignment between components in a web page. Based on these metrics, we propose a new fitness function that optimizes both the usability and aesthetics scores simultaneously. We propose two repair algorithms to search for a solution that not only addresses mobile friendliness problems but also ensures a pleasing layout. In the first algorithm, we focus on the accuracy of a solution with a high level of usability and aesthetics at a reasonable time cost. In the second algorithm, we compensate the accuracy for the running time. Specifically, we try to reduce the running time as much as possible while maintaining an acceptable level of usability and aesthetics. The first algorithm follows Particle Swarm Optimization (PSO) [9], and the second is based on Tabu Search [8]. We compare our approach against MFix [14] -the state-of-the-art of mobilefriendly web repair. Similar to MFix, our algorithms generate and improve repair candidates over several generations. Different from MFix, at the core of our algorithms, we design a fitness function that optimizes for both usability and aesthetics.
We evaluated our approaches and MFix on 38 web pages, which is provided by Mahajan et al. [14] on two tasks: mobile-friendliness (usability) and aesthetic layout (aesthetics). Experiment results show that our approaches significantly outperform MFix. Notably, our PSO-based repair algorithm outperforms MFix on 29 of 38 web pages in the aesthetic layout task and on 26 of 38 web pages in the mobilefriendliness task. Also, our Tabu search-based algorithm shows superior efficiency despite being less effective than the PSObased algorithm that we proposed. Finally, we also evaluate different parameter settings for PSO and Tabu Search to show their impact. Experiments with the Wilcoxon signedrank test [25] show that the superiority of our approaches over MFix is significant.
In summary, our contributions include: • We propose a new fitness function that optimizes both the usability and aesthetics scores to guarantee generated solutions that not only address mobile friendliness problems but also ensure a pleasing layout. • We proposed two algorithms, namely PBRA and TERA, to search for solutions based on the proposed fitness function. The first algorithm focuses on the accuracy of solutions, while the second algorithm reduces the running time. • We demonstrate that our approach significantly outperforms the current state-of-the-art MFix on aesthetic layout and mobile-friendliness tasks. The remainder of the paper is organized as follows. Section III introduces the overview of an automated web repair process. We present the mathematical formulation of the problem in Section IV and describe our proposed algorithms in Section V. Section VI presents the experimental results, and Section VIII concludes the paper.

II. RELATED WORK
Recently, there has been a growing interest in automated fixing of presentation issues in web pages. Mahajan et al. propose XFix [15] technique that repairs presentation failures arising from the inconsistencies in the rendering of a website across different browsers, i.e., layout Cross Browser Issues (XBIs). XFix assumes that one of the browsers presents the correct presentation of the page, which must be rendered the same in other browsers. In this work, we focus on the usability, and aesthetic of web pages, which is different from the XBIs addressed by XFix. That is, there is no correct reference available to the repair process in our approach. Mahajan et al. propose MFix [14] technique which is a novel automated approach for repairing mobile-friendly problems in web pages. They also provided effective metrics to assess the aesthetic usability of web pages. However, MFix mainly focuses on usability while letting the aesthetic best effort. Mahajan et al. recently proposed IFix ++ [16], a search-based technique to automatically repair Internationalization Presentation Failures in web application. However, this work only fixes layout disruption, while our work focuses on both usability and layout disruption at the same time.
Cassius et al. [20], proposes to use automated reasoning for debugging and repairing buggy CSS. The technique takes as input faulty source lines in CSS files and user-provided examples that can be used to guide the repair synthesis. These inputs are, however, not readily available in the problem domain that our work addresses. PhpRepair [22] and PhpSync [19] detect and repair HTML syntax issues. Wang et al. [24] propose to use static and dynamic analysis together for repairing web applications. This is achieved by propagating a generated presentation fix to the server-side source code.
Less relevant to our work in this paper is a recent research effort in automated repair of source code in Java or C programming languages. Le Goues et al. spark the exciting pioneering work on automated fixing of bugs in C programs [13]. The research in automated bug fixing has since then seen a plethora of proposed techniques for fixing bugs in both C and Java programs, using an array of techniques including search-based software engineering and formal methods, etc [11], [12], [17], [18]. These approaches assume the existence of a test suite to guard against the correctness of the programs under repair. Similar to the search-based program repair, e.g., [12], [13], our work uses evolutionary search to find the best candidates. However, different from them, we do not rely on a test suite for patch validation, but instead, we use readily available tools that can automatically assess usability and aesthetics to evaluate repair candidates.
Overall, despite the recent interest in repairing web pages, there are still more works needed to push the boundary of this research area further. Our work attempts to enhance the existing state-of-the-art work on web page repair. The focus of our work is on repairing web pages while preserving usability and aesthetics altogether. By doing this, our work is quite unique on its own to tackle the aforementioned problem. In the next section, we describe the details of our proposed framework.

III. OUR FRAMEWORK
Our repair framework consists of three phases: segmentation, localization, and repair. The main novelty of our framework lies in the third phase, i.e., repair, while the first two phases are inspired by [14]. Below, we describe each phase in more detail.
The segmentation phase groups web pages into regions. Each region is a set of HTML elements that should be repaired together to guarantee the visual consistency of the repaired page. This can be achieved by readily available tools, such as VIPS [6], Block-o-Matic [23], correlation clustering [7], and cluster-based partition [21]. In this work, we leverage an automated clustering-based partitioning algorithm proposed by Romero et al. [21] as our segmentation tool. This approach is based on the Document Object Model (DOM) tree of the page, in which each leaf element of the DOM tree is assigned to its segment. Then, the segments are merged if they are close enough, i.e., their distance measured by the average depth of leaves in the DOM tree is below a predefined threshold.
The localization phase consists of two steps. The first step is for classifying issues associated with problematic segments that should be repaired. There are five main types of issues, including font-sizing, tap target spacing, content sizing, viewport configuration, and flash usage as reported by popular mobile testing tools [4], [1]. In this paper, we only focus on addressing the first three of these problems following MFix [14]. The result of this step is a set of tuples with the form of s, T , where s is a potentially problematic segment and T is the set of issues associated with s. Note that, T is a subset of {font-sizing, tap target spacing, content sizing}. To achieve this, Google Mobile-Friendly Test Tool (GMFT) can be used.
Next, the second step determines CSS properties that may need to be adjusted for patching the issues of the problematic segments. For example, a font-sizing issue may be patched by adjusting font-size, line-height, width, height properties. As all the elements in the same segment should have a reciprocal relationship, their CSS properties should be adjusted together. To address this problem, MFix [14] proposed a structure named Property Dependence Graph (PDG), which represents the relationship between elements in a segment with respect to a given issue type. Specifically, each node of a PDG graph corresponds to an HTML element in the segment. Two nodes in the PDG are connected if there exists a dependency relationship between them. There is a function M that maps each edge of the PDG to the ratio of between the values of the CSS properties associating with the two end nodes of the edge. Intuitively, if we know the value corresponding to the root node of a PDG, we can deduce the value corresponding to the other nodes by using function M . This results in a set of segments and their corresponding PDGs.
The last phase, i.e., repair, is to determine the optimal values of CSS properties for repairing all the found issues. In this paper, we focus on this phase and propose two algorithms which will be described in the following sections.

IV. FORMULATION AND FITNESS FUNCTION
The core of our algorithms is how we define a fitness function to search for the best solution during the evolution of repair candidates. A well-designed fitness function can lead to better repair candidates, and thus, we carefully crafted the fitness function as explained below. We first give the detailed formula for the fitness function that we designed and then explain the high-level idea following the design of the function.
Given the potentially problematic segments, their issues, and the corresponding PDG identified by the segmentation and localization algorithms described in the previous section, our objective is to determine the optimal patching solution to address all the issues. Let S denote the set of all segments, S = {S 1 , ..., S n }. For each S i (i = 1, n), let I i be the set of issues associated with S i . Let I j i be an issue belonging to I i , then j ∈ {1, 2, 3}, where I 1 i , I 2 i , I 3 i correspond to issues related to font-size, tap target spacing and content sizing, respectively. Let PDG S i , I j i denote the PDG corresponding to S i and I j i . As mentioned in Section III, to fix issue I j i , we just need to determine the value of CSS properties associated with the root node of PDG S i , I j i (then, the other elements can be adjusted by using PDG). Our problem can be mathematically formulated as follows. Input where X j i denotes the value of CSS properties associated with the root node of S i , I j i .

Objectives
Maximize Usability score U (X) Minimize Aesthetic score A(X) where U (X) indicates the mobile friendliness of the page which can be measured by using PSIT; the aesthetic score quantifies the layout differences between the original page and a transformed page to which a candidate patch has been applied. The aesthetic score is measured based on Segment Model (SM) and Intra-Segment Model (ISM), where: • A Segment Model (SM) is defined as a directed complete graph where the nodes are the segments in the segmentation phase and the edges are labeled by one of four relationships: (1) intersection, (2) containment, (3) directional (i.e., above, below, left, and right). • A Intra-segment Model (ISM) is similar to SM but is built for each segment, and the nodes are the HTML elements within the segment.
Using these models, the aesthetic score is computed by summing the size of the symmetric difference between each edge's labels in the SM and ISM of the original page and transformed page [14]. This is a multi-objective optimization problem, and thus we define the total fitness function as follows.
, where α and β are negative and positive weight factors that could be tuned to obtain the best performance; and E(Y ) is a function defined as: We explain the high-level idea of the above fitness function as follows. First, it is worth noting that the maximum mobilefriendliness score given by Google Insight Tool is 100 and that the usability score should be at least 80 to guarantee the page's friendliness as indicated in [14]. Motivated by this observation, we design the first item in the fitness function (i.e., the one representing the usability score) such that its value is significantly large when the usability score is smaller than the threshold, which is 80. Therefore, G(Y ) should be a reciprocal function that increases rapidly compared to a linear function. The selection of G(Y ) is discussed in detail in Section VI Note that as α is negative real number and β is positive real number, maximizing U (X) and minimizing A(X) correspond to minimizing F (X). Accordingly, the objective now is to minimize the fitness function F (X).

V. PROPOSED ALGORITHMS
The core of our method is a Particle Swarm Optimization that repairs web pages by determining the optimal values of the fitness function as proposed in Section IV. PSO iteratively tries to improve candidate solutions, here dubbed particles, by moving these particles around in the search space according to simple mathematical formulae over the particles' position and velocity. Using PSO instead of random searches helps to leverage knowledge about the changing trends of particles, improving and directing particles in the next iterations. PSO often provides high-quality solutions, but it may have a lowconvergence rate which can be time-consuming. To further tackle this issue, we also propose a second approach, which optimizes for running time. This second approach is based on the Tabu search algorithm, which shows a higher convergence rate than PSO, as will be discussed in Section VI.
In the following, we describe the proposed algorithms in detail.

A. PBRA: A PSO Based Repair Algorithm
In this section, we leverage Particle Swarm Optimization approach to determine the optimal values for CSS properties that need to be adjusted. Basically, PSO approach simulates the behaviours of bird flocking. In PSO, every single solution can be seen as a bird, and we call it as a particle. The goodness of every particle is measured by a fitness value based on a fitness function. All of the particles have velocities that direct the flying of the particles by following the current optimum particles. PSO starts with a set of random particles and searches for the optima by updating generations. In each iteration, each particle is updated by following the best values found so far. Algorithm 1, presents the details of our proposed algorithm, which we will describe more in detail below.
Let us assume that we have a set of m particles denoted as P = {P 1 , ...., P T }. The algorithm then follows the steps below.
• Step 1: The first step in the process of a Particle Swarm Optimization is the generation of an initial population.
Algorithm 1 PBRA Input: The set of segments and their corresponding issues and PDGs, S i , I j i , PDG S i , I j i (i = 1, n; j ∈ 1, 2, 3) Output: Repair values for the root nodes of PDGs 1: x ← a candidate is suggested by GMFT [3] 2: Initializing the first generation consisting of m particles P 0 = P 0 1 , P 0 2 , . . . , P 0 T by using Gaussian distribution 3: gBest ← F (P 0 1 ) 4: for P 0 t ∈ P 0 do 5: v 0 (P t ) ← random number in (0, 1) 6: pBest t ← P 0 t 7: if F (pBest t ) ≤ F (gBest) then 8: gBest ← pBest u 9: end if 10: end for 11: k ← 0 12: while The termination condition has not been met do 13: for end fork ← k + 1 23: end while 24: return gBest Each particle of the initial population represents a possible solution to the problem. The initialization process is often designed to ensure the diversity of the population, which plays an important role in population-based algorithms. In mobile-friendly problems, we create the first particle by using the suggested value of GMFT [3]. Then, we generate the initial population by perturbing adjustment factors based on a Gaussian distribution around the original values of the first particle. • Step 2: Let gBest denote the best value obtained so far by all particles in the population, and let pBest t denote the best value that the t-th particle (i.e., P t ) achieved so far. If the current value of the t-th particle is better than pBest t , then pBest t is replaced by the current value of the t-th particle. The algorithm then searches for all particles to find the best candidate with the highest score and assigns it to gBest. • Step 3: The algorithm leverage information obtained from previous generations to improve the quality of particles by adjusting particles direction based on gBest and pBest t , which is determined in step 2. In detail, the velocity and updating the value for every particle are defined as follow. The velocity of particle P t at iteration where v k (P t ) is the velocity of particle P i at the previous iteration. Based on the velocity, particle P t updates its value as follows.
Step 4: Step 2 is repeated until the termination condition is satisfied. The final value of gBest is then deemed as optimal solution. In this work, we used the number of evaluations, i.e., the number of calls to the fitness functions, as our termination condition.

B. TERA: a Time Efficient Repair Algorithm
In this section, we present the detail of a repair algorithm based on Tabu search. The insight of our algorithm is that we start with a repair candidate that is suggested by GMFT [3]. Then, we attempt to iteratively improve repair candidates over generations, following a genetic programming approach [10]. Algorithm 2 shows more details of our proposed algorithm. We explain Algorithm 2 step by step below.

Algorithm 2 TERA
Input: The set of segments and their corresponding issues and PDGs, S i , I j i , PDG < S i , I j i >(i = 1, n; j ∈ 1, 2, 3) Output: Repair values for the root nodes of PDGs x ← a candidate is suggested by GMFT [3] sBest ← Step 1: First, the algorithm creates a repair candidate is suggested by GMFT [3], assigning this candidate as the best candidate denoted by sBest. It then inserts sBest into TabuList which is a list consisting of all the candidates that has been visited.
• Step 2: Let bestCandidate be the best repair candidate found in the previous iteration, and suppose that bestCandidate is represented by where x j i is the value for repairing issue I j of segment x i . The algorithm then determines neighbors of best-Candidate whose values fall into the following range: {x 1 1 ±δ, x 2 1 ±δ, x 3 1 ±δ, ...., x 1 n ±δ, x 2 n ±δ, x 3 n ±δ}. Among the neighbors, it then assigns the one with the highest fitness as bestCandidate of the current iteration. If best-Candidate achieves a better fitness than the current sBest then sBest is replaced by bestCandidate. bestCandidate is then inserted into the TabuList. • Step 3: Step 2 is repeated until the termination condition is matched. In this work, we used the number of evaluations, i.e., the number of calls to the fitness functions, as our termination condition.

C. Implementation Detail
We implemented our approach in Java, consisting of approximately 5000 lines of code. For identifying the mobilefriendly problems in a web page, we used well-established tools such as GMFT [3] and PSIT [4] APIs. For evaluating aesthetics, we identify segments in a web page and build the Segment Model and Intra-Segment Model by building a DOM tree following a similar method in [14] with Chrome browser v60.0 and Selenium WebDriver.

VI. EXPERIMENTAL RESULT
In this section, we empirically compare our approach with the well-known automated webpage repair algorithm MFix [14] on a dataset of real-world webpages. Below, we describe our experimental methodology, including the data set and evaluation metrics, followed by research questions and numerical results.

A. Dataset and Evaluation metrics
To evaluate all algorithms, we perform the experiments on the dataset Alexa used in MFix [14]. The dataset contains 38 real-world subject web pages collected from the top 50 most visited websites across all seventeen categories. Table I presents the dataset's details, such as the URL and category of each web page. The datasets can be publicly accessed in the Github repository of MFix † . In terms of evaluation, we use two evaluation metrics proposed by S.Mahajan et al. [14]: • Usability Score: We use Google PageSpeed Insights Tool [4], which assigns pages a score in the range of 0 to 100, with 80 being an entirely mobile-friendly page, to evaluate the usability of a web page. The higher the score, the better the usability of a web page is. • Aesthetic Score: The aesthetic score is a metric proposed in [14] for measuring how beautiful a web page looks. Based on the assumption that the original web † https://github.com/USC-SQL/mfix/tree/master/ICSE paper data/subjects pages have the aesthetic layout, it is accounted for by the relative visual positioning within the segments of a page, such as Segment Model and Intra-Segment Model, between machine-patched and original web pages. In this way, the lower the aesthetic score, the more the page is aesthetically pleasing. Note that, in this study, we chose to use automated validation, which measures the differences between the repaired page and the original page, instead of a manual human to account for aesthetics. Although we acknowledge the imperfection of automated evaluation, human evaluation is expensive, e.g., the cost to hire professional developers to ensure the quality of the manual evaluation, and still this evaluation method suffers from human biases. Automated validation, on the other hand, is less expensive and does not suffer from human biases. That is a reason why we choose automated validation instead of manual human validation.
For each web page, we run TERA, PBRA, and MFIX, each for ten times, and calculate the average to mitigate the nondeterminism inherent in the approximation algorithms. To ensure fairness between algorithms, we also use the same number of evaluations (number of fitness calls) for all algorithms. We also perform Wilcoxon signed-rank statistical tests [25] to ensure that the performance difference between our proposed solutions and the baseline is statistically significant. Our experiments were conducted on a Macbook Pro machine with an Intel Core i7 2.2 GHz and 16 GB of RAM, running macOS Mojave 10.14.5

B. Research Question
We answer four research questions as described below.
RQ1: How effective is our approach in repairing mobilefriendly problems in web pages as compared to MFix? In this research question, we investigate the effectiveness of our approach, as compared to MFix, to generate repairs in the benchmark dataset that we describe in Section VI-A. Note that our comparison relies on mobile-friendly tasks as well as aesthetic layout tasks.
RQ2: How do different fitness functions impact on PBRA and TERA effectiveness? By default, our approach uses a fitness function with function G(Y ) is quadratic. In this research question, we compare different fitness functions to explain why we selected the quadratic function as default.
RQ3: How efficient is TERA as compared to PBRA? In this question, we investigate the efficiency between TERA and PRBA by performing a comparison on their convergence rate.
RQ4: How do different population sizes impact on PBRA effectiveness? By default, our approach selects an initial population size of 10 for our approach. In this research question, we compare different values of initial population size to explain why 10 is the best option.   To answer RQ1, we compare the effectiveness of our approach including PBRA and TERA (with default settings on Table II and Table III) with a well-known automated repair technique in mobile-friendly problems, MFix [14]. We also performed statistical tests to study the significance of the improvements by our approaches over the baseline. Rationale behind default settings on table II and table III will be explained by RQ3 and RQ4. Experiment results show that PBRA outperforms MFix by generating repairs that have much better usability and aesthetic scores. Note that, as we described in Section VI-A, the higher the usability score, the better the usability of a web page, and the lower the aesthetic score, the more aesthetically pleasing a web page is. From Figure 2, we can clearly find that our approach, namely PBRA, outperforms MFix in 26 out of 38 pages and achieves almost the same performance with MFix in other subject web pages. The results show that PBRA is significantly better than TERA and MFix in the mobilefriendliness task. Figure 2 represents the usability of the repaired web pages by using TERA, PBRA, and MFix. The red line shows a threshold value of 80 -which is the minimum PSIT score to be considered for good usability by GMFT [3]. We can see that most of the repairs pass the PSIT threshold (36 out of 38 subjects). The numerical results concerning aesthetic score are depicted in Figure 3. In Figure 3, we show the relative performance of our approaches versus the baseline technique MFix by dividing the aesthetics score of PBRA-patched and TERApatched pages by that of the MFix-patched pages. We do this to normalize the results because of a wide range of aesthetics' value, e.g., the aesthetics' value can range from anywhere greater than zero. Figure 3 shows that PBRA improves the aesthetic score significantly, outperforming MFix on 29 out of 38 subjects in aesthetics. TERA also shows considerably better results than MFix on 24 out of 38 subjects.
We also perform Wilcoxon signed-rank test [25] to in-vestigate the performance difference between our approaches and MFix. The results point out that PBRA is significantly better than both TERA and MFix in usability scores (with pvalues of 1.4 e-07 and 0.003, effect-size of 0.627 and 0.222, respectively). PBRA and TERA also are better than MFix in aesthetics (with p-values of 0.004 and 0.041, effect-size of 0.991 and 0.435, respectively).
RQ2: Impacts of the fitness function. In Section IV, we introduce a novel fitness function to combine usability score and aesthetic score. In the formula, G(Y ) function plays an important role in making adjustments to speed up the search for friendly patches. However, the G(Y ) function also needs to be designed to avoid ignoring potentially good candidate patches.
We have chosen some functions which satisfy the conditions of G(Y) (in Section IV) to experiment with, such as: We compare the effectiveness of the fitness functions on two tasks: convergence rate, i.e., how fast a fitness function helps the approaches to converge, and overall result. In the overall effect, from Figure 4, we can clearly find that the quadratic function is the most effective, followed by the cubic function and exponential function. In convergence rate tasks, Figure  4 shows that our approach with exponential fitness function converges quickly with values exceeding 80.
Indeed, exponential fitness function can converge well just only within a small number of evaluations of candidate solutions. However, when the number of evaluations is increased, there is no significant improvement. Our experiments show that cubic and quadratic fitness functions need more computation time to achieve better results. One potential reason could be that the exponential fitness function focuses on improving usability score to exceed the threshold so that it ignores potentially good candidate patches. In conclusion, the quadratic function has the best fitness function for both PBRA and TERA. We, therefore, have chosen the quadratic function as the default fitness function for our approach.
RQ3: Convergence rates of PBRA and TERA. The first RQ has demonstrated that PBRA outperforms TERA (36 out of 38 on the mobile-friendliness test and 30 out of 38 on aesthetics). In this RQ, we further investigate the convergence rates of both PBRA and TERA to assess the efficiency of the algorithms.
Our experiments on all pages suggest a similar trend as shown in Figure 5, which is the result for the page http://www.wiley.com. Figure 5 depicts the values of usability score, aesthetic score, and fitness over generations. As shown, TERA converges more quickly than PBRA, but PBRA achieves better values than TERA.
Concerning usability score, TERA converges when the number of evaluations is 50, while PBRA needs 150 evaluations to converge (see Figure 5a). TERA only needs 30 evaluations to reach score 80 -which is the threshold to pass GMFT [3], while PBRA needs 50 evaluations. This means that TERA and PBRA can guarantee a high level of friendliness for repaired pages, but TERA consumes less running time.
The convergence times of TERA and PBRA concerning the aesthetic score are similar, as depicted in Figure 5b. Both of them converge when the number of evaluations is 150. The converged value of PBRA is much smaller than that of TERA. Specifically, PBRA attains the aesthetic score that equals to 86% that of TERA. This result means that PBRA can provide a better aesthetic than TERA.
RQ4: Impacts of Population Size Population size plays an essential role in population-based meta-heuristic methods. A small population size may not be sufficient to find good results, while a large population size may hamper the convergence rate. Therefore, we perform experiments with different population sizes on two tasks: the convergence rate and overall result. Figure 6 shows the impact of the population size on the convergence rate of the algorithm. From the figure, we can clearly find that the population size of 10 is the most suitable value of the approach, while the population size of 5 and population size of 15 shows the worse result. Small populations are more likely to experience the loss of diversity over time because the mating between individuals with similar genetic structures occurred regularly. Therefore, the search space of the algorithm is narrowed, which usually leads to poor results like the result of the population size of 5 in this case. In contrast, large populations could make the algorithm consume more computation time in finding an optimal solution. That is the reason why algorithms with a population size of 15 have poor results. This observation is further reinforced through the experiment on the overall result, which is shown in Figure 7, with five subjects and five different values of population size.

VII. THREATS TO VALIDITY
External Validity. Threats to external validity correspond to the generalizability of our findings. Our evaluation dataset contains 38 real-world web pages. Still, these subjects may not represent all possible cases in reality, and thus our results may not generalize. We tried to mitigate this by selecting top-ranked websites from various categories. We plan to experiment on a larger dataset in the future.
Internal Validity. Threats to internal validity refers to errors in our implementation and experiments. To mitigate this risk, we have carefully examined our code and experiments to avoid potential errors. We have also repeated our experiments several times to ensure that the results reported are correct.
Construct Validity. Threats to construct validity correspond to the suitability of our evaluation metrics. Following MFix, we consider web pages as mobile-friendly if it passes a threshold value of 80, which is the minimum PSIT score for good usability. These ratings are, however, subject to the PSIT evaluation criteria only. Although PSIT is a popular tool to assess usability, we plan to also include other tools to more thoroughly assess usability. In this paper, we introduced automated approaches based on a meta-heuristic algorithm for repairing mobile-friendly problems in web pages. Our approaches strive for improving mobile-friendliness (usability) while minimizing layout disruption (aesthetic) at the same time. To achieve this, we designed a new novel fitness function to allow our search algorithms to find better repairs more efficiently and effectively. Our evaluations on a data set of 38 real-world defective webpages show that our approaches generate good mobilefriendly patches for 94% of the subjects and significantly outperform existing state-of-the-art baseline technique MFix. We have also studied the impact of different fitness functions and parameters of the meta-heuristic algorithm on the overall result and convergence rate. Overall, experimental results are promising and indicate the usefulness of our approaches, which may support developers in designing mobile-friendly web pages.
In the future, we plan to improve the effectiveness and efficiency of our solution further by designing better fitness function to better optimize for both usability and aesthetic. We also plan to curate more data to expand our experiments on a larger dataset containing many more real-world projects. Furthermore, we plan to integrate our approach into development pipeline to interactively help developers develop better web pages.