{'fraction_non_alphanumeric': 0.022448015122873345, 'fraction_numerical': 0.007608695652173913, 'mean_word_length': 5.251403249630724, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Methods of Deep Learning and Big Data Analysis in Promoting Sustainable Architecture

These days, sustainability in different aspects has been among the main discussions of architecture and building science. At the same time, historic architecture has evolved over centuries and has adapted to environmental conditions, it can be a great source of inspiration in using smart ways to achieve sustainable architecture. A good illustration of this adaptation can be found in using vernacular materials, the spatial configuration according to climate conditions, and different elements of historic architecture that have helped to improve the occupant’s comfort. In response, one plausible solution for improving the sustainability of architecture is translating the concept of the sustainable elements and features of historic architecture to be used in contemporary architecture. Therefore, these elements need to be studied thoroughly to comprehend their features and characters. There are several studies, investigating sustainable historic architecture to find and measure sustainable solutions by using conventional methods. Although the accuracy of studying the sustainable historic elements has been fairly high, the number of features and variety of these elements in historic architecture have made this task highly challenging. It has been suggested to study and evaluate a considerable number of these elements in different historic architecture to reduce the errors and increase the reliability of results. Since the conventional methods are labor-intensive, time-consuming, and costly, this paper proposed a robust AI method to study the sustainable elements of historic architecture by using Deep Learning. In this study, by introducing and developing a new method for detecting sustainable elements in historic architecture, their features were comprehensively extracted by means of mining meaningful data from areal images of historic cities to produce big data. The proposed method has a sophisticated workflow starting from subdividing the High-Resolution Aerial Images to detecting the sustainable elements and using data science to analyze the extracted features of the segmented objects. Results of a sample analysis of this method showed its high accuracy and its applicability in analyzing sustainable elements of historic architecture, by which designers are expected to design more sustainable buildings inspired by historic architecture.


Introduction
For centuries, people have adapted and improved their housing to the climate. A central courtyard house is one of the types of housing that proved to be suitable for hot climates. It is characterized by the presence of rooms surrounding a yard, separating the living area from harsh environmental conditions and can be traced back to ancient civilizations in Iran, but also other parts of the Middle East and North  [2]. In nower days modern courtyard houses can be found all around the globe. La Hacienda Jardín in Mexico from Práctica Arquitectura, House in Ecuador by Estudio Felipe Escudero, Hotel San Miguel Allende in Chile by architect Ian Pablo Amores are some examples of contemporary courtyard houses that deal with clime issues in their countries.
In Iran, a study by Soflaei et al. [3], found that courtyard houses also contain additional natural features of about 17%, which can be divided into water (8%) and plant (9%) elements. Studying six houses from Kerman and Isfahan, the authors found fourteen parameters that determine the design pattern of courtyard houses. Further, they provide recommendations for future designs of courtyard houses based on the geometrical proportion logic of the existing courtyard houses. In addition, linear fitting was used to determine the best line fitting of the scatter plot. Using the calculated ratio based on observation and prior research, these houses are more comfortable for the occupants. A passive cooling principle based on geometry and orientation along with embedded natural elements was the subject of their quantitative study. The authors have combined the results into one optimized central courtyard model based on its physical-environmental behavior.
Computing techniques have dominated the domains of structural analysis, design, and urban development in various fields. One of these techniques involves creating digital data by analyzing visually the surrounding and transforming the captured image. This technology's biggest advantage is its ability to automatically classify objects from images. Over the last few years, machine learning (ML) and the development of deep neural networks has pushed this development. For example, AlexNet was developed in 2012 [4]. There are new research fields arising from airborne and satellite data, especially in the field of archaeology, that are helping to provide deeper information and discover patterns that were previously unknown [5], [6]. Even though the technology is available, there is still a void in architectural research. We are confronted with the reality of analog labeling and visual inspection when the idea of automatized data collection and analysis from areal images is considered. As a consequence, big data is a problem for researchers, since they cannot process large volumes of data using classical methods. Traditional techniques cannot keep up with the increasing quality of new information, as academics employ the same strategy for all images, such as making visual choices based on their prior knowledge of the data. As a solution to these issues, imaging operations associated with extracting characteristics from existing designs have to be automated [7]- [9].
In this paper, we introduce methods and a workflow for automatic feature extraction of sustainable architectural elements or typologies. Additionally, a case study that is presented here is used to test the proposed techniques. We describe single steps of how the deep learning method is implemented, so the technique can be applied to other types of buildings and urban structures. Through the use of these AIaided methods, the concept of sustainable elements and characteristics of historical architecture can be translated into contemporary architecture, with the aim of improving the sustainability of future buildings. Through the use of these methods, further investigation of buildings can be undertaken to help with the future development of new courtyard architecture based on vernacular patterns.

2.
Methods In studying ancient cities, automatic identification and segmentation of elements in aerial and satellite photographs using ML techniques has become increasingly popular. There are number of researches and industrial fields that implements neural networks (NN) for image classification, including aerial photographs, gait recognition, medical images microorganisms classification, diverse objects and urban environment recognition [10]. In architectural heritage utilizing techniques such as retrieval [11], pattern detection [12], computer vision algorithms [13], Gabor filters, and support vector machines [14], local feature learning and clustering [15], [16] is getting more common. Nevertheless, automated feature extraction does not eliminate the need for researchers. Its purpose is to serve as an alternative method IOP Publishing doi:10.1088/1755-1315/1078/1/012136 3 for scholars to rapidly assemble a collection of characteristics of vernacular architecture elements and gain methods to improve sustainable architecture and urbanism.
The proposed process is based on our previous research [17] and can be divided into five stages and four processes.

2.2.
Detection Process VHR aerial images differ in many ways from natural photography in detecting objects because of occlusion, shadows, and low resolution. According to the previous studies, there are two main methods for detecting the objects in VHR images. One is the traditional methods that used handcrafted factors [18] and second approach is using deep learning method [4] [19]. We are using annotation tools like Remo_app [20] for the training and evaluation tasks as the natural features in courtyard houses are easily detectable and annotatable. Specifically, we used PyTorch toolkit, which is an open source machine learning python framework that is fundamentally a mathematics toolkit that enables fast computation and automatic differentiation of graph-based models. To expand our training dataset, we also had to apply data augmentation. In the augmentation task, the machine increases the dataset variety by rotating, flipping, shifting, and scaling the images. Furthermore, the training dataset should include variations with distinct characteristics of the observed elements, such as elements from different geographic locations, to enhance recognition capabilities. We describe the training details in [17] where the object detection is done with the COCO-train2017 pre-trained model and further trained on FasterRCNN-ResNet-50 Backbone. This process is explained in [21].

Segmentation Process
The segmentation is necessary as it will extract each found element into its own image. The feature extraction is often a multilevel task. The element will be observed in micro and macro scale. For the urban constellation neighboring images are important as well as for recognition of sub elements inside or near the observed object. The order of the segmentation process depends on these micro and macro dependencies. So one workflow scenario can be a primary search on some very specific identifier, their extraction and widening the frame around the object for identifying other adjunct features (figure 2). As previously mentioned, a reasonable file naming is crucial for a good post processing.

Figure 2.
A demonstration of the data in both datasets after cropping the images [17].
Moreover, the segmentation was carried out using the Mask R-CNN model. The suggested approach recognizes items in an image in a short amount of time while concurrently constructing a segmentation mask. When compared to Faster R-CNN, which operates at a rate of five frames per second, Mask R-CNN is simple to train and offers very little in the way of additional burden. Another benefit of using the identical foundation to estimate human posture is that Mask R-CNN is readily broadly applicable to other applications [22].

2.4.
Feature extraction After a high number of data has been gathered a feature extraction can be performed. This is a process that uses classical image processing algorithms for geometric features detection like size, area, volume, proportion, direction. The segmented polygons of each object should be merged to have one polygon for each object and some tools like the minimum bounding box will find the minimum rectangle which IOP Publishing doi:10.1088/1755-1315/1078/1/012136 5 is fitted to the object. This tool is helpful for rectangular objects like the courtyard to find the accurate direction and dimensions of the object.
Some other features can be further extracted by the previously described segmentation method to recognize and analyze containing elements. In the later described case study, known features were extracted out of single courtyard images as shadows (for height estimation), green and water areas, and the surrounding house. Outlines with geometrical information are further automatically executed and added.
The geographical, climatic, and socio-economic factors that are directly associated with each of these elements are hugely important. For example, in the case study of this research, the climatic information of each city is extracted from the 14 years of weather data and added to the courtyard samples according to their city label. Furthermore, it is possible to extract anonymized socio-economic data of each city or neighborhood from the yearly statistics by governments. For instance, the population, family size, family wealth, jobs of occupants.

Data analysis
The preliminary processing of data would be the first stage in the analytical procedure. For the duration of the preprocessing of the data, it is necessary to locate and remove any outlayers or characteristics that were not properly detected. For the purpose of illustration, elements that have a surface area that is smaller than some predetermined threshold will be eliminated from the dataset when these components are detected. Wrong data can be caused by many things, but in this case, a 10% mistake in object recognition and a 10% mistake in picture segmentation led to wrong data in about 20% of the dataset.
During the whole of the phase devoted to the analysis of the data, an effort was made to identify connections between the characteristics of the dataset. In order to find the strongest link amongst different variables, we used correlation analysis. Here's how to figure out Pearson's correlation coefficient, which is used to measure how closely two variables are linked: The covariance of x and y is denoted by the symbol Sxy, while the standard deviations of x and y are denoted by the symbols Sx and Sy in that order. This formulation transforms the linear correlation coefficient, r, into a value between 1 and -1. In scatter plots, data points cluster around a line when r is close to one, which indicates the two variables are highly correlated. As r increases, the data points become more dispersed. Data points are mainly distributed horizontally when r is close to zero, suggesting that the variables do not have a linear relationship [23]. Finally, a mathematical model for the highly correlated variables in a dataset could be found. The most suitable approach for determining the model is Linear Regression, which involves fitting a line to the scatter diagrams. Several supervised and unsupervised machine learning methods are used to discover relations between features, classify them, cluster them, and build a prediction model. For example, the Sklearn library includes SVM, Naive Bayes, Logistic Regression, Decision Tree, Random Forest, and K-means. When a dataset contains more samples, the prediction model's output will be more accurate.

Case Study
The investigation of prehistoric Iranian inner courts is the subject of numerous of ongoing research projects in Iran. The investigators have conducted a systematic review in which they are contrasting several aspects of inner courts and residences, such as their alignment and proportion [3], [24]- [27]. On the basis of the most recent research [17] a situation is described in which the authors used machine learning algorithms to analyze aerial VHR photography using data gathered from inner courts. This research collected and evaluated a substantial quantity of information on the features of inner courts in order to identify heretofore unrecognized meteorological and geometrical factors that had a role in the planning of inner courts. As part of this investigation, a case study was conducted in which observations were made of nine cities located throughout Iran and categorized given the Koppen climate classification system. Due to the fact that each of these cities has a unique environment, the samples will exhibit a wide range of characteristics depending on the climatic zone in which they were collected. Arid/steppe and arid/desert climatic zones (i.e., BS and BW, respectively) divide these cities. BS and BW climates each feature two subclass climatic zones known as "h" and "k" that fall under the chilly and hot classifications, correspondingly. As a result, there are 4 climate categories: Bsh, Bsk, Bwh, and Bwk.

Central courtyard detection
For the detection it was necessary to subdivide aerial images and prepare them for the training step. In a nutshell, the process starts with exceptionally high-resolution aerial photos and the Fast RCNN approach for locating the inner courts. The system is trained using statistics from all nine districts. The outcome of the object recognition was determined established upon the assessment dataset, and that finding shows a reliability of almost 90% on the assessment dataset. Figure 3 illustrates several examples of the test dataset after they have been subjected to the final trained model. After detecting courtyards in large datasets, the entire dataset was cropped to single courtyard images. As described in the methods chapter, the first step was to identify the courtyards and in the second step the belonging house. For this, an offset of the image was necessary. To end with, the court recognition and crop function are responsible for the production of two distinct datasets: the first dataset contains just the courts, while the second dataset, which includes the home that belongs to the court, is referred to as the offset dataset. As further courtyard segmentation can only be performed on the pure courtyard file, and the offset file contains unnecessary data, both images have to be saved.

Image segmentation
When the model is trained using the two datasets of courtyards and courtyards offset, the model is then assessed using a test dataset that constitutes 20% of the total. The results of the examination show that courts, shadows, and homes attain a reliability of more than 90 percent when it comes to categorization, bounding-box, and masking missions. Water and vegetation, on the other hand, had an 80 percent reliability rate. Figure 4 depicts many instances of assessment datasets that were generated after the segmentation operation was completed.

Feature extraction and Dataset gathering
The last phase was to extract features and gather datasets in producing the desired complete datasets, which consisted of approximately 26,000 samples and 76 features. The minimal boundary box for courts, residences, and water sites was then established for additional measuring [28]. Furthermore, the size and orientation of the minimal bounding box were calculated in order to extract the geometrical characteristics of the item that was picked. The dataset currently contains many meteorological characteristics for each one of city, in place of the geometric elements that were previously included. The meteorological data for every single city was then extracted from the EnergyPlus Weather Format (EPW) files. Environmental Protection Agency (EPW) records comprised 14 years' worth of climate data for every one of the cities (2004-2018).

4.
Conclusion This paper is presenting one possible approach towards discovering patterns for sustainable architecture from already existing architectural and urban systems with deep learning techniques and big data analysis beyond classical methods. We present a workflow for collecting data of a specific typology on a large scale independent of its geographic distribution, extracting their geometric features, and connecting these with local information, such as the geography and economy. Furthermore, a case study of the creation of an exemplary dataset of courtyard houses in Iran illustrates the applicability of this approach. Unexpectedly, the research found that such large-scale pattern recognition in architecture and urbanism are understudied and that no publicly available datasets exist.
The obtained data can help in analyzing specific sustainable typologies, finding unknown relationships, as well as in predicting the best configuration for buildings in a given region and based on specific use requirements. Due to the model's reliance on existing architectural types, urban structures, and other available information, it is essential to develop a diversity of attached features. Further work will concentrate on the systematic development of an expandable dataset framework so researchers can enrich the existing dataset with their own findings. Additionally, a connection can be tested between extraction of geometrical features and more detailed layers of information, such as floor plans. Eventually, such trained models can be used to develop an automatic floor layout recommendation system.
In order to reduce errors and increase the reliability of the results, we propose a further study and evaluation of a large number of elements in different historical architectures that have a high sustainability impact, thus reducing error rates, so that results are more reliable. With the proposed deep learning method combined with traditional image analysis tools, we have developed a sophisticated workflow, subdividing the high-resolution aerial images, detecting the sustainable elements, and then analyzing the extracted features using data science. Furthermore, a possible future research project could be using our dataset and adding exact simulated microclimate information of each courtyard. By this new dataset, it is possible to generate a data-driven prediction model for courtyard design counting a thousand microclimatic data in addition to geometric features. In this way, we hope to make sustainable buildings in harmony with historic vernacular architecture more accessible to architects and designers. The software and datasets are publicly available on GitHub (https://github.com/hadi-yazdi/yazd).