{'fraction_non_alphanumeric': 0.024074191704249508, 'fraction_numerical': 0.011687734231086832, 'mean_word_length': 5.107059736229635, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Analysis of Dimensional Influence of Convolutional Neural Networks for Histopathological Cancer Classification

Convolutional Neural Networks can be designed with different levels of complexity depending upon the task at hand. This paper analyzes the effect of dimensional changes to the CNN architecture on its performance on the task of Histopathological Cancer Classification. The research starts with a baseline 10-layer CNN model with (3 X 3) convolution filters. Thereafter, the baseline architecture is scaled in multiple dimensions including width, depth, resolution and a combination of all of these. Width scaling involves inculcating greater number of neurons per CNN layer, whereas depth scaling involves deepening the hierarchical layered structure. Resolution scaling is performed by increasing the dimensions of the input image, and compound scaling involves a hybrid combination of width, depth and resolution scaling. The results indicate that histopathological cancer scans are very complex in nature and hence require high resolution images fed to a large hierarchy of Convolution, MaxPooling, Dropout and Batch Normalization layers to extract all the intricacies and perform perfect classification. Since compound scaling the baseline model ensures that all the three dimensions: width, depth and resolution are scaled, the best performance is obtained with compound scaling. This research shows that better performance of CNN models is achieved by compound scaling of the baseline model for the task of Histopathological Cancer Classification.


I. INTRODUCTION
India has almost 32% of its population getting affected by cancer at some point in their lifetime. Cancer detection has always been an issue of major concern for the pathologists and medical practitioners for diagnosis and treatment planning. The manual detection of cancer through microscopic measures like biopsy images through histology is often subjective in nature and in most cases, varies from expert to expert.
Through the advancements of deep learning in the form of Convolutional Neural Networks, we can easily work on classification problems by detecting the core patterns in the dataset. CNNs have consistently been competitive with other techniques for image classification and recognition tasks. Nowadays, CNNs are outperforming their competing methodologies due to the availability of larger data sets, better models, and training algorithms and the availability of GPU (Cloud Computing) to enable the implementation of larger and deeper models. They have demonstrated excellent performance at tasks such as hand-written digit classification and facial detection. Several papers have further elucidated their capacity to deliver outstanding performance on more challenging visual classification tasks.
The objective of this work is to analyze the performance of CNN models subject to their dimensionality for the task of Histopathological Cancer Classification. In order to perform the task of histopathological cancer detection, the neural network must detect and learn the core differences between the two classes, namely, malignant and benign. A CNN architecture can be of different dimensions depending upon its purpose. For example, if an image classification task needs to be performed on images with few classes, the network shall have only a few layers and nodes, whereas on the other hand, if the problem was a multiclass classification problem, the architecture shall have relatively more number of layers and neurons per layer. Thus, this paper attempts to analyze and compare the classification performance of various CNN architectures, namely, baseline, depth scaled, width scaled, resolution scaled and compound scaled architecture for the binary classification task of histopathological scans into Malignant (Cancer Positive Metastatic Tissue) and Benign (Harmless) categories.
The paper is organised into multiple sections, the first section comprising the introduction to the topic. Second section highlights the related works, third section delves into the proposed work. Fourth section discusses the results of our research, and conclusion is presented in the fifth section.
II. RELATED WORK Mingxing Tan et al. [1] studied CNN model scaling and identified that optimal performance can be obtained by carefully balancing width, depth, and resolution scaling. They proposed a new scaling methodology that makes use of compound coefficient. They used this methodology to scale a baseline neural architecture to obtain a family of ConvNets titled as EfficientNets.
Or Sharir et al. [2] and Chenxi Liu et al. [3] studied the expressive powers of Neural Network Architectures, particularly, the effect of architectural similarity or overlapping upon the expressive efficiency between two architectures. They start their research using Convolutional Arithmetic Circuits (ConvACs) and extrapolate the results onto traditional ConvNets as well. They conclude that denser connectivity in the network architecture leads to an exponential increase in the expressive capacity of the neural networks.
Ningning Ma et al. [4] study the effect of speed, memory access cost, platform characteristics, and other direct factors upon the NN performance. They explore the effect of indirect metrics, such as FLOPs, and direct metrics upon the classification performance, and propose a new architecture called ShuffleNet V2 that exhibits exceptional speed and accuracy in image classification tasks.
Karen Simonyan et al. [5] and Dan Ciresan et al. [6] study the effect of depth scaling on the classification performance of ConvNets. Their research was particularly focused upon large scale image recognition using architectures with very small convolution filters (3 X 3), and the performance variations with depth scaling. They demonstrate that depth scaling had a positive impact on the classification accuracy for the ImageNet challenge dataset, and they generalize the results of depth scaling onto other classification tasks and complex recognition problems.
Andrew G. Howard [7] and Mathew D. Zeiler et al. [8] studied the effect of resolution scaling on the classification performance of CNNs. They explore the effect of inculcating image transformation in both the training dataset as well as the test dataset. They conclude that resolution scaling helped in improving the classification accuracy for the ImageNet Large Scale Visual Recognition Challenge.
Ali Sharif Razavian et al. [9] explore the generic descriptors extracted from CNNs, particularly from the OverFeat network to tackle a diverse range of object recognition tasks of image classification, scene recognition, fine-grained recognition, attribute detection, and image retrieval applied to a diverse set of datasets. They determine that, indeed, generic descriptors from complex CNNs are very powerful in object classification tasks.
Alex Krizhevsky et al. [10] analyze and perform the ImageNet classification task of 1.2 million high-resolution images into 1000 classes. They develop a custom 8-layered convolutional neural network consisting of 60 million parameters and 650,000 neurons. The CNN architecture comprises of 2 Convolutional Layers, 3 MaxPooling Layers, and 3 Fully-Connected (Dense) Layers. ReLu activation function is applied to the output of every convolutional and fully-connected layer. A final 1000-way softmax activation function is applied. Dropout regularization is applied to reduce overfitting in the fully-connected layers. The research demonstrates that the complexity of a CNN architecture depends primarily upon the complexity of the task at hand.

III. PROPOSED WORK
This research aims to design, implement, and compare the influence of dimensionality modifications in deep neural networks and suggest the best model for performing the task of histopathological cancer classification. For this purpose, the research involves training 5 different CNNs on the Kaggle dataset. The different networks would be based on the baseline network, a network with scaled width, scaled depth, scaled resolution, and a compound scaled network.
The presented work helps to understand the effect of networks with different dimensions on an image classification problem, in this case, the histopathological cancer classification. The initial analysis of the network design starts with a baseline architecture with few layers and having approximately optimal neural width. The subsequent networks are built on top of the baseline network by increasing the number of neurons per layer (width scaling), the addition of more layers (depth scaling), increasing the dataset image resolution (resolution scaling), and a combination of all these (compound scaling).

A. Dataset Description
The dataset used for this research is an open-source histopathological cancer detection dataset containing 160,000 images of which 144000 are used for training and 16000 for the purpose of validation [12,13]. The dataset consists of two classes viz, benign and malignant. The dataset is presented in the form of train and test folders with images and a separate CSV file containing the corresponding class labels for the images. A small sample of benign and malignant images of the dataset is shown in Fig. 1.

B. Data Preprocessing
The data is segregated into different directories depending upon its class label and the split it belongs to. Since the number of images is too large to train in a single batch for machine with lower RAM capacity, the research involves making use of generators to read images in batches with the help of ImageDataGenerator class of keras.preprocessing [14] as depicted in Fig. 2.

Fig. 2. Preprocessing Flow Diagram
Before this step, the data is normalized to have a uniform data distribution and help attain convergence faster while training.

C. Convolutional Neural Network
CNN is a Data Mining Algorithm that employs deep learning methodology for classification problems. It consists of a layered structure with three types of layers: 1. Convolution Layer: 2. Sub-sampling Layer (Max Pooling Layer)

Full Connection Layer
The complexity of a neural network depends upon the intricacy of the dataset involved as shown in Fig. 3, and can be scaled in multiple ways:  Resolution scaling involves increasing the number of pixels of the images on which the model is trained. This helps retain more information in the input images, thereby, increases the model's ability to extract and learn more details from the dataset. But, increasing the image resolution may be time-consuming, require higher RAM, and increase the time required for training the network.
Compound scaling involves increasing the CNN dimensions using one or more of the above methodologies. This research involves implementing a custom compound scaled model by increasing the width and depth of the CNN and also increasing the resolution of input images.

D. Adam Optimizer
Optimizers are algorithms that help compute the errors upon forward propagations and thus help in adjusting the attributes of a neural network such as its weights and learning rate in order to reduce the losses [15].
µ and are initialized to 0 is the Learning Rate # = 0.9 $ = 0.999 is the Regularization Term This research involves using Adam optimizer for all the networks in order to have a common base for training purposes. Adam Optimizer is a combination of momentum and RMSprop. It acts upon the gradient component using the exponential moving average of gradients (m) and the learning rate component by dividing the learning rate α by √ , the exponential moving average of squared gradients. The initial learning rate is set to α = 0.0001.

E. Loss Function
Training the Neural Networks involves an optimization process that employs a loss function to calculate the model error. A loss function in simple terms is an objective function that needs to be minimized. Loss functions can be broadly classified into two major categories depending upon the type of learning task, namely, Regression and Classification losses [16]. This research involves making use of a classification loss function called binary cross-entropy since the task at hand is a binary classification problem.

IV. RESULTS
The research involves training all the aforementioned CNN architectures with the open-source Histopathological Cancer Detection Dataset. The performance is measured using various performance metrics including Precision, Recall, F1-Score, and AUC of ROC curves. Also, performance graphs including Training and Validation Accuracy and Loss curves, ROC curves, and Confusion Matrix are plotted for each CNN architecture trained for 20 epochs using Adam Optimizer. Fig. 4

. (a) Confusion Matrix (b) ROC curve (c) Training and Validation Accuracy Curve (d) Training and Validation
Loss Curve for Baseline Model The Baseline Model successfully predicted 7609 benign and 7307 malignant out of 8000 and 8000 images respectively, while 391 benign got classified as malignant and 693 malignant got classified as benign. The AUC value for the ROC curve in Fig. 4(b) is 0.979. From Fig. 4(c), it is understood that the baseline model overfits the given dataset as there is a large deviation between training and validation accuracy after 10 epochs. Fig. 4(d) demonstrates that the training loss is decreasing and is about to flatten whereas the validation loss is having chaotic fluctuations. Hence, from all figures, we can conclude that the baseline model overfits the dataset. The baseline model's architecture is depicted in TABLE 1.   Fig. 5(b) is 0.983. The gap between Training and Validation accuracy as well as loss curve is increasing with the number of epochs as shown in Fig. 5(c) and Fig. 5(d). Thus, we conclude that width scaling is resulting in extreme overfitting.  Fig. 6(b) is 0.985. Training accuracy and Validation accuracy is increasing and the gap between the curves is also small. Training loss and validation loss is decreasing. Thus, the Depth Scaling model does not overfit or underfit on the data but is also not considered as the most efficient model as validation accuracy and validation loss is fluctuating. The Confusion Matrix in Fig. 7(a) shows that 7634 benign and 7462 malignant images are classified correctly, whereas 366 benign images are classified as malignant and 538 malignant images are classified as benign. The AUC value for the ROC curve in Fig. 7(b) is 0.983. From Fig. 7(c), it can be observed that the model works well up to the 7th epoch, after which the validation accuracy starts fluctuating and training accuracy keeps increasing resulting in overfitting. The Confusion Matrix for Compound Scaling as shown in Fig. 8(a) demonstrates that 7440 benign and 7415 malignant images are correctly classified, whereas 560 benign images are classified as malignant and 585 malignant images are classified as benign. The AUC value for the ROC curve in Fig. 8(b) is 0.978. Graph from Fig. 8(c) shows that Training and Validation accuracy values are both increasing, and the gap between them is decreasing. Fig. 8(d) shows that both training and validation loss values are decreasing with epochs and are tending to flatten. Hence, from all the above observations, we conclude that Compound Scaling Model does not overfit or underfit on the data and is the most efficient model for the given dataset. The baseline model's architecture is depicted in TABLE 2. It is understood from the above analysis that the performance of a CNN model varies with modifications in its architecture, as also evident in TABLE 3. Keeping the number of epochs constant ( = 20) and Adam optimizer ensures that these variations are primarily due to the architectural/hierarchical modifications.