{'fraction_non_alphanumeric': 0.032227930873423634, 'fraction_numerical': 0.012167211583372256, 'mean_word_length': 4.73162896533262, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
MAgNET: A Graph U-Net Architecture for Mesh-Based Simulations

Mesh-based approaches are fundamental to solving physics-based simulations, however, they require significant computational efforts, especially for highly non-linear problems. Deep learning techniques accelerate physics-based simulations, however, they fail to perform efficiently as the size and complexity of the problem increases. Hence in this work, we propose MAgNET: Multi-channel Aggregation Network, a novel geometric deep learning framework for performing supervised learning on mesh-based graph data. MAgNET is based on the proposed MAg (Multichannel Aggregation) operation which generalises the concept of multi-channel local operations in convolutional neural networks to arbitrary non-grid inputs. MAg can efficiently perform non-linear regression mapping for graph-structured data. MAg layers are interleaved with the proposed novel graph pooling operations to constitute a graph U-Net architecture that is robust, handles arbitrary complex meshes and scales efficiently with the size of the problem. Although not limited to the type of discretisation, we showcase the predictive capabilities of MAgNET for several non-linear finite element simulations.


Introduction
Computational models are playing an important role in the development and advancement of diverse engineering systems. These models are based on governing physical laws, which are expressed with partial differential equations (PDEs). PDEs associated with physics-based simulations are most commonly solved with mesh-based numerical methods. However, these numerical simulations remain prohibitively expensive for most practical situations, despite the huge growth in processing capacity over the last decade. Speeding up such models whilst maintaining the desired accuracy is an active area of research, and one of the main motivations of this work.
Recently data-driven methods have taken a center stage across many disciplines of engineering & science. In particular, deep learning (DL) methods have proven to accurately identify a nonlinear map of input and output data pairs. They leverage the capabilities of DL techniques to efficiently solve mesh-based problems by performing supervised learning on either experimental or numerically generated data. In the context of physics-based simulations, deep-learning surrogate models have been successfully used as a replacement for computationally expensive numerical methods. Some of these approaches constrain the neural network optimisation objective function with physics-based information such as boundary conditions, energy, and residuals and are often termed as Physics Informed Neural Networks [Raissi et al., 2019] [Samaniego et al., 2020] [Henkes et al., 2022] [Klein et al., 2022] [Zhang et al., 2022]. While other approaches learn the underlying physics implicitly by learning on relatively more amount of data [Runge et al., 2017] [Aydin et al., 2019] [Daniel et al., 2020] [Kochkov et al., 2021], the strategy that has been followed in this work.
Although DL techniques have shown great success as surrogate models to replace computationally expensive numerical methods, they still suffer from high training costs. This is because a majority of the existing scientific machine learning approaches are based on fullyconnected networks, which often have far too many network parameters than warranted by the size of the problem, thus over-complicating (overfitting) the non-convex optimization task of neural network training. In this regards Convolutional Neural Networks (CNNs) have achieved a promising performance in a wide variety of applications, including accelerating non-linear finite element simulations [Obiols-Sales et al., 2020] [Rao and Liu, 2020] [Krokos et al., 2022b] [Deshpande et al., 2022]. CNNs are designed to learn a set of fixed-sized trainable localized filters, thus reducing the parameter space. They leverage the fact that nearby nodes of the FEM mesh show strong local correlation, and provide computationally efficient topology that can capture non-linearities. However, they work only on Euclidean or gridlike structure data, thus prohibiting performance over many real-world applications, where data comes in an unstructured format. In the context of FEM, some efforts to accommodate unstructured meshes for CNN based approach include combining finite elements with an immersed-boundary method [Brunet et al., 2019], embedding a precomputed coordinate mapping into the classic grid [Gao et al., 2021]. These methods have associated preprocessing costs, and importantly their effectiveness is limited to simple irregular domains and remains challenging for complex geometries in general.
Recently, Geometric Deep Learning (GDL) has emerged as a type of deep learning method which focuses on the development of neural networks that can learn from non-Euclidean input such as graphs, manifolds [Bronstein et al., 2017] [Wu et al., 2021]. An important factor in conventional machine learning techniques is that input instances are considered independent of each other, which is not valid when the data is represented as graphs. Because each node is related to others by links of various types. And therefore machine learning methods can be enriched by encoding the topological information in the data to model the relationships between entities and provide more potential insights underneath the data by expressing it as graphs. At present, GDL, and the underlying Graph Neural Networks (GNNs) are gaining increasing importance in solving scientific machine learning problems because of their ability to handle unstructured data [Sanchez-Gonzalez et al., 2020] [Vlassis et al., 2020] [Pfaff et al., 2021] [Krokos et al., 2022a] [Gao et al., 2022]. However most of these methods are based on the less capable message passing paradigm, thus models are insufficient for nonlinear applications demanding high engineering accuracy, such as surgical simulations [Bui et al., 2018] . To overcome these limitations, in this work we propose a novel local aggregation technique we denote as Multichannel Aggregation layer (MAg), a powerful message-passing scheme. MAg has been inspired by the traditional convolution layer in terms of incorporating multiple channels and can efficiently perform non-linear regression mapping for graph-structured data.
Additionally, most of the above-mentioned GDL approaches pose a significant computational challenge with the increase in the graph size. To alleviate this issue graph coarsening approaches were proposed [Bianchi et al., 2019] [Lee et al., 2019] [Cai et al., 2021], similar to pooling layers in CNNs, which help to reduce the size of a graph while maintaining essential properties. In the context of GNN accelerated FEM simulations, recently [Black and Najafi, 2022] proposed a multi-fidelity GNN for learning the finite element convergence behavior. However, their implementation is only limited to regular meshes for simple two-dimensional geometries and linear elastic problems. In this work we propose a novel graph pooling operation, which produces coarsened mesh representations for any arbitrary complex 2D or 3D mesh. This enables a computationally efficient learning for problems involving complex fine meshes.
Our work particularly focuses on encoder-decoder architecture frameworks like U-Net. U-Net based approaches have been successfully implemented for various applications in diverse fields ranging from computer vision [Ronneberger et al., 2015] [Ç içek et al., 2016], signal processing [Hennequin et al., 2020] [Ren et al., 2021], to scientific machine learning   [Pant et al., 2021] [Le et al., 2022]. While the CNN-based U-Net approaches are limited to the grid (structured) data, graph-based U-Net approaches are being proposed as an alternative for the non-Euclidean data such graphs [Gao and Ji, 2019].
To sum up, we offer a novel graph U-Net framework comprised of the proposed MAg and graph pooling layers. MAg captures local regularities in the input data, whereas the interleaved pooling layers reduce the graph representation to a smaller size while ideally preserving important structural information. And thus enabling us to efficiently implement our framework for large-scale problems. The proposed MAg and graph pooling layers are compatible with state-of-the-art neural network layers. Also, MAgNET is compatible with any type of discretisation of a PDE and can be used with a range of numerical methods. Although, we validate the strength of our framework by implementing it to several nonlinear finite element examples. In particular, we show predictive capabilities of MAgNET in finding force-displacement mappings for large deformations of soft bodies, which is an extension of our previous work using CNN U-Net [Deshpande et al., 2022]. Source code and procedure to generate numerical datasets utilised in this work are available at MAgNET.
The paper is organized as follows. In Section 2 we present the novel MAgNET framework followed by the formulation of the hyperelastic finite element problem. Then, in Section 3, an extensive study of the proposed framework is performed with implementations to several 2D and 3D benchmark non-linear FEM examples. The conclusions and future research directions are summarised in Section 4.

MAgNET Deep Learning Framework
In this section, we will propose a novel graph-based encoder-decoder (U-Net) deep-learning framework. We will provide a general formulation, in which inputs and outputs follow a certain graph topology (that is expressed by an adjacency matrix A). The graph expresses an assumed structure of correlations within input/output data and allows us to devise a robust DNN architecture defining a non-linear mapping between inputs and outputs. We will apply this general framework to mesh-based graphs. Such mesh topology of data is characteristic to spatially discretised numerical solution schemes for PDEs in scientific computing. In particular, we will focus on hyperelastic problems in solid mechanics, for which the training/testing data is obtained through the finite element method (see also the schematics in Figure 1). In Section 2.1 we will provide an overview of the proposed Graph U-Net framework MAg-NET. Next, in Sections 2.2-2.4 we will introduce the building blocks of MAgNET. In particular, in Section 2.2, we will introduce the adjacency matrix representation of the mesh-based graph, which will be utilised later in this paper; and in Sections 2.3-2.4 we will specify a new graph Multi-channel Aggregation (MAg) layer, as well as new graph pooling/unpooling layers. Finally, in Section 2.5 we will introduce a particular application of the framework to mesh-based datasets that are generated from FEM solutions of problems in hyper-elasticity.

MAgNET Architecture Overview
The MAgNET graph neural network architecture can be classified as a graph U-Net network and is an extension to the well-known class of convolution-based U-Net architectures (see [Ronneberger et al., 2015]). As such, the graph U-Net comprises of aggregation ('convolution'), pooling, unpooling, and concatenation layers (see the schematics in Figure 2), which are here suitably adjusted to work with general (non-grid) topologies of inputs/outputs.
The graph U-Net architecture has two stages: encoding and decoding. In the encoding stage, first, we apply one or more aggregation (MAg) layers, which are analogues of convolution layers in non-graph U-Net networks. Next, we apply a single graph pooling layer, which is a particular contraction of the graph, and which downsamples (coarsens) the problem. This aggregation-pooling sequence is repeated several times to achieve the desired level of contraction (coarsening). At the coarsest level, the MAg aggregation is performed one or more times, after which the decoding stage begins, which is the opposite to the encoding stage. At each level of decoding, the graph unpooling layer is followed by one or more MAg layers. At the upmost level, the last MAg layer is applied with linear activation to get the output. More formally, the Graph U-Net network, G, is constructed as follows. First, we set the input layer d 0 as a vector of N nodes, each of which being a vector of input values (also known as features or channels) of a constant length c 0 . (Further on, we will refer to the features as the channels.) Next, we subsequently add layers, d l , to form a U-Net architecture. The subsequent layers, d l−1 and d l , are linked by the following relationship where θ l is a vector of trainable parameters (e.g., weights and biases, θ l = k l ∪b l ), and T l (·) is one of three already introduced transformations: MAg(), gPool() or gUnpool(), which will be more precisely defined in the following sections. Additionally, we also consider remote concatenation links between respective layers from the encoding and decoding stages, see Fig. 2. The output layer, d L , is assumed to be of the same mesh format as the input layer but can have a different number of channels (features), c L . Finally, we define the Graph U-Net network as a parameterized transformation where θ = L l=1 {θ l } is a concatenated vector of network parameters.
The calibration of the Graph U-Net parameters is done through a supervised learning, by fitting a given known input-output training dataset. The training dataset, D tr = {(f 1 , u 1 ), ..., (f Mtr , u Mtr )}, is in the mesh format, and the training is done by minimizing the following mean squared error loss function which gives the optimal parameters θ * = arg min θ L(D tr , θ). For the purpose of this work, we will focus on sparse graphs that derive from data that is spatially organised in the form of meshes. Those can be 1D, 2D, or higher-dimensional meshes, of an arbitrary connection topology (see Fig. 3 for examples of 2D meshes). The graph can be conveniently represented by a symmetric, square, Boolean adjacency matrix, A, whose order is equal to the number of nodes in the original mesh. To simplify the further notation, all nodes (vertices) are self-connected (have loops), which results in having 1 on the diagonal of A. This allows us to more easily express certain graph operations that are used in this work, for instance, the k-th power of a graph A, and the selection of pooling sub-graphs that is presented in Section 2.4.
It is fairly straightforward to generate an adjacency matrix from an element connectivity matrix of the mesh. For that reason we will not discuss it in detail. The only point to be emphasized is that we make all nodes belonging to a given element mutually interconnected in the resulting graph (as they can be assumed to be strongly inter-related). We can visually represent it by adding more links as compared to a standard wire-frame visualization of finite-elements (see, e.g., the dashed lines in Fig. 3a).
Remark: In our work we do not consider any attributes for the edges of a graph. Therefore, the data is only represented through nodal features and node-node connections which are defined through the adjacency matrix A.

Multi-channel Aggregation (MAg) layer
The proposed novel neural network layer, MAg, is a multi-channel local aggregation layer that can operate on graph-structured data. Its architecture is a direct extension to the standard convolutional layer in CNNs, in which a shareable convolution window is used, making CNNs restricted to grid-structured data. In the MAg layer, instead, we propose to use fully-trainable local weighted aggregations (the so-called message passing scheme), where the aggregation neighborhood of a given node is prescribed through the graph connectivity (the adjacency matrix). As such, the scheme is very well suited for sparse graphs and can be directly applied to graphs that derive from arbitrary 2D or 3D meshes.
The use of multiple channels aims to improve the capabilities of the network to capture nonlinearities. In the multi-channel version, each node represents a vector of values (features), which can be visualised as multiple layers (channels) of the same graph structure (see the schematics in Figure 4a). The transformation between the input-and output multichannel graphs is realised by applying multiple MAg aggregations on vector data to produce respective multiple components of output vectors. Note that the input/output channels of the whole network have usually a certain meaning, and their sizes are fixed (e.g., three RGB channels of a color image at the input and a single channel of a segmented image at the output). The number of channels in the latent layers can be chosen arbitrarily, which is up to the choice of a designer of a particular graph U-Net architecture. More formally, we will consider the MAg layer as a parameterized transformation between the input and output nodes, defined as where N i = {j | A ij = 1} is a set of neighbours of a node i to be aggregated, α and β represent the output and input channels, respectively, while k l+1 and b l+1 are trainable weights and biases, respectively. In this multi-channel definition, for a given component, d l+1 i,α , of an output, a single aggregation is performed throughout the neighborhood, N i , and all the input channels, β ∈ {1, . . . , c l }. The kernel parameters of MAg transformation, k l+1 i,j,α,β , are not shared, i.e, they can be independently trained for each aggregation window (note the free indexes i and α).

Comparison to existing graph aggregation/convolution layers
As already mentioned in the introduction, the very idea of generalisation of convolution layers to arbitrary graph structure is not new. In fact, various concepts have emerged so far, [Zhou et al., 2020] [Chen et al., 2020], most of which are compatible with the U-Net framework proposed in the present work. Below, we will discuss several of them, introducing a unified notation that will facilitate a qualitative comparison with respect to the proposed MAg layer, see Table 1.

Layer Transformation
GCN [Kipf and Welling, 2016] GAT [Veličković et al., 2017] Table 1: Comparison of the MAg layer with selected state of the art graph convolution layers, biases are omitted for the sake of brevity. In GAT formulation, α = (t − 1)Ng + γ, which represents the stacking operation for the multi-head attention mechanism, with t ∈ (1, . . . , N l+1 h ) is the attention head index, and γ ∈ (1, . . . , Ng) is the internal channel index (cardinality of each node in layer l + 1).
Graph neural network layers aim to utilize the information about assumed correlations in data, with the graph structure expressing those correlations. The general approach is to specify a suitable (possibly nonlinear) trainable local transformation that can aggregate the information from a node in consideration and its neighbours. (This aggregation is followed by a chosen activation function before being propagated to the next layer.) Such transformations form a wide class of, so-called, message passing schemes, and can combine shareable (independent of a node) and non-shareable (dependent on a node, i.e., independently-trainable) sets of parameters.
The simplest and most lightweight realisations of the graph aggregation/convolution layer concept only utilise shareable weights, see, e.g., the Graph Convolutional Network (GCN), [Kipf and Welling, 2016]. In those approaches, a non-trainable (arbitrary) weighted aggregation is performed prior to application of a shareable trainable operator -something completely opposite to our MAg layer, which is fully trainable. This enables to keep the number of trainable parameters low, which is achieved at the cost of relatively low capacity of such networks. This low capacity can not be straightforwardly increased by simply deepening the network because of the well-known over-smoothing phenomenon.
We will discuss two out of many available approaches to increase the capacity of graph neural networks. The first approach relies on the multi-head attention mechanism which allows to assign different importance to nodes in the neighbourhood, see, e.g, the Graph Attention Network (GAT), [Veličković et al., 2017]. In the attention mechanism, the weights used in local aggregation depend on input nodal features, which makes the concept qualitatively different from all approaches (including the MAg layer) that use input-independent aggregation weights. The second class of approaches resemble the MAg layer more closely.
Particular notable examples of that approach are the Spatial-Temporal Graph Convolution Network (ST-GCN), [Yan et al., 2018], and the Semantic Graph Convolution Network (SemGCN), [Zhao et al., 2019], which have been introduced in the particular context of human pose recognition problem (the computer vision domain). The common features of MAg and SemGCN layers are the input-independent learnable weighted aggregation and the use of channels to increase the model capacity. The difference is that the MAg doesn't use a shared transformation matrix (w) nor the softmax normalisation -both used in the case of SemGCN.
To summarize, the proposed MAg layer relies on one of the most flexible message-passing schemes, with no shareable parameters. This promises a very high capacity of the MAg network. Also, as shown above, the proposed MAg layer is compatible with other graph convolution/aggregation layer concepts, and thus can be straightforwardly exchanged, if needed.

Graph pooling and unpooling layers
We introduce a novel clustering-based graph pooling layer, which enables to down-sample the input graph to a reduced size thereby propagating only important information to the further layers. This enables a reduction of the number of computations in the network. We inherit the idea of our pooling layer from the traditional pooling operation in CNNs. I.e. we extend the concept of local patches in regular grids in CNNs to variable-size patches in graphs. Similar to pooling layers in CNNs, our approach is purely topological as we do not use any input nodal feature information in generating the pooled graph.
For a given input graph G, we arbitrarily generate a set of non-overlapping sub-graphs S 1 , S 2 ..., S N pool over which pooling operation is performed as show in Figure 5.
The set S i stands for the nodes present in the i th subgraph. The procedure to generate these subgraphs and then the pooled adjacency matrix A pool from them is described in Algorithm 1. Dimension of the pooled graph is always equal to the number of subgraphs generated. Graph pooling layers do not affect the channel dimension i.e. pooling is performed individually per each channel of the input. The pooling layer is described as follows: Here the 'aggr' operation can be a max/min/avg etc. Graph unpooling is the reverse of the pooling operation, i.e. the output graph of the unpooling layer will exhibit the exact similar topology as that of the original graph over which pooling was performed. While the features of pooled nodes are replicated to all the nodes in the respective subgraph regions of the unpooled/original graph, similar to the upsampling operation in CNNs. The unpooling layer formally reads as:  Remark: During a single forward pass of MAg, local aggregation is performed at each node i.e. each node of the graph will have the feature information of its adjacent nodes only. And therefore, spatially further located nodes do not exchange information. One way to handle this issue would be to apply the MAg layer several times as shown in Figure 6. But depending on the size and topology of the domain, one might require multiple number of MAg operations, and thus the process can become computationally far too expensive. And this brings a natural motivation behind using the pooling layers. The proposed pooling layer in this work is a type of clique pooling approach, where each clique represents a subgraph. A similar strategy has been implemented in some of the recent works [Luzhnica et al., 2019]. However, these approaches generate overlapping cliques (subgraphs) which doesn't reduce the input graph dimension to a desirable extent. Hence we introduce a non-overlapping clique generation approach for our layer.
The pooled graph can be seen as a reduced space representation of the parent graph, and each node in it has feature information corresponding to multiple nodes of the parent graph. The pooled graph has a less complex topology when compared to the parent graph, and hence spatially further located nodes can communicate feature information using only a few MAg layers as illustrated in Figure 7.

Hyperelastic FEM formulation
Capabilities of the proposed MAgNET framework are demonstrated on the numerically generated non-linear finite element datasets. All the utilised benchmark examples in this work follow the hyperelastic deformation law. For a boundary volume problem in the domain Ω and Γ D , Γ N as Dirichlet and Neumann boundary conditions respectively; the virtual work principle for nonlinear elastostatic equations is as follows: Here P(u) is the first Piola-Kirchhoff stress tensor.b andt are corresponding body and traction forces. While u and δu belong to appropriate functional spaces, u =ū and δu = 0 on Γ u . For the deformation gradient tensor F = I + ∇u, strain-energy density function of a hyperelastic material is related to P(u) through the relation P(F ) = ∂W (F ) ∂F .
Finite element discretisation of the domain reduces the Eq.(9) to the system of non-linear equations which expresses the balance between external and internal nodal forces as: Now for a given external traction/body force, we represent the corresponding nodal external force vector as f ext = f , we solve the above system of equations using a Newton-Raphson scheme to obtain a solution in the form of nodal displacements. All the finite element computations are performed using the AceGen/AceFem framework [Korelc, 2002].

Generation of mesh based data using FEM
Nonlinear finite element datasets are generated for the 2D and 3D benchmark cases as described in the Figure 8. We show the capacity of our framework for arbitrary mesh topology inputs (triangular, quad, hexahedron, tetrahedron), unlike the CNN based approaches which demand quad or hexahedron meshes only (grid inputs). For all the cases, Neo-hookean hyperelastic law with the following strain energy potential is used.
where the invariants J and I c are given in terms of deformation gradient F as while µ and λ are Lame's parameters, which can be expressed in terms of the Young's modulus, E, and the Poisson's ratio, ν, as The material properties are used in this work are provided in the   The datasets are randomly split into training sets, M tr (95%), and testing sets, M te (5%). The characteristics of FE meshes and datasets for all three problems are provided in Table 3 198 f x , f y = -5 to 5 N 4560 + 240 3D breast 3105 b x , b y = -6 to 6 N/m 3 , b z = -3 to 3 N/m 3 7600 + 400

Validation Metric
In this work mean absolute error norm is used as the validation metric. For each of the M te force-displacement pairs in the test dataset D te = {(f 1 , u 1 ), ..., (f Mte , u Mte )}, following error metric is used: F is the number of dofs of the mesh. For m th test example, G(f m ) is the network prediction and u m is the true finite element solution. To have a single validation metric over the entire test set, we compute the average mean normē and the corrected sample standard deviation σ(e) (standard deviation of averaged errors) as below, we also note the maximum error e max , associated to the particular dof prediction in the entire test set.

Implementation details
In this work, we implement MAgNET architectures (see Figure 2) to four cases described in Section 3.1. For the two structured mesh examples, we also implement the CNN U-Net framework proposed in our previous work [Deshpande et al., 2022]. For both CNN and MAgNET, the number of U-Net levels and the number of channels are chosen in a way to have a similar number of trainable parameters for both frameworks. In the case of CNNs, 2D/3D convolution layers are used depending on the dimension of the problem; whereas MAg layers in MAgNET can handle both 2D/3D cases. Table 4 summarises the exact network architectures implemented in this work.
For both graph & CNN, maximum pooling is implemented and also more number of pooling operations are used as the size of the problem increases (i.e. number of nodes in the mesh increase). In the case of the graph pooling layers, the random seed of the first pooling layer can be selected through a grid search so that the lowest-level graph has the least number of nodes. This can ensure the propagation of information between the farthest nodes is done by using the minimum number of MAg operations in the lowest graph representation.  All the examples are trained with minimizing the loss function using Adam optimizer with an initial learning rate of 1 × 10 −4 which linearly decreases to 1 × 10 −6 during the training.
In the case of MAgNETs, leaky ReLU activation (negative slope = 0.01) is used while ReLU activation is used in the case of CNNs. Network trainings are performed using TensorFlow on Tesla V100-SXM2 GPU, on HPC facilities of the University of Luxembourg [Varrette et al., 2014] using a batch size of 4.

Prediction on test examples
We validate the MAgNet framework on different examples described in Section 3.1. Our framework is primarily useful when the input data is in the unstructured format, however, we also validate it to the structured inputs to compare its performance with the CNN approach. Hence this section is divided into two parts, starting with the implementation to the structured meshes and then moving on to unstructured meshes which usually arise in real word applications.

Structured meshes
At first, we implement MAgNET to structured meshes so that it can be straightforwardly compared with the CNN framework developed in our previous work [Deshpande et al., 2022]. It has to be noted that the CNN approach can only be used for the structured meshes and it comes with an additional preprocessing step to make the topology compatible to be used with the CNN framework. For example see Figure 9, zero padding is necessary to convert L-shape mesh into a structured mesh. On the other hand, graph-based approaches such as MAgNET do not require this step.

24
Graph U-Net input CNN U-Net input zero padding Figure 9: Zero-padding to make the topology compatible to be used with the CNN framework.
Error metrics for the entire test sets of structured mesh examples obtained using MAg-NET as well as CNN approach are summarised in  To get a fair idea of performance over the entire test set we also plot the mean error (e) of each test example for both the L-shape and beam case. In Figure 10 these errors are sorted as per the increasing order of maximum nodal displacements of the example. Errors are small and also they are less sensitive to increasing nodal displacement magnitudes. 3D beam Line fit to 3D beam 2D L-shape Line fit to 2D L-shape Now we look at the individual examples with the highest nodal displacement magnitude for both structured mesh cases. For the 2D L-shape example in Figure 11, the nodal displacement for the right corner node is 2.7 m. As one can see in Figure 11a, MAgNET prediction is perfectly coinciding the reference FEM solution. Figure 11b indicate very low nodal L 2 error values between MAgNET and FEM solutions. For the same example, we also provide the nodal L 2 errors obtained for the CNN U-Net framework in Figure 11c. Both MAgNET and CNN approaches are able to predict non-linear deformations with very low errors. Similarly, we showcase the highest nodal displacement magnitude example for the 3D-beam case. The nodal displacement for the right corner for this example is 1.47 m. Figure 12a shows that MAgNET is able to correctly predict the deformed mesh, while Figure 12b& Figure 12c show low L 2 error contours of displacements for MAgNET and CNN solutions respectively. Thus MAgNET performs on par with the CNN framework for structured meshes without any need of the preprocessing step.
As described in Section 2.3, the ability to add more channels in the MAg layer enables to increase the model capacity with which we can capture high non-linearities in the underlying data (similar to channels in CNNs). To show the advantage of channels, we train five MAgNET networks on the L-shape dataset. In each, we use a 4-level MAgNET architecture with a constant number of channels at all levels. In each network, we only change the number of channels used. Figure 13 shows that the error significantly drops down as the number of channels increases (so the network parameters). The MAgNET architecture with 32 channels at each level (∼ 7M parameters) gives a very low error of e = 3 E-4 [m]

Unstructured meshes
The proposed MAgNET framework is capable of dealing with unstructured meshes with no extra effort and is also very efficient in making accurate predictions. Here we consider two cases, first one is deformation under the application of point loads (for 2D Beam) as in the case of structured examples, and the second, is deformation under body forces (for 3D breast).  (-1.75,1.31,-1.7)N on the second last node (a) Deformed mesh predicted using MAgNET (blue), for comparison FEM solution is presented (red) and undeformed mesh is represented by gray (b) L2 error of nodal displacements between MAgNET and FEM solution. The relative error for the right corner node displacement using MAgNET is 2.3% (c) L2 error of nodal displacements between CNN U-Net and FEM solution. The relative error for the right corner node displacement using CNN is 1.6%. Table 6 provides the error metrics for the entire test set for the two unstructured meshes considered. Again we also plot the mean error e, of each test example in both cases and sort it as per the ascending order of the maximum nodal displacement. Figure 14 shows that the errors for all the examples are reasonably low and at the same time they are less sensitive to the increase of the displacement magnitude.  Now we look at two examples, one with the force applied at the tip (this is the the example with highest nodal displacement) and another with the force applied in the middle region. The displacement of the right tip node for the first example is 1.4 m and for the second example, it is 0.94 m. Figure 15 shows strong agreement of deformed meshes predicting using MAgNET and FEM solution for both the cases. Also, the nodal error contour plots indicate very low prediction error for both examples. Usually, errors are high at the point of application of node as these nodes have high local non-linear deformations.
The proposed MAgNET framework has only been trained by minimizing the loss for displacement values and has no additional information about the governing physics. However, we still showcase the capabilities of MAgNET in retrieving physics-relevant quantities such as residuals and stresses. Figure 16 shows nodal residual force values for the same two examples described in Figure 15. We observed that the framework is still able to retrieve  nodal residual forces at the constrained degrees of freedom with good accuracy (dofs with Dirichlet and Neumann boundary conditions). However, the MAgNET solution is observed to predict undesired local force imbalance. Ideally, we expect to have zero residual forces at the unconstrained nodes, although we observe local non-zero hot spots for both examples. For the same examples, we also show the comparison of von Mises stresses obtained using MAgNET and FEM solutions respectively. Figure 17 shows that the MAgNET solution is able to correctly predict stress distribution for both examples. Figure 17e-17f show low absolute error between von Mises stresses obtained using MAgNET and FEM solution respectively, they indicate localised errors.
Above mentioned localised errors in computing residual and von Mises stresses can be avoided by enriching the loss function of the MAgNET network with relevant physics-based information such as residuals and stresses, for example as implemented by [Odot et al., 2021][As'ad et al., 2022. MAgNET is a type of neural network architecture and it doesn't modify optimisation objective which is used to train the network. Hence MAgNET can be easily integrated into any state of the art PINN version by simply using it along with the relevant physics informed training objective, to convert it into a Physics Informed MAgNET.

3D breast: deformation under body forces
Breast deformation simulations are of significant importance in breast cancer surgical planning. Patient orientation while preoperative medical image acquisition differs from the intra-operative orientation and it is beneficial to track the exact location of the tumor under different orientations [Lavigne et al., 2022] [Alcañiz et al., 2022]. Hence we showcase the capabilities of our framework to accurately predict deformations of breast anatomy under body forces.
As described in the Section 3.1, a dataset is generated by applying different body force densities in all the three direction. Although, body force densities are not the direct input to the network as we first compute corresponding nodal body force values which is then given as an input to MAgNET. This ensures that the training data to MAgNET is very randomly generated. Table 6 shows very low test set errors for the breast dataset. The maximum displacement of the tip of breast for this dataset is 0.065 m.
Further, we analyse a test example with the highest displacement magnitude. Figure 18 shows the overlap of deformed meshes obtained using the MAgNET and true FEM solution,  Figure 15 (plotted on deformed meshes). The relative error for retrieving the total reaction force at the fixed interface is (a) 4.7% for the first example (b) 0.1% for the second example.
they are in well agreement with each other for both examples. Nodal displacement for the tip of the breast for this test example is 0.065 m. Figure 18b shows the low L 2 error of nodal displacements between the MAgNET and FEM solution. MAgNET is able to accurately predict deformed meshes for unseen test examples within the training range. During the training, we only considered z-direction body force density (b z ) within -3 to 3 N/m 3 , but now at the inference time, we apply body forces from -7 to 7 N/m 3 . Figure 19a shows that the error is fairly low and is not increasing within the training region and it increases rapidly outside. This is a well-known effect that the neural networks are prone to high errors in the extrapolated region. Figure 19b-19c show deformed meshes predicted for b z = 5 & b z = 9 N/m 3 , both outside the training data region. MAgNET is observed to give visually acceptable results although the accuracy of the framework decreases as we move away from the training data.
It has to be noted that our framework learns only from the data, without any explicit input about boundary conditions. The proposed framework is not only able to detect the boundary conditions by learning it implicitly from the data but is also capable of tracking high non-linear deformations near the fixed interface as shown in Figure 20 (for the same example in Figure 18). We also plot the residual values on the deformed mesh. It is observed that MAgNET is again able to accurately compute the reaction forces at the fixed interface as shown in Figure 21. However, as visible in the plot, MAgNET performed poorly at some local high non-linear deformation regions. This can be corrected by using a physics-aware loss function as motivated in the earlier paragraph.

Conclusion
In this work, we proposed MAgNET, a novel geometric deep learning framework for efficient supervised learning on graph-structured data. MAgNET is comprised of two neural network operations: MAg and graph pooling layers which constitute a graph U-Net architecture capable of learning on large-scale inputs. MAg successfully extends the concept of convolutional neural networks to arbitrary non-grid (unstructured) inputs and hence it will be useful for the wider scope of engineering applications.  We showed the capabilities of MAgNET through application to nonlinear FEM simulations, although it can be straightforwardly utilised for any form of discretisation. MAgNET can incorporate any arbitrary mesh and can efficiently learn on irregular geometries. In particular, we showed that MAgNET could accurately learn a highly nonlinear force-displacement mapping for multiple cases including real-world geometries such as those arising in biomechanical simulations.
We demonstrated numerically that MAgNET could capture the underlying physics of the problem with good confidence by learning implicitly only from the data. However as pointed out in the article, performance can be further improved by integrating quantities of interest with the learning objective function. Hence, Physics Informed MAgNET is a very natural extension of the framework to be implemented in immediate future. The proposed MAg operation can also be extended to the Bayesian version by performing local aggregations with probability distributions in place of discrete weights, as done in the case of CNNs in our previous work [Deshpande et al., 2022]. Bayesian MAgNET will be capable of tracking uncertainties that are inherent to the choice of network architecture as well as that of the real-world data.

Acknowledgements:
This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 764644. Jakub Lengiewicz would like to acknowledge the support from EU Horizon 2020 Marie Sklodowska Curie Individual Fellowship MOrPhEM under Grant 800150. This paper only contains the author's views and the Research Executive Agency and the Commission are not responsible for any use that may be made of the information it contains.
Stéphane Bordas and Jakub Lengiewicz are grateful for the support of the Fonds National de la Recherche Luxembourg FNR grant QuaC C20/MS/14782078. Stéphane Bordas received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 811099 TWINNING Project DRIVEN for the University of Luxembourg. True breast tip displacement is 1.2 cm and relative error for its prediction using MAgNET is 11.8% (c) True breast tip displacement is 2.4 cm while relative error for MAgNET prediction is 21.2%. Figure 20: MAgNET is able to implicitly learn fixed boundary from the data. Also high local nonlinear deformation near the interface is accurately captured as shown in the inset. This is the first example described in Figure 18