#set page(numbering: "1", number-align: center)
#set heading(numbering: "1.1.1")

#counter(page).update(1)
#counter(heading).update(0)

#set math.equation(numbering: it => {[1.#it]})
#counter(math.equation).update(0)

#counter(figure).update(0)

= Introduction <intro-sec>

We live in an exciting time. Now dawns an age of remarkable confluence and innovation; machine learning and artificial intelligence stand poised to remake almost every aspect of science, technology, and everyday life. At the same time, our understanding of the world is increasing with ever more fervour. The era of gravitational-wave astronomy has well and truly begun, and bountiful volumes of new data will soon be at our fingertips. The machine learning community is unlocking novel new tools every month, and it would be a missed opportunity not to turn some of these powerful techniques onto the data hidden beneath interferometer noise.

This thesis is an exploration of the use of artificial neural networks to solve problems within gravitational-wave data analysis. Certainly, it is not the first work to attempt such a task, and indeed the intersection between the two fields has attracted much interest in recent years. Perhaps that is unsurprising --- both fields are new and exciting; it was only natural to try and merge the two. Despite this busy environment, we find that there is some room for innovation. Crucially, we try and work upward from first principles, and present our results in a manner that is easily comparable with other studies. The entirety of the codebase used in the creation of this thesis and all associated work is available here: #link("https://github.com/mrknorman/evolving_attention"), although, unfortunately, some of the data acquisition methods are gated behind LIGO-VIRGO-Kagra collaboration membership.

@gravitational-waves-sec and @machine-learning-sec, are introductory chapters that present background information in the areas of gravitational waves and machine learning respectively. They aim to both contextualize the data analysis methods presented throughout the rest of the thesis and, in the case of the machine learning chapter, introduce many of the analysis methods and nomenclature used throughout.  @gravitational-waves-sec gives a brief introduction to gravitational wave science, but it is considered less necessary for the understanding of this thesis than the following chapter. @machine-learning-sec is a more rigorous examination of the basics of artificial neural networks, which form the backbone of this work.

@application-sec does not present any novel work but instead acts as a general methodology for the data analysis techniques used in later chapters. This chapter introduces the two pieces of custom software written to facilitate the faster iteration of machine learning model training: cuPhenom @cuphenom_ref a GPU implementation of IMRPhenomD @imrphenom_d, and GravyFlow, a custom data acquisition and model training pipeline designed for machine learning in gravitational wave science. In @application-sec we also review the relevant literature to explore the work others have contributed to the field. We perform a series of experiments to evaluate the performance of unspecialized artificial neural networks to show the necessity of experimentation. We conclude this chapter by recreating some prominent results from the literature, which act as a baseline for comparison in further chapters.

When designing artificial neural network models, there are a large number of free parameters that are not optimised during model training, these parameters are known as hyperparameters. They pose a significant barrier when attempting to find optimal deep-learning solutions for data analysis tasks. Often, hyperparameter optimization is performed by human-guided trial and error. This increases the already difficult task of comparisons between the performance of models presented in the literature. It is never clear how optimal the hyperparameter optimisation process has been. @dragonn-sec presents Dragonn. This original work consists of a hyperparameter optimisation method that utilises genetic algorithms to find optimal network solutions. In this chapter, we optimise the hyperparameters of a network designed to detect the gravitational wave emissions of a pair of binary black holes in the second before they merge. We show that genetic algorithms can be used to optimise network hyperparameters, however, due to time constraints, we do not arrive at any solutions that are better than the current standard. We do, however, observe that training efficiency can be improved by adding artificial glitches into the noise-only training examples. It is hypothesised that this "inoculation" method prevents the model from relying on excess power alone, and thus forces the model to examine signal shape. It is hoped that by exploring methods for automatic hyperparameter selection we might move a little toward a more robust and repeatable method for optimizing and comparing the performance of different model architectures. This work was used in the early development of the models that form the core of the MLy rapid burst detection pipeline @MLy.

@skywarp-sec is also original work. In this chapter, we explore the use of a more contemporary neural network architecture, the transformer @attention_is_all_you_need, which has found the most prominent use in natural language processing applications such as ChatGPT @chatgpt_ref. More precisely, we explore the use of the core innovations presented by the initial transformer paper @attention_is_all_you_need, the attention mechanism. We apply attention-based models to the binary black hole merger detection problem and find a marginal improvement over more widely used convolutional neural networks.

@crosswave-sec is the final piece of original work presented in this thesis. Here, we focus on a more specific problem that is currently not an issue but will become a major component of search pipelines when next-generation gravitational wave detectors come online. We explore the possibility that artificial neural networks can aid in the differentiation between single gravitational wave signals and pairs of overlapping gravitational wave signals. We also build a large network utilizing cross-attention layers in order to attempt to extract the parameters from both signals in an overlapping pair. We do this both to provide an established parameter estimation method with more information to improve its operation and as an exercise in its own right, potentially as a prelude to an entire machine-learning-based method to perform parameter estimation on overlapping signals. We find success in both tasks, with the smaller network able to match the classification performance of matched filtering when distinguishing between overlapping and single signals. The larger model was able to extract the merger times of each event in two overlapping binary signals with high accuracy, though the other parameter estimation tasks attempted were less successful.

Lastly, in @conclusion-sec, we summarise the key results of each chapter and discuss future steps that could yield further interesting results.