{'fraction_non_alphanumeric': 0.010338345864661654, 'fraction_numerical': 0.0, 'mean_word_length': 5.454545454545454, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A multilevel evaluation method for heuristics with an application to the VRPTW

The ﬁeld of combinatorial optimisation has inspired the development of a large number of heuristic solution procedures. These methods are commonly assessed using a competitive evaluation methodology that may give an indication of which algorithm has a better performance. A next step in the experimental analysis is to uncover ‘why’ one algorithm performs better. Which elements are responsible for good or bad performance? How does the performance of elements vary across the design space? What is the inﬂuence of the speciﬁc problem instance that is being solved? We focus on gaining a better understanding of heuristic algorithm performance and demonstrate that the application of a proper statistical methodology can provide researchers insight into how performance is aﬀected by the diﬀerent algorithm parameters and components. As an example, we apply a multilevel statistical analysis to a large neighbourhood search algorithm for the vehicle routing problem with time windows.