{'fraction_non_alphanumeric': 0.022208121827411168, 'fraction_numerical': 0.0, 'mean_word_length': 5.136186770428016, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Selecting Optimal Completion to Partial Matrix via Self-Validation

In many applications such as film recommendation, one often encounters the problem of estimating the unseen entries in a partially observed matrix, formally known as matrix completion. Over the past several decades, lots of effective methods have been established in the literature, and each method may contain several hyper-parameters. For a partial matrix, one can use those methods with certain parametric settings to obtain a large number of completions. Now, a critical question is, how to select the optimal completion from a number of candidates? This question is indeed a hard to answer, because in practice the true values of the missing entries are unknown. Thus far, the only approach for dealing with the issue is through data-validation, which is to first split the observations into two subsets, a training set and a validation set, and then choose the model that performs best on the validation set as the winner to produce the final results. Though straightforward, this approach might fall in a non-optimal model that overfits the validation set. In this work, we shall suggest a different approach called self-validation, which accounts on a special metric that can evaluate the “goodness” of a completion without using any validation data. The metric is derived from the recently established isomeric condition, measuring the identifiable degree of the completion itself. Extensive experiments demonstrate that our self-validation approach is better than the commonly used data-validation.