{'fraction_non_alphanumeric': 0.02132579650565262, 'fraction_numerical': 0.006166495375128468, 'mean_word_length': 4.871794871794871, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
CLOSE ENOUGH RECOGNITION OF (STEP) FUNCTIONS USING RECOGNITION COEFFICIENTS BASED ON SQUARE WAVES

We present a technique to make a close enough recognition of a function, based on recognition coefficients. The recognition coefficients are determined using inner products 5 of the function to be recognized with a complete set of functions. This is how generalized Fourier coefficients are calculated. However they may not give a series representation of the function to be recognized as the complete set of functions need not be orthogonal. In particular step functions are investigated, and formulae derived for recognition coefficients of a step function based on a complete set of square wave functions. These recognition 10 coefficients are simpler faster and more accurate to calculate than coefficients of both regular Fourier series, and series based on square waves. Introduction Our objective is to make a close enough recognition of a function, usually a step function 15 which takes only a finite number of values. Measurements may be approximate, or there may be noise in the input, or there may be other changes in the data when it collected on different occasions. Also, if the experiment or process is repeated, there may be variations in the data received. In view of this, it is not possible to make an exact comparison, and we need to make a close enough recognition of such data. 20 One possibility is to represent such functions by a (generalized) Fourier series [1], and check corresponding coefficients for closeness. These series use orthogonal bases e.g. sine/cosine functions, Walsh functions, Haar functions or other wavelets [1]. An important requirement for making close enough recognition of such functions, is that small changes in the data should 25 cause small changes in the coefficients. So Haar functions or other wavelets are not a good choice, as successive Haar functions and wavelets use smaller and smaller segments of the functions we are trying to recognize. So a small shift in the data can cause a large change in the coefficients. (Haar functions and other wavelets can be used for image compression [3].) A better choice is to use sine/cosine functions or Walsh functions. 30 The sine/cosine functions are smooth, have a regular form, and use almost all the values of the function we are trying to recognize when calculating coefficients. The Walsh functions are step functions, do not have such a regular form, and use almost all the values of the function we are trying to recognize when calculating coefficients. The regularity and smoothness of the 35 sine/cosine functions are an advantage for the close recognition of smooth functions. The step like form of Haar functions are an advantage for the close recognition of step functions, but their less regular form may be a disadvantage. The s/c functions (later) are square waves, have a regular form, and use almost all the values 40 of the function we are trying to recognize when calculating coefficients. They form a complete set [4] but are not orthogonal. They can be used to calculate series coefficients [4] but it is no easy task, and is a slower process than calculating standard Fourier series coefficients. The recognition coefficients we calculate are simpler faster and more accurate to calculate than coefficients of both regular Fourier series and series based on square waves. Standard 45 generalized Fourier series coefficient formulae are calculated using inner products, but they provide recognition coefficients, not series coefficients. Recognition coefficients are compared for closeness. This is the basis of our recognition technique. Furthermore, the uniformity and step like nature 50 of the s/c functions makes them a good choice for the close enough recognition of step functions. They may also work well for smooth functions, but we have not studied this case.