Unsupposable Test-data Generation for Machine-learned Software
  As for software development by machine learning, a trained model is evaluated
by using part of an existing dataset as test data. However, if data with
characteristics that differ from the existing data is input, the model does not
always behave as expected. Accordingly, to confirm the behavior of the model
more strictly, it is necessary to create data that differs from the existing
data and test the model with that different data. The data to be tested
includes not only data that developers can suppose (supposable data) but also
data they cannot suppose (unsupposable data). To confirm the behavior of the
model strictly, it is important to create as much unsupposable data as
possible. In this study, therefore, a method called "unsupposable test-data
generation" (UTG)---for giving suggestions for unsupposable data to model
developers and testers---is proposed. UTG uses a variational autoencoder (VAE)
to generate unsupposable data. The unsupposable data is generated by acquiring
latent values with low occurrence probability in the prior distribution of the
VAE and inputting the acquired latent values into the decoder. If unsupposable
data is included in the data generated by the decoder, the developer can
recognize new unsupposable features by referring to the data. On the basis of
those unsupposable features, the developer will be able to create other
unsupposable data with the same features. The proposed UTG was applied to the
MNIST dataset and the House Sales Price dataset. The results demonstrate the
feasibility of UTG.
