CACTUS: Detecting and Resolving Conflicts in Objective Functions
  Machine learning (ML) models are constructed by expert ML practitioners using
various coding languages, in which they tune and select models hyperparameters
and learning algorithms for a given problem domain. They also carefully design
an objective function or loss function (often with multiple objectives) that
captures the desired output for a given ML task such as classification,
regression, etc. In multi-objective optimization, conflicting objectives and
constraints is a major area of concern. In such problems, several competing
objectives are seen for which no single optimal solution is found that
satisfies all desired objectives simultaneously. In the past VA systems have
allowed users to interactively construct objective functions for a classifier.
In this paper, we extend this line of work by prototyping a technique to
visualize multi-objective objective functions either defined in a Jupyter
notebook or defined using an interactive visual interface to help users to: (1)
perceive and interpret complex mathematical terms in it and (2) detect and
resolve conflicting objectives. Visualization of the objective function
enlightens potentially conflicting objectives that obstructs selecting correct
solution(s) for the desired ML task or goal. We also present an enumeration of
potential conflicts in objective specification in multi-objective objective
functions for classifier selection. Furthermore, we demonstrate our approach in
a VA system that helps users in specifying meaningful objective functions to a
classifier by detecting and resolving conflicting objectives and constraints.
Through a within-subject quantitative and qualitative user study, we present
results showing that our technique helps users interactively specify meaningful
objective functions by resolving potential conflicts for a classification task.
