Concept Representation Learning with Contrastive Self-Supervised
  Learning
  Concept-oriented deep learning (CODL) is a general approach to meet the
future challenges for deep learning: (1) learning with little or no external
supervision, (2) coping with test examples that come from a different
distribution than the training examples, and (3) integrating deep learning with
symbolic AI. In CODL, as in human learning, concept representations are learned
based on concept exemplars. Contrastive self-supervised learning (CSSL)
provides a promising approach to do so, since it: (1) uses data-driven
associations, to get away from semantic labels, (2) supports incremental and
continual learning, to get away from (large) fixed datasets, and (3)
accommodates emergent objectives, to get away from fixed objectives (tasks). We
discuss major aspects of concept representation learning using CSSL. These
include dual-level concept representations, CSSL for feature representations,
exemplar similarity measures and self-supervised relational reasoning,
incremental and continual CSSL, and contrastive self-supervised concept (class)
incremental learning. The discussion leverages recent findings from cognitive
neural science and CSSL.
