Subverting machines, fluctuating identities: Re-learning human
  categorization
  Most machine learning systems that interact with humans construct some notion
of a person's "identity," yet the default paradigm in AI research envisions
identity with essential attributes that are discrete and static. In stark
contrast, strands of thought within critical theory present a conception of
identity as malleable and constructed entirely through interaction; a doing
rather than a being. In this work, we distill some of these ideas for machine
learning practitioners and introduce a theory of identity as autopoiesis,
circular processes of formation and function. We argue that the default
paradigm of identity used by the field immobilizes existing identity categories
and the power differentials that co$\unicode{x2010}$occur, due to the absence
of iterative feedback to our models. This includes a critique of emergent AI
fairness practices that continue to impose the default paradigm. Finally, we
apply our theory to sketch approaches to autopoietic identity through
multilevel optimization and relational learning. While these ideas raise many
open questions, we imagine the possibilities of machines that are capable of
expressing human identity as a relationship perpetually in flux.
