Using Deep Reinforcement Learning for Zero Defect Smart Forging
  Defects during production may lead to material waste, which is a significant
challenge for many companies as it reduces revenue and negatively impacts
sustainability and the environment. An essential reason for material waste is a
low degree of automation, especially in industries that currently have a low
degree of digitalization, such as steel forging. Those industries typically
rely on heavy and old machinery such as large induction ovens that are mostly
controlled manually or using well-known recipes created by experts. However,
standard recipes may fail when unforeseen events happen, such as an unplanned
stop in production, which may lead to overheating and thus material degradation
during the forging process. In this paper, we develop a digital twin-based
optimization strategy for the heating process for a forging line to automate
the development of an optimal control policy that adjusts the power for the
heating coils in an induction oven based on temperature data observed from
pyrometers. We design a digital twin-based deep reinforcement learning (DTRL)
framework and train two different deep reinforcement learning (DRL) models for
the heating phase using a digital twin of the forging line. The twin is based
on a simulator that contains a heating transfer and movement model, which is
used as an environment for the DRL training. Our evaluation shows that both
models significantly reduce the temperature unevenness and can help to automate
the traditional heating process.
