Learning to Design Circuits
  Analog IC design relies on human experts to search for parameters that
satisfy circuit specifications with their experience and intuitions, which is
highly labor intensive, time consuming and suboptimal. Machine learning is a
promising tool to automate this process. However, supervised learning is
difficult for this task due to the low availability of training data: 1)
Circuit simulation is slow, thus generating large-scale dataset is
time-consuming; 2) Most circuit designs are propitiatory IPs within individual
IC companies, making it expensive to collect large-scale datasets. We propose
Learning to Design Circuits (L2DC) to leverage reinforcement learning that
learns to efficiently generate new circuits data and to optimize circuits. We
fix the schematic, and optimize the parameters of the transistors automatically
by training an RL agent with no prior knowledge about optimizing circuits.
After iteratively getting observations, generating a new set of transistor
parameters, getting a reward, and adjusting the model, L2DC is able to optimize
circuits. We evaluate L2DC on two transimpedance amplifiers. Trained for a day,
our RL agent can achieve comparable or better performance than human experts
trained for a quarter. It first learns to meet hard-constraints (eg. gain,
bandwidth), and then learns to optimize good-to-have targets (eg. area, power).
Compared with grid search-aided human design, L2DC can achieve
$\mathbf{250}\boldsymbol{\times}$ higher sample efficiency with comparable
performance. Under the same runtime constraint, the performance of L2DC is also
better than Bayesian Optimization.
