Transitioning from Real to Synthetic data: Quantifying the bias in model
  With the advent of generative modeling techniques, synthetic data and its use
has penetrated across various domains from unstructured data such as image,
text to structured dataset modeling healthcare outcome, risk decisioning in
financial domain, and many more. It overcomes various challenges such as
limited training data, class imbalance, restricted access to dataset owing to
privacy issues. To ensure the trained model used for automated decisioning
purposes makes a fair decision there exist prior work to quantify and mitigate
those issues. This study aims to establish a trade-off between bias and
fairness in the models trained using synthetic data. Variants of synthetic data
generation techniques were studied to understand bias amplification including
differentially private generation schemes. Through experiments on a tabular
dataset, we demonstrate there exist a varying levels of bias impact on models
trained using synthetic data. Techniques generating less correlated feature
performs well as evident through fairness metrics with 94\%, 82\%, and 88\%
relative drop in DPD (demographic parity difference), EoD (equality of odds)
and EoP (equality of opportunity) respectively, and 24\% relative improvement
in DRP (demographic parity ratio) with respect to the real dataset. We believe
the outcome of our research study will help data science practitioners
understand the bias in the use of synthetic data.
