Data-Driven Innovation: The Potential of Synthetic Data through Generative AI

Abstract

Data scarcity constitutes a protracted challenge to AI decision-making across data-driven industries. Research has shown that demand for larger training datasets is growing, but the current method for data collection has become restricted due to scarcity and privacy concerns. This thesis aims to solve data scarcity through synthetic data generation by exploring the capability of Generative AI.

Based on a literature review on Generative AI and synthetic data, the crucial insight is that synthetic data can substitute insufficient real-world data in limitless quantities. While preserving the statistical representation of the original data (i.e. existing real-world data), synthetic data enhances the AI model development process, eliminates the risk of sensitive data exposure, and mitigates biases. In addition, a demonstration of synthetic data generation is presented using a Variational Autoencoder model (VAE) and a customized open-source library (Syngen) to reinforce the presented findings. Analysis of the output shows that generated synthetic data exhibits less than 2% variation with the original dataset when used for the same prediction task.

The results suggest that synthetic data can be a practical solution to overcome limitations in data acquisition. On this basis, it is recommended that data-driven industries utilize synthetic data to unleash new opportunities in the current data-driven era. Further research is required to identify possible factors that could diminish data cardinality while handling complex categorical data, aiming to improve the reliability and effectiveness of synthetic data.