A Multi-Dimensional Evaluation of Synthetic Data Generators

Fida K. Dankar, Mahmoud K. Ibrahim, Leila Ismail

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)

Abstract

Synthetic datasets are gradually emerging as solutions for data sharing. Multiple synthetic data generators have been introduced in the last decade fueled by advancement in machine learning and by the increased demand for fast and inclusive data sharing, yet their utility is not well understood. Prior research tried to compare the utility of synthetic data generators using different evaluation metrics. These metrics have been found to generate conflicting conclusions making direct comparison of synthetic data generators very difficult. This paper identifies four criteria (or dimensions) for masked data evaluation by classifying available utility metrics into different categories based on the measure they attempt to preserve: attribute fidelity, bivariate fidelity, population fidelity, and application fidelity. A representative metric from each category is chosen based on popularity and consistency, and the four metrics are used to compare the overall utility of four recent data synthesizers across 19 datasets of different sizes and feature counts. The paper also examines correlations between the selected metrics in an attempt to streamline synthetic data utility.

Original languageEnglish
Pages (from-to)11147-11158
Number of pages12
JournalIEEE Access
Volume10
DOIs
Publication statusPublished - 2022

Keywords

  • Data utility
  • privacy enhancing technologies
  • synthetic data generators

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint

Dive into the research topics of 'A Multi-Dimensional Evaluation of Synthetic Data Generators'. Together they form a unique fingerprint.

Cite this