Big data quality: A data quality profiling model

Ikbal Taleb, Mohamed Adel Serhani, Rachida Dssouli

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

Big Data is becoming a standard data model, and it is gaining wide adoption in the digital universe. Estimating the Quality of Big Data is recognized to be essential for data management and data governance. To ensure a fast and efficient data quality assessment represented by its dimensions, we need to extend the data profiling model to incorporate also quality profiling. The latter encompasses more value-added quality processes that go beyond data and its corresponding metadata. In this paper, we propose a Data Quality Profiling Model (BDQPM) for Big Data that involves several modules such as sampling, profiling, exploratory quality profiling, quality profile repository (QPREPO), and the data quality profile (DQP). Thus, the QPREPO plays an important role in managing many quality-related elements such as data quality dimensions and their related metrics, pre-defined quality actions scenarios, pre-processing activities (PPA), their related functions (PPAF), and the data quality profile. Our exploratory quality profiling method discovers a set of PPAF from systematic predefined quality actions scenarios to leverage the quality trends of any data set and show the cause and effects of such a process on the data. Such a quality overview is considered as a preliminary quality profile of the data. We conducted a series of experiments to test different features of the BDQPM including sampling and profiling, quality evaluation, and exploratory quality profiling for Big Data quality enhancement. The results prove that quality profiling tracks quality at the earlier stage of Big data life cycle leading to quality improvement and enforcement insights from exploratory quality profiling methodology.

Original languageEnglish
Title of host publicationServices – SERVICES 2019 - 15th World Congress, Held as Part of the Services Conference Federation, SCF 2019, Proceedings
EditorsYunni Xia, Liang-Jie Zhang
PublisherSpringer Verlag
Pages61-77
Number of pages17
ISBN (Print)9783030233808
DOIs
Publication statusPublished - 2019
Event15th World Congress on Services, SERVICES 2019 held as part of the Services Conference Federation, SCF 2019 - San Diego, United States
Duration: Jun 25 2019Jun 30 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11517 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th World Congress on Services, SERVICES 2019 held as part of the Services Conference Federation, SCF 2019
Country/TerritoryUnited States
CitySan Diego
Period6/25/196/30/19

Keywords

  • Big data quality
  • Data quality profile
  • Data quality profiling
  • Profile repository

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Big data quality: A data quality profiling model'. Together they form a unique fingerprint.

Cite this