Big data quality: A data quality profiling model

Ikbal Taleb, Mohamed Adel Serhani, Rachida Dssouli

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    7 Citations (Scopus)

    Abstract

    Big Data is becoming a standard data model, and it is gaining wide adoption in the digital universe. Estimating the Quality of Big Data is recognized to be essential for data management and data governance. To ensure a fast and efficient data quality assessment represented by its dimensions, we need to extend the data profiling model to incorporate also quality profiling. The latter encompasses more value-added quality processes that go beyond data and its corresponding metadata. In this paper, we propose a Data Quality Profiling Model (BDQPM) for Big Data that involves several modules such as sampling, profiling, exploratory quality profiling, quality profile repository (QPREPO), and the data quality profile (DQP). Thus, the QPREPO plays an important role in managing many quality-related elements such as data quality dimensions and their related metrics, pre-defined quality actions scenarios, pre-processing activities (PPA), their related functions (PPAF), and the data quality profile. Our exploratory quality profiling method discovers a set of PPAF from systematic predefined quality actions scenarios to leverage the quality trends of any data set and show the cause and effects of such a process on the data. Such a quality overview is considered as a preliminary quality profile of the data. We conducted a series of experiments to test different features of the BDQPM including sampling and profiling, quality evaluation, and exploratory quality profiling for Big Data quality enhancement. The results prove that quality profiling tracks quality at the earlier stage of Big data life cycle leading to quality improvement and enforcement insights from exploratory quality profiling methodology.

    Original languageEnglish
    Title of host publicationServices – SERVICES 2019 - 15th World Congress, Held as Part of the Services Conference Federation, SCF 2019, Proceedings
    EditorsLiang-Jie Zhang, Yunni Xia
    PublisherSpringer Verlag
    Pages61-77
    Number of pages17
    ISBN (Print)9783030233808
    DOIs
    Publication statusPublished - Jan 1 2019
    Event15th World Congress on Services, SERVICES 2019 held as part of the Services Conference Federation, SCF 2019 - San Diego, United States
    Duration: Jun 25 2019Jun 30 2019

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume11517 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference15th World Congress on Services, SERVICES 2019 held as part of the Services Conference Federation, SCF 2019
    Country/TerritoryUnited States
    CitySan Diego
    Period6/25/196/30/19

    Keywords

    • Big data quality
    • Data quality profile
    • Data quality profiling
    • Profile repository

    ASJC Scopus subject areas

    • Theoretical Computer Science
    • Computer Science(all)

    Fingerprint

    Dive into the research topics of 'Big data quality: A data quality profiling model'. Together they form a unique fingerprint.

    Cite this