TY - GEN
T1 - Big Data Quality
T2 - 7th IEEE International Congress on Big Data, BigData Congress 2018
AU - Taleb, Ikbal
AU - Serhani, Mohamed Adel
AU - Dssouli, Rachida
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/9/7
Y1 - 2018/9/7
N2 - With the advances in communication technologies and the high amount of data generated, collected, and stored, it becomes crucial to manage the quality of this data deluge in an efficient and cost-effective way. The storage, processing, privacy and analytics are the main keys challenging aspects of Big Data that require quality evaluation and monitoring. Quality has been recognized by the Big Data community as an essential facet of its maturity. Yet, it is a crucial practice that should be implemented at the earlier stages of its lifecycle and progressively applied across the other key processes. The earlier we incorporate quality the full benefit we can get from insights. In this paper, we first identify the key challenges that necessitates quality evaluation. We then survey, classify and discuss the most recent work on Big Data management. Consequently, we propose an across-the-board quality management framework describing the key quality evaluation practices to be conducted through the different Big Data stages. The framework can be used to leverage the quality management and to provide a roadmap for Data scientists to better understand quality practices and highlight the importance of managing the quality. We finally, conclude the paper and point to some future research directions on quality of Big Data.
AB - With the advances in communication technologies and the high amount of data generated, collected, and stored, it becomes crucial to manage the quality of this data deluge in an efficient and cost-effective way. The storage, processing, privacy and analytics are the main keys challenging aspects of Big Data that require quality evaluation and monitoring. Quality has been recognized by the Big Data community as an essential facet of its maturity. Yet, it is a crucial practice that should be implemented at the earlier stages of its lifecycle and progressively applied across the other key processes. The earlier we incorporate quality the full benefit we can get from insights. In this paper, we first identify the key challenges that necessitates quality evaluation. We then survey, classify and discuss the most recent work on Big Data management. Consequently, we propose an across-the-board quality management framework describing the key quality evaluation practices to be conducted through the different Big Data stages. The framework can be used to leverage the quality management and to provide a roadmap for Data scientists to better understand quality practices and highlight the importance of managing the quality. We finally, conclude the paper and point to some future research directions on quality of Big Data.
KW - Big Data
KW - Data Quality
KW - Quality Management framework
KW - Quality of Big Data
UR - http://www.scopus.com/inward/record.url?scp=85054869974&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85054869974&partnerID=8YFLogxK
U2 - 10.1109/BigDataCongress.2018.00029
DO - 10.1109/BigDataCongress.2018.00029
M3 - Conference contribution
AN - SCOPUS:85054869974
T3 - Proceedings - 2018 IEEE International Congress on Big Data, BigData Congress 2018 - Part of the 2018 IEEE World Congress on Services
SP - 166
EP - 173
BT - Proceedings - 2018 IEEE International Congress on Big Data, BigData Congress 2018 - Part of the 2018 IEEE World Congress on Services
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 2 July 2018 through 7 July 2018
ER -