Classification and novel class detection in concept-drifting data streams under time constraints

Mohammad Masud, Jing Gao, Latifur Khan, Jiawei Han, Bhavani M. Thuraisingham

Research output: Contribution to journalArticlepeer-review

303 Citations (Scopus)

Abstract

Most existing data stream classification techniques ignore one important aspect of stream data: arrival of a novel class. We address this issue and propose a data stream classification technique that integrates a novel class detection mechanism into traditional classifiers, enabling automatic detection of novel classes before the true labels of the novel class instances arrive. Novel class detection problem becomes more challenging in the presence of concept-drift, when the underlying data distributions evolve in streams. In order to determine whether an instance belongs to a novel class, the classification model sometimes needs to wait for more test instances to discover similarities among those instances. A maximum allowable wait time Tc is imposed as a time constraint to classify a test instance. Furthermore, most existing stream classification approaches assume that the true label of a data point can be accessed immediately after the data point is classified. In reality, a time delay Tl is involved in obtaining the true label of a data point since manual labeling is time consuming. We show how to make fast and correct classification decisions under these constraints and apply them to real benchmark data. Comparison with state-of-the-art stream classification techniques prove the superiority of our approach.

Original languageEnglish
Article number5453372
Pages (from-to)859-874
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume23
Issue number6
DOIs
Publication statusPublished - 2011
Externally publishedYes

Keywords

  • Data streams
  • K-means clustering
  • concept-drift
  • ensemble classification
  • k-nearest neighbor classification
  • novel class
  • silhouette coefficient.

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Classification and novel class detection in concept-drifting data streams under time constraints'. Together they form a unique fingerprint.

Cite this