Deep Background Subtraction with Guided Learning

Recently, convolutional neural networks (CNNs) have been applied in background subtraction (change detection) and gained notable improvements. Two typical methods have been proposed. The first one learns a specific CNN model for each video, but requires manual labeling of training frames on the fly. The other one learns a universal model offline, however, limits its performance in handling various surveillance scenarios. To address these problems, in this paper, a new deep background subtraction method is proposed by introducing a guided learning strategy. The main idea is to learn a specific CNN model for each video to ensure accuracy, but manage to avoid manual labeling. To achieve this, firstly we apply the SubSENSE algorithm [1] to get an initial segmentation, and then an adaptive strategy is designed to select reliable pixels to guide the CNN training. Besides, we also design a simple strategy to automatically select informative frames for guided learning. Experiments on the largest background subtraction benchmark CDnet2014 show that the proposed guided deep learning method outperforms existing state of the arts.