In civil infrastructures such as buildings, bridges, and tunnels, cracks are initial signs of degradation, which affect the structure's current and future performance adversely. Optimum maintenance plans in terms of cost and safety are important to evaluate the degree of deterioration of a structure. Manual inspection is usually performed, and cracks detected during inspections could help the inspectors to understand the damaged state of the concrete structures. However, these inspections are costly, laborious, and easily prone to human error. An automatic and fast crack detection at the earliest stage is crucial to avoid further degradation of the structure. In the past decades, various deep learning techniques have been introduced by researchers to automate the crack detection task. This paper introduces a deep learning-based multi-model ensemble approach for crack detection in concrete structures. The proposed architecture consists of five different customized convolutional neural networks (CNN) trained on data set created from two public datasets. The dataset consists of 8400 crack and non-crack images having a resolution of 224 * 224. Detailed experiments show that the majority voting ensemble technique shows better performance for crack detection in concrete structures. The accuracy of the individual CNN models 1, 2, 3, and 4 is recorded to be 95%,96%, 95%, and 97%, respectively, while the accuracy of the ensemble techniques is recorded to be 98%.