In recent years, there has been a growing attention to develop a Human-like Robot controller that hopes to move the robots closer to face real world applications. Several approaches have been proposed to support the learning phase in such a controller, such as learning through observation and\or a direct guidance from the user. These approaches, however, require incremental learning and memorizing techniques, where the robot can design its internal system and keep retraining it overtime. This study, therefore, investigates a new idea to develop incremental learning and memory model, we called, a Hierarchical Constructive BackPropagation with Memory (HCBPM). The validity of the model was tested in teaching a robot a group of names (colors). The experimental results indicate the efficiency of the model to build a social learning environment between the user and the robot. The robot could learn various color names and its different phases, and retrieve these data easily to teach another user what it had learned.