Crowd management in urban settings has mostly relied on either classical, non-automated mechanisms or spontaneous notifications/alerts through social networks. Such management techniques are heavily marred by lack of comprehensive control, especially in terms of averting risks in a manner that ensures crowd safety and enables prompt emergency response. In this paper, we propose a Markov Decision Process Scheme MDP to realize a smart infrastructure that is directly aimed at crowd management. A key emphasis of the scheme is a robust and reliable scalability that provides sufficient flexibility to manage a mixed crowd (i.e., pedestrian, cyclers, manned vehicles and unmanned vehicles). The infrastructure also spans various population settings (e.g., roads, buildings, game arenas, etc.). To realize a reliable and scalable crowd management scheme, the classical MDP is decomposed into Local MDPs with smaller action-state spaces. Preliminarily results show that the MDP decomposition can reduce the system global cost and facilitate fast convergence to local near-optimal solution for each L-MDP.