Many Cyber-Physical Systems (CPS) at different scales are being developed for a variety of applications that provide valuable interactions between the cyber world and the physical systems and environments. However, the development of these systems face many challenges due to the complexity of the applications. An appropriate middleware is needed to provide infrastructural support and assist the development and operations of diverse CPS applications. This paper studies utilizing the service-oriented middleware (SOM) approach for CPS and discusses the advantages and requirements for such utilization. In addition, it proposes an SOM for CPS, called CPSWare. This middleware views all CPS components as a set of services and provides a service-based infrastructure to develop and operate CPS applications. This approach provides systemic solutions for solving many computing and networking issues in CPS. It also enables the integration of CPS with other systems such as Cloud and Fog Computing. Furthermore, as CPS can be developed for various applications at different scales, this paper provides a classification for CPS applications and discusses how CPSWare can effectively deal with the different issues in each category.