Development and optimization of parallel code for large-scale petroleum reservoir simulation

This paper discusses the use of large field-scale reservoir simulation to model multiphase fluid flow processes that occur in giant oil reservoirs. The goal of large-scale studies is to model fluid flow with sufficient details to account for reservoir heterogeneity, various reservoir-wellbore configurations, and complex fluid-rock interactions. In this paper, we developed a black-oil reservoir simulator for distributed-memory parallel environment. We ported serial code of a black-oil model to the CRAY T3E and IBM SP2 systems. We analysed the code and benchmarked the performance. To parallelize the code, we used a domain decomposition algorithm, whereby the reservoir is divided into several subdomains, with each subdomain assigned to a separate processor element (PE). The message-passing interface (MPI) is used to exchange information across subdomains. We validated the parallel simulator using data from the Society of Petroleum Engineers comparative solution projects. Because the linear equation solver accounts for over 90% of the CPU time in a typical reservoir simulation run, we evaluated the performance of several parallel algorithms in the project, including LSOR, Gauss-Siedel (GS) and strongly implicit procedure (SIP). We found that the convergence behaviour of the solver depends on the number of processors and on the permeability anisotropy of the reservoir. For the problems we tested, the SIP alogrithms provided the best performance. We compared the computational efficiency of the parallel code against the serial code using models containing up to 350,000 grid blocks in 4-, 8-, 16- 32-, 64-, and 80-PE environments. The paper discusses programming and computational performance issues in large-scale reservoir simulation for parallel systems.