In order to solve the membrane scaling and fouling problem of a multi-stage reverse osmosis (RO) system in the zero discharge process of high salinity coal mine water, the multi-stage reverse osmosis (RO) system was optimized based on computational fluid dynamics (CFD) simulation. An optimization scheme was proposed regarding the arrangement and combination of membrane elements in two-stage membrane concentration unit. Technical analysis and economic analysis were carried out. The results show that, after optimization, the distribution of ion pollution in the membrane element and the dosage of scale inhibitors remain the same. Six membrane elements are reduced in each of the primary membrane concentration unit and secondary membrane concentration unit, which accounting for approximately 5% of the whole membranes elements of the multi-stage reverse osmosis (RO) system. The total initial investment of membrane elements is saved by 48 000 yuan. The recovery rate of coal mine water increases from 96.5% to 97%. The power cost rises by 0.017 yuan/m3, while the evaporation and crystallization cost is reduced by 0.395 yuan/m3. Therefore, the total operation cost is saved by 0.379 yuan/m3.