The optimization of a reverse osmosis (RO) system includes optimization of the design of the individual membrane modules, the system structure and the operating conditions of the system. Most previous studies considered either the optimal design of individual modules only or optimization of system structure and operating conditions for fixed module dimensions. This thesis developed a method to simultaneously optimize the module dimensions, system structure and operating conditions. The method comprised rules for generating a general superstructure for an RO system given the number of modules along with rules for generating technically and mathematically feasible sub-structures. The superstructure was based on maximum connectivity between unit operations. A connectivity matrix was used to represent the superstructure. The matrix was useful for checking sub-structure's feasibility and deriving a model for the sub-structure's optimization, comprising the minimum number of variables and constraints which minimized computational time and increased accuracy. For optimization, a nonlinear objective function of the annualized profit of the RO system was formulated, consisting of the revenue obtained from permeate sales, capital costs of the unit operations and operating costs for the system. It was found that RO system optimization is a nonconvex optimization problem. The most effective optimization procedure involved a combination of evolutionary computation, which was good for locating the global optimum, and a gradient-based method, which was superior in finding the exact optimum. Small population size, adaptive mutation rate and steady state replacement were the most efficient parameter settings for the evolutionary computation. Optimal design of two-stage RO systems with and without energy recovery, bypass and recycle streams was studied. Dimensions of predicted optimal modules approached those of current commercial modules but with much shorter feed channels. The mathematical optimum also had higher operating pressures. The optimum system structure was a series arrangement with different module dimensions in each stage. A sensitivity analysis showed that trends in the optimal design were similar when unit costs changed. An investigation of the scalability of the method for a three-stage RO system revealed several weaknesses. These are probably surmountable with the addition of more RO system specific knowledge.