Operating batteries in parallel is attempted to overcome the problems with conventionally used battery bank, in which batteries are connected in series. The problems and the management with the operation of serial connected batteries are first addressed. The related topics to the parallel configuration are reviewed. Then, the parallel configuration with battery power modules is proposed. The battery power module can be realized with different dc-to-dc converters for different applications. When batteries are charged in parallel, the problem of over-charge can be avoided. With parallel operation, the discharging currents of the batteries are independently controlled but are coordinated to execute a full amount load current. This allows for scheduling the discharging profiles under different operating conditions. As a result, a sophisticated discharging profile can be realized to utilize the available stored energy in batteries. On the other hand, some of the batteries may take rest or be isolated from the system for the detections at a time. This facilitates the estimations of the state of charge (SOC) and the state of health (SOH). Moreover, the completely exhausted or damaged batteries can be isolated from the battery power supply bank without interrupting the system operation. Experiments are carried out on battery power modules with lead-acid batteries incorporating with associated buck-boost converters. The experimental results demonstrate that a more efficient utilization of battery energy can be achieved. On the other hand, a more reasonable management can be done with simple estimation methods of the SOC and the SOH.