The main objective of this research is to develop a computational design tool for multilevel optimization of product-material systems under uncertainty. To accomplish this goal, an exponential penalty function (EPF) formulation based on method of multipliers is developed for solving multilevel optimization problems within the framework of Analytical Target Cascading (ATC). The original all-at-once constrained optimization problem is decomposed into a hierarchical system with consistency constraints enforcing the target-response coupling in the connected elements. The objective function is combined with the consistency constraints in each element to formulate an augmented Lagrangian with EPF. The EPF formulation is implemented using double-loop (EPF I) and single-loop (EPF II) coordination strategies and two penalty-parameter-updating schemes. The computational characteristics of the proposed approaches are investigated using different nonlinear convex and non-convex optimization problems. An efficient reliability-based design optimization method, Single Loop Single Vector (SLSV), is integrated with Augmented Lagrangian (AL) formulation of ATC for solution of hierarchical multilevel optimization problems under uncertainty. In the proposed SLSV+AL approach, the uncertainties are propagated by matching the required moments of connecting responses/targets and linking variables present in the decomposed system. The accuracy and computational efficiency of SLSV+AL are demonstrated through the solution of different benchmark problems and comparison of results with those from other optimization methods. Finally, the developed computational design optimization tool is used for design optimization of hybrid multiscale composite sandwich plates with/without uncertainty. Both carbon nanofiber (CNF) waviness and CNF-matrix interphase properties are included in the model. By decomposing the sandwich plate, structural and material designs are combined and treated as a multilevel optimization problem. The application problem considers the minimum-weight design of an in-plane loaded sandwich plate with a honeycomb core and laminated composite face sheets that are reinforced by both conventional continuous fibers and CNF-enhanced polymer matrix. Besides global buckling, shear crimping, intracell buckling, and face sheet wrinkling are also treated as design constraints.