This thesis investigates several design problems concerning two-channel conjugate quadrature (CQ) filter banks and orthogonal wavelets, as well as orthogonal cosine-modulated (OCM) filter banks. It is well known that optimal design of CQ filters and wavelets and optimal design of prototype filters (PFs) of OCM filter banks in the least squares (LS) or minimax sense are nonconvex problems and to date only local solutions can be claimed. In this thesis, we first make some improvements over several direct design techniques for local design problems in terms of convergence and solution accuracy. By virtue of the recent progress in global polynomial optimization and the improved local design methods mentioned above, we describe an attempt at developing several design strategies that may be viewed as our endeavors towards global solutions for LS CQ filter banks, minimax CQ filter banks, and OCM filter banks. In brief terms, the proposed design strategies are based on several observations made among globally optimal impulse responses of low-order filter banks, and are essentially order-recursive algorithms in terms of filter length combined with some techniques in identifying a desirable initial point in each round of iteration. This main idea is applied to three design scenarios in this thesis, namely, LS design of orthogonal filter banks and wavelets, minimax design of orthogonal filter banks and wavelets, and design of orthogonal cosine-modulated filter banks. Simulation studies are presented to evaluate and compare the performance of the proposed design methods with several well established algorithms in the literature.