The technological development is enabling the production of increasingly complex electronic systems. All such systems must be verified and tested to guarantee their correct behavior. As the complexity grows, testing has become one of the most significant factors that contribute to the total development cost. In recent years, we have also witnessed the inadequacy of the established testing methods, most of which are based on low-level representations of the hardware circuits. Therefore, more work has to be done at abstraction levels higher than the classical gate and register-transfer levels. At the same time, the automatic test equipment based solutions have failed to deliver the required test quality. As a result, alternative testing methods have been studied, which has led to the development of built-in self-test (BIST) techniques. In this thesis, we present a novel hybrid BIST technique that addresses several areas where classical BIST methods have shortcomings. The technique makes use of both pseudorandom and deterministic testing methods, and is devised in particular for testing modern systems-on-chip. One of the main contributions of this thesis is a set of optimization methods to reduce the hybrid test cost while not sacrificing test quality. We have devel oped several optimization algorithms for different hybrid BIST architectures and design constraints. In addition, we have developed hybrid BIST scheduling methods for an abort-on-first-fail strategy, and proposed a method for energy reduction for hybrid BIST. Devising an efficient BIST approach requires different design modifications, such as insertion of scan paths as well as test pattern generators and signature analyzers. These modifications require careful testability analysis of the original design. In the latter part of this thesis, we propose a novel hierarchical test generation algorithm that can be used not only for manufacturing tests but also for testability analysis. We have also investigated the possibilities of generating test vectors at the early stages of the design cycle, starting directly from the behavioral description and with limited knowledge about the final implementation. Experiments, based on benchmark examples and industrial designs, have been carried out to demonstrate the usefulness and efficiency of the proposed methodologies and techniques.