Yes / Age and gender classification are two important problems that recently gained popularity in the research community, due to their wide range of applications. Research has shown that both age and gender information are encoded in the face shape and texture, hence the active appearance model (AAM), a statistical model that captures shape and texture variations, has been one of the most widely used feature extraction techniques for the aforementioned problems. However, AAM suffers from some drawbacks, especially when used for classification. This is primarily because principal component analysis (PCA), which is at the core of the model, works in an unsupervised manner, i.e., PCA dimensionality reduction does not take into account how the predictor variables relate to the response (class labels). Rather, it explores only the underlying structure of the predictor variables, thus, it is no surprise if PCA discards valuable parts of the data that represent discriminatory features. Toward this end, we propose a supervised appearance model (sAM) that improves on AAM by replacing PCA with partial least-squares regression. This feature extraction technique is then used for the problems of age and gender classification. Our experiments show that sAM has better predictive power than the conventional AAM.

Yes / Optimization plays a pivotal role in computer graphics and vision. Learning-based optimization algorithms have emerged as a powerful optimization technique for solving problems with robustness and accuracy because it learns gradients from data without calculating the Jacobian and Hessian matrices. The key aspect of the algorithms is the least-squares method, which formulates a general parametrized model of unconstrained optimizations and makes a residual vector approach to zeros to approximate a solution. The method may suffer from undesirable local optima for many applications, especially for point cloud registration, where each element of transformation vectors has a different impact on registration. In this paper, Reweighted Discriminative Optimization (RDO) method is proposed. By assigning different weights to components of the parameter vector, RDO explores the impact of each component and the asymmetrical contributions of the components on fitting results. The weights of parameter vectors are adjusted according to the characteristics of the mean square error of fitting results over the parameter vector space at per iteration. Theoretical analysis for the convergence of RDO is provided, and the benefits of RDO are demonstrated with tasks of 3D point cloud registrations and multi-views stitching. The experimental results show that RDO outperforms state-of-the-art registration methods in terms of accuracy and robustness to perturbations and achieves further improvement than non-weighting learning-based optimization.

We study the Feasible Generalized Least-Squares (FGLS) estimation of the parameters of a linear regression model in which the errors are allowed to exhibit heteroskedasticity of unknown form and to be serially correlated. The main contribution is two fold; first we aim to demystify the reasons often advanced to use OLS instead of FGLS by showing that the latter estimate is robust, and more efficient and precise. Second, we devise consistent FGLS procedures, robust to misspecification, which achieves a lower mean squared error (MSE), often close to that of the correctly specified infeasible GLS. In the first chapter we restrict our attention to the case with independent heteroskedastic errors. We suggest a Lasso based procedure to estimate the skedastic function of the residuals. This estimate is then used to construct a FGLS estimator. Using extensive Monte Carlo simulations, we show that this Lasso-based FGLS procedure has better finite sample properties than OLS and other linear regression-based FGLS estimates. Moreover, the FGLS-Lasso estimate is robust to misspecification of both the functional form and the variables characterizing the skedastic function. The second chapter generalizes our investigation to the case with serially correlated errors. There are three main contributions; first we show that GLS is consistent requiring only pre-determined regressors, whereas OLS requires exogenous regressors to be consistent. The second contribution is to show that GLS is much more robust that OLS; even a misspecified GLS correction can achieve a lower MSE than OLS. The third contribution is to devise a FGLS procedure valid whether or not the regressors are exogenous, which achieves a MSE close to that of the correctly specified infeasible GLS. Extensive Monte Carlo experiments are conducted to assess the performance of our FGLS procedure against OLS in finite samples. FGLS achieves important reductions in MSE and variance relative to OLS. In the third chapter we consider an empirical application; we re-examine the Uncovered Interest Parity (UIP) hypothesis, which states that the expected rate of return to speculation in the forward foreign exchange market is zero. We extend the FGLS procedure to a setting in which lagged dependent variables are included as regressors. We thus provide a consistent and efficient framework to estimate the parameters of a general k-step-ahead linear forecasting equation. Finally, we apply our FGLS procedures to the analysis of the two main specifications to test the UIP.

This dissertation proposes an iterative confidence passing (ICP) approach for parameter estimation. The dissertation describes three different algorithms that follow from this ICP approach. These three variations of the ICP approach are applied to (a) macrodiversity and user cooperation diversity reception problems, (b) the co-operative multipoint MIMO reception problem (pertinent to the LTE Advanced system scenarios), and (c) the satellite beamforming problem. The first two of these three applications are some of the significant open DSP research problems that are currently being actively pursued in academia and industry. This dissertation demonstrates a significant performance improvement that the proposed ICP approach delivers compared to the existing known techniques. The proposed ICP approach jointly estimates (and, thereby, separates) two sets of unknown parameters from the receiver measurements. For applications (a) and (b) mentioned above, one set of unknowns is comprised of the discrete-valued information-bearing transmitted symbols in a multi-channel communication system, and the other set of unknown parameters is formed by the coefficients of a Rayleigh or Rician fading channel. Application (a) is for interference-free, cooperative or macro, transmit or receive, diversity scenarios. Application (b) is for MIMO systems with interference-rich reception. Finally, application (c) is for an interference-free spacecraft array calibration system model in which both the sets of unknowns are complex continuous valued variables whose magnitude follows the Rician distribution. The algorithm described here is the outcome of an investigation for solving a difficult channel estimation problem. The difficulty of the estimation problem arises because (i) the channel of interest is intermittently observed, and (ii) the partially observed information is not directly of the channel of interest; it has dependency on another unknown and uncorrelated set of complex-valued random variables. The proposed ICP algorithmic approach for solving the above estimation problems is based on an iterative application of the Weighted Least Squares (WLS) method. The main novelty of the proposed algorithm is a back and forth exchange of the confidence or the belief values in the WLS estimates of the unknown parameters during the algorithm iterations. The confidence values of the previously obtained estimates are used to derive the estimation weights at the next iteration, which generates an improved estimate with a greater confidence. This method of iterative confidence (or belief) passing causes a bootstrapping convergence to the parameter estimates. Besides the ICP approach, several alternatives are considered to solve the above problems (a, b and c). Results of the performance simulation of the alternative methods show that the ICP algorithm outperforms all the other candidate approaches. Performance benefit is significant when the measurements (and the initial seed estimates) have non-uniform quality, e.g., when many of the measurements are either non-usable (e.g., due to shadowing or blockage) or are missing (e.g., due to instrument failures). / Ph. D.

Self-excited systems are open loop unstable plants having a nonlinearity that prevents an exponentially increasing time response. The resulting limit cycle is induced by any slight disturbance that causes the response of the system to grow to the saturation level of the nonlinearity. Because there is no external disturbance, control of these self-excited systems requires that the open loop system dynamics are altered so that any unstable open loop poles are stabilized in the closed loop. This work examines a variety of adaptive control approaches for controlling a thermoacoustic instability, a physical self-excited system. Initially, a static feedback controller loopshaping design and associated system identification method is presented. This design approach is shown to effectively stabilize an unstable Rijke tube combustor while preventing the creation of additional controller induced instabilities. The loopshaping design method is then used in conjunction with a trained artificial neural network to demonstrate stabilizing control in the presence of changing plant dynamics over a wide variety of operating conditions. However, because the ANN is designed specifically for a single combustor/actuator arrangement, its limited portability is a distinct disadvantage. Filtered-X least mean squares (LMS) adaptive feedback control approaches are examined when applied to both stable and unstable plants. An identification method for approximating the relevant plant dynamics to be modeled is proposed and shown to effectively stabilize the self-excited system in simulations and experiments. The adaptive feedback controller is further analyzed for robust performance when applied to the stable, disturbance rejection control problem. It is shown that robust stability cannot be guaranteed because arbitrarily small errors in the plant model can generate gradient divergence and unstable feedback loops. Finally, a time-averaged-gradient (TAG) algorithm is investigated for use in controlling self-excited systems such as the thermoacoustic instability. The TAG algorithm is shown to be very effective in stabilizing the unstable dynamics using a variety of controller parameterizations, without the need for plant estimation information from the system to be controlled. / Ph. D.

A class of immersed finite element (IFE) spaces is developed for solving elliptic boundary value problems that have interfaces. IFE spaces are finite element approximation spaces which are based upon meshes that can be independent of interfaces in the domain. Three different quadratic IFE spaces and their related biquadratic IFE spaces are introduced here for the purposes of solving both forward and inverse elliptic interface problems in 1D and 2D. These different spaces are constructed by (i) using a hierarchical approach, (ii) imposing extra continuity requirements or (iii) using a local refinement technique. The interpolation properties of each space are tested against appropriate testing functions in 1D and 2D. The IFE spaces are also used to approximate the solution of a forward elliptic interface problem using the Galerkin finite element method and the mixed least squares finite element method. Finally, one appropriate space is selected to solve an inverse interface problem using either an output least squares approach or the least squares with mixed equation error method. / Ph. D.

Individuals are considered the weakest link when it comes to securing a personal computer system. All the technological solutions can be in place, but if individuals do not make appropriate security protection decisions they introduce holes that technological solutions cannot protect. This study investigates what personal characteristics influence differences in individual security behaviors, defined as behaviors to protect against security threats, by adapting Protection Motivation Theory into an information security context. This study developed and validated an instrument to measure individual security behaviors. It then tested the differences in these behaviors using the security research model, which built from Protection Motivation Theory, and consisted of perceived security vulnerability, perceived security threat, security self-efficacy, response efficacy, and protection cost. Participants, representing a sample population of home computer users with ages ranging from 20 to 83, provided 279 valid responses to surveys. The behaviors studied include using anti-virus software, utilizing access controls, backing up data, changing passwords frequently, securing access to personal computers, running software updates, securing wireless networks, using care when storing credit card information, educating others in one's house about security behaviors, using caution when following links in emails, running spyware software, updating a computer's operating system, using firewalls, and using pop-up blocking software. Testing the security research model found different characteristics had different impacts depending on the behavior studied. Implications for information security researchers and practitioners are provided, along with ideas for future research. / Ph. D.

Product reliability is an important characteristic for all manufacturers, engineers and consumers. Industrial statisticians have been planning experiments for years to improve product quality and reliability. However, rarely do experts in the field of reliability have expertise in design of experiments (DOE) and the implications that experimental protocol have on data analysis. Additionally, statisticians who focus on DOE rarely work with reliability data. As a result, analysis methods for lifetime data for experimental designs that are more complex than a completely randomized design are extremely limited. This dissertation provides two new analysis methods for reliability data from life tests. We focus on data from a sub-sampling experimental design. The new analysis methods are illustrated on a popular reliability data set, which contains sub-sampling. Monte Carlo simulation studies evaluate the capabilities of the new modeling methods. Additionally, Monte Carlo simulation studies highlight the principles of experimental design in a reliability context. The dissertation provides multiple methods for statistical inference for the new analysis methods. Finally, implications for the reliability field are discussed, especially in future applications of the new analysis methods. / Ph. D.

The central goal of this work is to define and analyze least squares finite element methods for the Stokes and Navier-Stokes equations that are practical and optimal in a systematic and rigorous way. To accomplish this task we begin by developing the least squares theory for the linear Stokes equations. We introduce least squares methods based on the minimization of functionals that involve residuals of the equations of an equivalent first order formulation for the Stokes problem. We show that for the Stokes equations there are two general types of boundary conditions. For the first type, practical least squares methods can be defined and analyzed in a fairly standard way, based on application of the Agmon, Douglis and Nirenberg a priori estimates. For the second type of boundary conditions this task is more difficult and involves mesh dependent (weighted) least squares functionals. Among the main results are the optimal error estimates for the weighted least squares method in two and three space dimensions. Then, we formulate two least squares methods for the nonlinear Navier-Stokes equations written as a first order system. We consider the first method as a conforming discretization of an abstract nonlinear problem and the second weighted one, which is more practical, as a nonconforming discretization of the same abstract problem. As a result, the analysis of the first method fits into the framework of the approximation theory of Brezzi, Rappaz and Raviart and the analysis of the second does not. Thus, we develop an abstract approximation theory that is suitable for nonconforming discretizations of the abstract problem. The central result is based on the application of our abstract theory to the weighted least squares method. We prove that this method results in optimally accurate approximations for the Navier-Stokes equations. We believe that these error analyses of Chapter are the first treatment of a least squares formulation for a nonlinear problem in the current literature. We then discuss various implementation issues, including theoretical and numerical estimates of condition numbers and the presentation of numerical examples. In particular, we study the numerical convergence rates of various implementations of least squares methods and demonstrate that the weights are necessary for the optimal rates to hold. Finally, we compare numerical results for the driven cavity flow problem with some benchmark results reported in the literature. / Ph. D.

This dissertation presents a set of general numerical tools for simulating feedforward active noise control in the frequency domain. Feedforward control is numerically similar to linear least squares regression, and can take advantage of various numerical techniques developed in the statistics literature for use with regression. Therefore, an important theme of this work is to look at the control problem from a statistical point of view, and explore the analogies between feedforward control and basic statistical principles of regression. Motivating the numerical approach is the need to simulate active noise control for systems whose dynamics must be modeled numerically because analytical solutions do not exist, e.g., fluid-structure interaction problems. Plant dynamics for examples in the present work are modeled using a finite-element / boundary-element computer program, and the associated numerical methods are general enough for us with many types of problems. The derivation is presented in the context of active structural-acoustic control (ASAC), in which sound radiating from a vibrating structure is controlled by applying time-harmonic vibrational inputs directly on the structure. First, a feedforward control simulation is developed for a submerged spherical shell using both analytical and numerical techniques; the numerical formulation is found by discretizing the integrations used in the analytical approach. ASAC is shown to be effective for controlling radiation from the spherical shell. For a point-force disturbance at low frequencies, a single control input can reduce the radiated power by up to 20 dB (ignoring the possibility of measurement noise). A more general numerical methodology is then developed based on weighted least-squares regression in the complex domain. It is shown that basic regression diagnostics, which are used in the statistics literature to describe the quality and reliability of a regression, can be used to model the effects of error sensor measurement noise to produce a more realistic simulation. Numerical results are presented for a finite-length, fluid-loaded cylindrical shell with clamped, rigid end closures. It is shown that when the controller reduces the radiated power by less than 2 dB, the control simulation is usually invalid for statistical reasons. Also developed are confidence intervals for the individual control input magnitudes, and prediction intervals which help evaluate the sensitivity to measurement noise for the regression as a whole. Collinearity, a type of numerical ill-conditioning that can corrupt regression results, is demonstrated to occur in an example feedforward control simulation. The effects of collinearity are discussed, and a basic diagnostic is developed to detect and analyze collinearity. Subset selection, a numerical procedure for improving regressions, is shown to correspond to optimizing actuator locations for best control system performance. Exhaustive-search subset selection is used to optimize actuator locations for a sample structure. Finally, a convenient method is given for investigating alternate controller formulations, and examples of several alternate controllers are given including a wavenumber-domain controller. Numerical results for a cylindrical shell give insight to the mechanisms used by the control system, and a new visualization technique is used to relate farfield pressure distributions to surface velocity distributions using wavenumber analysis. / Ph. D.