The advances in data-collecting technologies provides great opportunities to access large sample-size data sets with high dimensionality. Variable selection is an important procedure to extract useful knowledge from such complex data. While in many real-data applications, appropriate selection of variables should facilitate the model interpretation and computation efficiency. It is thus important to incorporate domain knowledge of underlying data generation mechanism to select key variables for improving the model performance. However, general variable selection techniques, such as the best subset selection and the Lasso, often do not take the underlying data generation mechanism into considerations. This thesis proposal aims to develop statistical modeling methodologies with a focus on the structured variable selection towards better model interpretation and computation efficiency. Specifically, this thesis proposal consists of three parts: an additive heredity model with coefficients incorporating the multi-level data, a regularized dynamic generalized linear model with piecewise constant functional coefficients, and a structured variable selection method within the best subset selection framework. In Chapter 2, an additive heredity model is proposed for analyzing mixture-of-mixtures (MoM) experiments. The MoM experiment is different from the classical mixture experiment in that the mixture component in MoM experiments, known as the major component, is made up of sub-components, known as the minor components. The proposed model considers an additive structure to inherently connect the major components with the minor components. To enable a meaningful interpretation for the estimated model, we apply the hierarchical and heredity principles by using the nonnegative garrote technique for model selection. The performance of the additive heredity model was compared to several conventional methods in both unconstrained and constrained MoM experiments. The additive heredity model was then successfully applied in a real problem of optimizing the Pringlestextsuperscript{textregistered} potato crisp studied previously in the literature. In Chapter 3, we consider the dynamic effects of variables in the generalized linear model such as logistic regression. This work is motivated from the engineering problem with varying effects of process variables to product quality caused by equipment degradation. To address such challenge, we propose a penalized dynamic regression model which is flexible to estimate the dynamic coefficient structure. The proposed method considers modeling the functional coefficient parameter as piecewise constant functions. Specifically, under the penalized regression framework, the fused lasso penalty is adopted for detecting the changes in the dynamic coefficients. The group lasso penalty is applied to enable a sparse selection of variables. Moreover, an efficient parameter estimation algorithm is also developed based on alternating direction method of multipliers. The performance of the dynamic coefficient model is evaluated in numerical studies and three real-data examples. In Chapter 4, we develop a structured variable selection method within the best subset selection framework. In the literature, many techniques within the LASSO framework have been developed to address structured variable selection issues. However, less attention has been spent on structured best subset selection problems. In this work, we propose a sparse Ridge regression method to address structured variable selection issues. The key idea of the proposed method is to re-construct the regression matrix in the angle of experimental designs. We employ the estimation-maximization algorithm to formulate the best subset selection problem as an iterative linear integer optimization (LIO) problem. the mixed integer optimization algorithm as the selection step. We demonstrate the power of the proposed method in various structured variable selection problems. Moverover, the proposed method can be extended to the ridge penalized best subset selection problems. The performance of the proposed method is evaluated in numerical studies. / Doctor of Philosophy / The advances in data-collecting technologies provides great opportunities to access large sample-size data sets with high dimensionality. Variable selection is an important procedure to extract useful knowledge from such complex data. While in many real-data applications, appropriate selection of variables should facilitate the model interpretation and computation efficiency. It is thus important to incorporate domain knowledge of underlying data generation mechanism to select key variables for improving the model performance. However, general variable selection techniques often do not take the underlying data generation mechanism into considerations. This thesis proposal aims to develop statistical modeling methodologies with a focus on the structured variable selection towards better model interpretation and computation efficiency. The proposed approaches have been applied to real-world problems to demonstrate their model performance.

The number of technology alternatives has lately grown to satisfy the increasingly demanding goals in modern engineering. These technology alternatives are handled in the design process as either concepts or categorical design inputs. Additionally, designers desire to bring into early design more and more accurate, but also computationally burdensome, simulation tools to obtain better performing initial designs that are more valuable in subsequent design stages. It constrains the computational budget to optimize the design space. These two factors unveil the need of a conceptual design methodology to use more efficiently sophisticated tools for engineering problems with several concept solutions and categorical design choices. Enhanced initial designs and discrete alternative selection are pursued. Advances in computational speed and the development of Bayesian adaptive sampling techniques have enabled the industry to move from the use of look-up tables and simplified models to complex physics-based tools in conceptual design. These techniques focus computational resources on promising design areas. Nevertheless, the vast majority of the work has been done on problems with continuous spaces, whereas concepts and categories are treated independently. However, observations show that engineering objectives experience similar topographical trends across many engineering alternatives. In order to address these challenges, two meta-models are developed. The first one borrows the Hamming distance and function space norms from machine learning and functional analysis, respectively. These distances allow defining categorical metrics that are used to build an unique probabilistic surrogate whose domain includes, not only continuous and integer variables, but also categorical ones. The second meta-model is based on a multi-fidelity approach that enhances a concept prediction with previous concept observations. These methodologies leverage similar trends seen from observations and make a better use of sample points increasing the quality of the output in the discrete alternative selection and initial designs for a given analysis budget. An extension of stochastic mixed-integer optimization techniques to include the categorical dimension is developed by adding appropriate generation, mutation, and crossover operators. The resulted stochastic algorithm is employed to adaptively sample mixed-integer-categorical design spaces. The proposed surrogates are compared against traditional independent methods for a set of canonical problems and a physics-based rotor-craft model on a screened design space. Next, adaptive sampling algorithms on the developed surrogates are applied to the same problems. These tests provide evidence of the merit of the proposed methodologies. Finally, a multi-objective rotor-craft design application is performed in a large domain space. This thesis provides several novel academic contributions. The first contribution is the development of new efficient surrogates for systems with categorical design choices. Secondly, an adaptive sampling algorithm is proposed for systems with mixed-integer-categorical design spaces. Finally, previously sampled concepts can be brought to construct efficient surrogates of novel concepts. With engineering judgment, design community could apply these contributions to discrete alternative selection and initial design assessment when similar topographical trends are observed across different categories and/or concepts. Also, it could be crucial to overcome the current cost of carrying a set of concepts and wider design spaces in the categorical dimension forward into preliminary design.

La conception d'un système électrique est une tâche très complexe qui relève d’expertises dans différents domaines de compétence. Dans un contexte compétitif où l’avance technologique est un facteur déterminant, l’industrie cherche à réduire les temps d'étude et à fiabiliser les solutions trouvées par une approche méthodologique rigoureuse fournissant une solution optimale systémique.Il est alors nécessaire de construire des modèles et de mettre au point des méthodes d'optimisation compatibles avec ces préoccupations. En effet, l’optimisation unitaire de sous-systèmes sans prendre en compte les interactions ne permet pas d'obtenir un système optimal. Plus le système est complexe plus le travail est difficile et le temps de développement est important car il est difficile pour le concepteur d'appréhender le système dans toute sa globalité. Il est donc nécessaire d'intégrer la conception des composants dans une démarche systémique et globale qui prenne en compte à la fois les spécificités d’un composant et ses relations avec le système qui l’emploie.Analytical Target Cascading est une méthode d'optimisation multi niveaux de systèmes complexes. Cette approche hiérarchique consiste à décomposer un système complexe en sous-systèmes, jusqu’au niveau composant dont la conception relève d’algorithmes d'optimisation classiques. La solution optimale est alors trouvée par une technique de coordination qui assure la cohérence de tous les sous-systèmes. Une première partie est consacrée à l'optimisation de composants électriques. L'optimisation multi niveaux de systèmes complexes est étudiée dans la deuxième partie où une chaîne de traction électrique est choisie comme exemple / The design of an electrical system is a very complex task which needs experts from various fields of competence. In a competitive environment, where technological advance is a key factor, industry seeks to reduce study time and to make solutions reliable by way of a rigorous methodology providing a systemic solution.Then, it is necessary to build models and to develop optimization methods which are suitable with these concerns. Indeed, the optimization of sub-systems without taking into account the interaction does not allow to achieve an optimal system. More complex the system is more the work is difficult and the development time is important because it is difficult for the designer to understand and deal with the system in its complexity. Therefore, it is necessary to integrate the design components in a systemic and holistic approach to take into account, in the same time, the characteristics of a component and its relationship with the system it belongs to.Analytical Target Cascading is a multi-level optimization method for handling complex systems. This hierarchical approach consists on the breaking-down of a complex system into sub-systems, and component where their optimal design is ensured by way of classical optimization algorithms. The optimal solution of the system must be composed of the component's solutions. Then a coordination strategy is needed to ensure consistency of all sub-systems. First, the studied and proposed optimization algorithms are tested and compared on the optimization of electrical components. The second part focuses on the multi-level optimization of complex systems. The optimization of railway traction system is taken as a test case

[pt] O Partial Digest Problem (problema de digestão parcial), também conhecido como o Turnpike Problem, consiste na construção de um conjunto de pontos na reta real dadas as distâncias não designadas entre todos os pares de pontos. Uma variante deste problema, chamada Min Distance Superset Problem (problema de superset de distância mínimo), lida com entradas incompletas em que algumas distâncias podem estar faltando. O objetivo deste problema é encontrar um conjunto mínimo de pontos na reta real, tal que as distâncias entre cada par de pontos contenham todas as distâncias de entrada. As principais contribuições deste trabalho são duas formulações de programação matemática diferentes para o Min Distance Superset Problem: uma formulação de programação quadrática e uma formulação de programação inteira. Mostramos como aplicar um método de cálculo direto de limites de valores de variáveis através de uma relaxação Lagrangeana da formulação quadrática. Também introduzimos duas abordagens diferentes para resolver a formulação inteira, ambas baseadas em buscas binárias na cardinalidade de uma solução ótima. A primeira baseia-se num subconjunto de variáveis de decisão, na tentativa de lidar com um problema de viabilidade mais simples, e o segundo é baseado na distribuição de distâncias entre possíveis pontos disponíveis. / [en] The Partial Digest Problem, also known as the Turnpike Problem, consists of building a set of points on the real line given their unlabeled pairwise distances. A variant of this problem, named Min Distance Superset Problem, deals with incomplete input in which distances may be missing. The goal is to find a minimal set of points on the real line such that the multiset of their pairwise distances is a superset of the input. The main contributions of this work are two different mathematical programming formulations for the Min Distance Superset Problem: a quadratic programming formulation and an integer programming formulation.We show how to apply direct computation methods for variable bounds on top of a Lagrangian relaxation of the quadratic formulation. We also introduce two approaches to solve the integer programming formulation, both based on binary searches on the cardinality of an optimal solution. One is based on a subset of decision variables, in an attempt to deal with a simpler feasibility problem, and the other is based on distributing available distances between possible points.

[pt] Este trabalho apresenta um método de Monitoramento Não Intrusivo de Carga de Aparelhos elétricos (do inglês Non-Intrusive Appliance Load Monitoring – NIALM) supervisionado, usando técnicas de análise de estados estacionários, para desagregação do consumo elétrico residencial a partir de uma única medição, sem a necessidade de instalação de medidores individuais nos dispositivos. A metodologia proposta divide o problema em duas etapas. Inicialmente, há um pré-processamento para identificação e desagregação dos aparelhos que apresentam comportamento periódico, modelados a partir da estimação dos parâmetros da série de Fourier. Na etapa seguinte, os resultados obtidos são combinados a um modelo de otimização linear-inteiro misto para desagregação dos equipamentos não-periódicos, buscando minimizar a diferença entre a curva de carga total lida e a soma das curvas de carga desagregadas por dispositivo. Uma aplicação didática é realizada para validação do método proposto com dados reais e, por fim, é apresentada uma análise de viabilidade econômica da migração para a tarifa branca aplicada no Brasil. Os resultados mostram que, ao utilizar dessa metodologia, é possível que o usuário avalie se há ou não vantagem em deslocar parte do seu consumo de energia para fora do horário de ponta para obter benefício na sua fatura de energia elétrica. / [en] This work presents a supervised Non-Intrusive Appliance Load Monitoring (NILM) method, or energy disaggregation, for residential consumption, which aims to decompose the aggregate energy consumption data collected from a single measurement point into device-level consumption estimation using steady state analysis techniques with no need to install individual meters on appliances. The proposed methodology considers two steps to face the problem. Firstly, periodical appliances are modeled from the estimation of Fourier series parameters and extracted from the total power measured. Secondly, the results obtained are combined with a Mixed Integer Linear Programming proposed to disaggregate the remaining appliances, which minimize the difference between the total aggregated load and the sum of the estimated load curves per appliance. A study case is performed with a real case to validate the proposed method and indicates that the model can be useful for practical applications, such as helping evaluate the possibility of the consumers changing the modality of their tariff contract from the conventional tariff to the new Brazilian modality called white tariff.