<p>In the system identification community a popular framework for the problem of estimating a parametrized model structure given a sequence of input and output pairs is given by the prediction-error method. This method tries to find the parameters which maximize the prediction capability of the corresponding model via the minimization of some chosen cost function that depends on the prediction error. This optimization problem is often quite complex with several local minima and is commonly solved using a local search algorithm. Thus, it is important to find a good initial estimate for the local search algorithm. This is the main topic of this thesis.</p><p>The first problem considered is the regressor selection problem for estimating the order of dynamical systems. The general problem formulation is difficult to solve and the worst case complexity equals the complexity of the exhaustive search of all possible combinations of regressors. To circumvent this complexity, we propose a relaxation of the general formulation as an extension of the nonnegative garrote regularization method. The proposed method provides means to order the regressors via their time lag and a novel algorithmic approach for the \textsc{arx} and \textsc{lpv-arx} case is given.</p><p> </p><p>Thereafter, the initialization of linear time-invariant polynomial models is considered. Usually, this problem is solved via some multi-step instrumental variables method. For the estimation of state-space models, which are closely related to the polynomial models via canonical forms, the state of the art estimation method is given by the subspace identification method. It turns out that this method can be easily extended to handle the estimation of polynomial models. The modifications are minor and only involve some intermediate calculations where already available tools can be used. Furthermore, with the proposed method other a priori information about the structure can be readily handled, including a certain class of linear gray-box structures. The proposed extension is not restricted to the discrete-time case and can be used to estimate continuous-time models.</p><p> </p><p>The final topic in this thesis is the initialization of discrete-time systems containing polynomial nonlinearities. In the continuous-time case, the tools of differential algebra, especially Ritt's algorithm, have been used to prove that such a model structure is globally identifiable if and only if it can be written as a linear regression model. In particular, this implies that once Ritt's algorithm has been used to rewrite the nonlinear model structure into a linear regression model, the parameter estimation problem becomes trivial. Motivated by the above and the fact that most system identification problems involve sampled data, a version of Ritt's algorithm for the discrete-time case is provided. This algorithm is closely related to the continuous-time version and enables the handling of noise signals without differentiations.</p>

In the system identification community a popular framework for the problem of estimating a parametrized model structure given a sequence of input and output pairs is given by the prediction-error method. This method tries to find the parameters which maximize the prediction capability of the corresponding model via the minimization of some chosen cost function that depends on the prediction error. This optimization problem is often quite complex with several local minima and is commonly solved using a local search algorithm. Thus, it is important to find a good initial estimate for the local search algorithm. This is the main topic of this thesis. The first problem considered is the regressor selection problem for estimating the order of dynamical systems. The general problem formulation is difficult to solve and the worst case complexity equals the complexity of the exhaustive search of all possible combinations of regressors. To circumvent this complexity, we propose a relaxation of the general formulation as an extension of the nonnegative garrote regularization method. The proposed method provides means to order the regressors via their time lag and a novel algorithmic approach for the \textsc{arx} and \textsc{lpv-arx} case is given.   Thereafter, the initialization of linear time-invariant polynomial models is considered. Usually, this problem is solved via some multi-step instrumental variables method. For the estimation of state-space models, which are closely related to the polynomial models via canonical forms, the state of the art estimation method is given by the subspace identification method. It turns out that this method can be easily extended to handle the estimation of polynomial models. The modifications are minor and only involve some intermediate calculations where already available tools can be used. Furthermore, with the proposed method other a priori information about the structure can be readily handled, including a certain class of linear gray-box structures. The proposed extension is not restricted to the discrete-time case and can be used to estimate continuous-time models.   The final topic in this thesis is the initialization of discrete-time systems containing polynomial nonlinearities. In the continuous-time case, the tools of differential algebra, especially Ritt's algorithm, have been used to prove that such a model structure is globally identifiable if and only if it can be written as a linear regression model. In particular, this implies that once Ritt's algorithm has been used to rewrite the nonlinear model structure into a linear regression model, the parameter estimation problem becomes trivial. Motivated by the above and the fact that most system identification problems involve sampled data, a version of Ritt's algorithm for the discrete-time case is provided. This algorithm is closely related to the continuous-time version and enables the handling of noise signals without differentiations.

Identification of nonlinear dynamical models of a black box nature involves both structure decisions (i.e., which regressors to use and the selection of a regressor function), and the estimation of the parameters involved. The typical approach in system identification is often a mix of all these steps, which for example means that the selection of regressors is based on the fits that is achieved for different choices. Alternatively one could then interpret the regressor selection as based on hypothesis tests (F-tests) at a certain confidence level that depends on the data. It would in many cases be desirable to decide which regressors to use, independently of the other steps. A survey of regressor selection methods used for linear regression and nonlinear identification problems is given. In this thesis we investigate what the well known method of analysis of variance (ANOVA) can offer for this problem. System identification applications violate many of the ideal conditions for which ANOVA was designed and we study how the method performs under such non-ideal conditions. It turns out that ANOVA gives better and more homogeneous results compared to several other regressor selection methods. Some practical aspects are discussed, especially how to categorise the data set for the use of ANOVA, and whether to balance the data set used for structure identification or not. An ANOVA-based method, Test of Interactions using Layout for Intermixed ANOVA (TILIA), for regressor selection in typical system identification problems with many candidate regressors is developed and tested with good performance on a variety of simulated and measured data sets. Typical system identification applications of ANOVA, such as guiding the choice of linear terms in the regression vector and the choice of regime variables in local linear models, are investigated. It is also shown that the ANOVA problem can be recast as an optimisation problem. Two modified, convex versions of the ANOVA optimisation problem are then proposed, and it turns out that they are closely related to the nn-garrote and wavelet shrinkage methods, respectively. In the case of balanced data, it is also shown that the methods have a nice orthogonality property in the sense that different groups of parameters can be computed independently.