Ensembles methods such as AdaBoost, Bagging and Random Forest have attracted much attention in the statistical learning community in the last 15 years. Zhu and Chipman (2006) proposed the idea of using ensembles for variable selection. Their implementation used a parallel genetic algorithm (PGA). In this thesis, I propose a stochastic stepwise ensemble for variable selection, which improves upon PGA. Traditional stepwise regression (Efroymson 1960) combines forward and backward selection. One step of forward selection is followed by one step of backward selection. In the forward step, each variable other than those already included is added to the current model, one at a time, and the one that can best improve the objective function is retained. In the backward step, each variable already included is deleted from the current model, one at a time, and the one that can best improve the objective function is discarded. The algorithm continues until no improvement can be made by either the forward or the backward step. Instead of adding or deleting one variable at a time, Stochastic Stepwise Algorithm (STST) adds or deletes a group of variables at a time, where the group size is randomly decided. In traditional stepwise, the group size is one and each candidate variable is assessed. When the group size is larger than one, as is often the case for STST, the total number of variable groups can be quite large. Instead of evaluating all possible groups, only a few randomly selected groups are assessed and the best one is chosen. From a methodological point of view, the improvement of STST ensemble over PGA is due to the use of a more structured way to construct the ensemble; this allows us to better control over the strength-diversity tradeoff established by Breiman (2001). In fact, there is no mechanism to control this fundamental tradeoff in PGA. Empirically, the improvement is most prominent when a true variable in the model has a relatively small coefficient (relative to other true variables). I show empirically that PGA has a much higher probability of missing that variable.

The work in this thesis is concerned with the mechanics of the initiation and growth of small fatigue cracks from notches under service load histories. Fatigue life estimates for components subjected to variable amplitude service loading are usually based on the same constant amplitude strain-life data used for constant amplitude fatigue life predictions. The resulting fatigue life estimates although they are accurate for constant amplitude fatigue, are always non conservative for variable amplitude load histories. Similarly fatigue life predictions based on small crack growth calculations for cracks growing from flaws in notches are non conservative when constant amplitude crack growth data are used. These non conservative predictions have, in both cases, been shown to be due to severe reductions in fatigue crack closure arising from large (overload or underload) cycles in a typical service load history. Smaller load cycles following a large near yield stress overload or underload cycle experience a much lower crack opening stress than that experienced by the same cycles in the reference constant amplitude fatigue tests used to produce design data. This reduced crack opening stress results in the crack remaining open for a larger fraction of the stress-strain cycle and thus an increase in the effective portion of the stress-strain cycle. The effective strain range is increased and the fatigue damage for the small cycles is greater than that calculated resulting in a non conservative fatigue life prediction. Previous work at Waterloo introduced parameters based on effective strain-life fatigue data and effective stress intensity versus crack growth rate data. Fatigue life calculations using these parameters combined with experimentally derived crack opening stress estimates give accurate fatigue life predictions for notched components subjected to variable amplitude service load histories. Information concerning steady state crack closure stresses, effective strain-life data, and effective stress intensity versus small crack growth rate data, are all obtained from relatively simple and inexpensive fatigue tests of smooth specimens in which periodic underloads are inserted into an otherwise constant amplitude load history. The data required to calibrate a variable amplitude fatigue crack closure model however, come from time consuming measurements of the return of crack closure levels for small cracks to a steady state level following an underload (large cracks for which crack closure measurements are easier to make cannot be used because at the high stress levels in notches under service loads a test specimen used would fracture). For low and moderately high hardness levels in metals crack growth and crack opening stress measurements have been made using a 900x optical microscope for the small crack length at which a test specimen can resist the high stress levels encountered when small cracks grow from notches. For very hard metals the crack sizes may be so small that the measurements must be made using a confocal scanning laser microscope. In this case the specimen must be removed from the test machine for each measurement and the time to acquire data is only practical for an extended research project. The parameters for the crack closure model relating to steady state crack closure levels vary with material cyclic deformation resistance which in turn increases with hardness. One previous investigation found that the steady state crack opening level was lower and the recovery to a steady state crack opening stress level after an underload was more rapid for a hard than for a soft metal. This observation can be explained by the dependence of the crack tip plastic zone size that determines crack tip deformation and closure level on metal hardness and yield strength. Further information regarding this hypothesis has been obtained in this thesis by testing three different steels of varying hardness levels (6 HRC, 35 HRC, and 60 HRC) including a very hard carburized steel having a hardness level (60 HRC) for which no crack opening stress data for small cracks had yet been obtained. This thesis introduced a new test procedure for obtaining data on the return of crack opening stress to a steady state level following an underload. Smooth specimens were tested under load histories with intermittent underload cycles. The frequency of occurrence of the underloads was varied and the changes in fatigue life observed. The changes in damage per block (the block consisted of an underload cycle followed by intermittent small cycles) were used to determine the value of the closure model parameter governing the recovery of the crack opening stress to its steady state level. Concurrent tests were carried out in which the crack opening stress recovery was measured directly on crack growth specimens using optical microscope measurements. These tests on metals ranging in hardness from soft to very hard were used to assess whether the new technique would produce good data for crack opening stress changes after underloads for all hardness levels. The results were also used to correlate crack closure model parameters with mechanical properties. This together with the steady state crack opening stress, effective strain-life data and the effective intensity versus crack growth rate data obtained from smooth specimen tests devised by previous researchers provided all the data required to calibrate the two models proposed in this investigation to perform strain-life and small crack growth fatigue analysis.

The problem of reconstruction of digital images from their degraded measurements has always been a problem of central importance in numerous applications of imaging sciences. In real life, acquired imaging data is typically contaminated by various types of degradation phenomena which are usually related to the imperfections of image acquisition devices and/or environmental effects. Accordingly, given the degraded measurements of an image of interest, the fundamental goal of image reconstruction is to recover its close approximation, thereby "reversing" the effect of image degradation. Moreover, the massive production and proliferation of digital data across different fields of applied sciences creates the need for methods of image restoration which would be both accurate and computationally efficient. Developing such methods, however, has never been a trivial task, as improving the accuracy of image reconstruction is generally achieved at the expense of an elevated computational burden. Accordingly, the main goal of this thesis has been to develop an analytical framework which allows one to tackle a wide scope of image reconstruction problems in a computationally efficient manner. To this end, we generalize the concept of variable splitting, as a tool for simplifying complex reconstruction problems through their replacement by a sequence of simpler and therefore easily solvable ones. Moreover, we consider two different types of variable splitting and demonstrate their connection to a number of existing approaches which are currently used to solve various inverse problems. In particular, we refer to the first type of variable splitting as Bregman Type Splitting (BTS) and demonstrate its applicability to the solution of complex reconstruction problems with composite, cross-domain constraints. As specific applications of practical importance, we consider the problem of reconstruction of diffusion MRI signals from sub-critically sampled, incomplete data as well as the problem of blind deconvolution of medical ultrasound images. Further, we refer to the second type of variable splitting as Fuzzy Clustering Splitting (FCS) and show its application to the problem of image denoising. Specifically, we demonstrate how this splitting technique allows us to generalize the concept of neighbourhood operation as well as to derive a unifying approach to denoising of imaging data under a variety of different noise scenarios.

A drainage basins runoff response can be determined by the connectivity of generated runoff to the stream network and the connectivity of the downstream stream network. The connectivity of a drainage basin modulates its ability to produce streamflow and respond to precipitation events and is a function of the complex and variable storage capacities along the drainage network. An improved means to measure and account for the dynamics of hydrological connectivity at the basin scale is needed to improve prediction of basin scale streamflow. The overall goal of this thesis is to improve the understanding of hydrological connectivity at the basin scale by measuring hydrological connectivity at the Baker Creek Research Basin during 2009. To this end, the objectives are to 1) investigate the dynamics of hydrological connectivity during a typical water year, 2) define the relationship between the contributing stream network and contributing area, 3) investigate how hydrological connectivity influences streamflow, and 4) define how hydrological connectivity influences runoff response to rainfall events. At a 150 km2 subarctic Precambrian Shield catchment where the poorly-drained heterogeneous mosaic of lakes, exposed bedrock, and soil filled areas creates variable contributing areas, hydrological connectivity was measured between April and September 2009 in 10 sub-basins with a particular focus on three representative sub-basins. The three sub-basins, although of similar relative size, vary considerably in the dominant typology and topology of their constituent elements. At a 10 m spatial resolution, saturated areas were mapped using both multispectral satellite imagery and in situ measurements of storage according to land cover. To measure basin scale hydrological connectivity, the drainage network was treated as a graph network with stream reaches being the edges that connect sub-catchment nodes. The overall hydrological connectivity of the stream network was described as the ratio of actively flowing relative to potentially flowing stream reaches, and the hydrological connectivity of the stream network to the outlet was described as the ratio of actively flowing stream reaches that were connected to the outlet relative to the potentially flowing stream reaches. Hydrological connectivity was highest during the spring freshet but the stream network began to disintegrate with its passing. In some drainage basins, large gate keepers were able to maintain connectivity of the stream network downstream during dry periods. The length of the longest stream was found to be proportional to contributing area raised to a power of 0.605, similar to that noted in Hacks Law and modified Hacks Law relationships. The length of the contributing stream network was also found to be proportional to contributing area raised to a power of 0.851. In general, higher daily average streamflows were noted for higher states of connectivity to the outlet although preliminary investigations allude to the existence of hysteresis in these relationships. Elevated levels of hydrological connectivity were also found to yield higher basin runoff ratios but the shape of the characteristic curve for each basin was heavily influenced by key traits of its land cover heterogeneity. The implications of these findings are that accurate prediction of streamflow and runoff response in a heterogeneous drainage basin with dynamic connectivity will require both an account of the presence or absence of connections but also a differentiation of connection type and an incorporation of aspects of local function that control the flow through connections themselves. The improved understanding of causal factors for the variable streamflow response to runoff generation in this environment will serve as a first step towards developing improved streamflow prediction methods in formerly glaciated landscapes, especially in small ungauged basins.

Mantle cell lymphoma (MCL) is one of several sub-types of B-cell lymphomas. The malignancy is very aggressive and average survival time is short. The hallmark of MCL is over expression of cyclin D1, however about 15% of all MCL cases do not display this over expression and are easily misdiagnosed. Recently the transcription factor Sox11 has been shown to be specifically over expressed in the nucleus of MCL-tumour cells, and polyclonal rabbit anti-Sox11 antibodies have been used to successfully identify MCL in both cyclin D1 positive and negative cases. Howev-er, human recombinant MCL-specific antibodies as have several advantages over these polyclonal rabbit antibodies; they can easily be produced in large quantities in vitro, their specificity is constant from batch to batch and they can possibly be used for therapeutic purposes. Because of this, it is desirable to produce human recombinant antibodies against proteins over expressed in MCL. In this study human recombinant IgGs have been produced towards two pro-teins over expressed in MCL, Sox11 and KIAA0882. This was done by cloning of single chain variable fragments (scFvs), previously selected from a large scFv library through phage display selection against Sox11- and KIAA0882-protein epitope signature tag (PrEST), into vectors containing human IgG constant regions followed by expression of human IgG antibodies in human embryonic kidney (HEK) 293 cells. One IgG clone for each antigen was shown to be functional and specific. Both clones were shown to have overlapping binding epitopes with their polyclonal rabbit antibody counterpart (rabbit anti-Sox11/KIAA0882) through competitive ELISA. The anti-Sox11 IgG was able to detect two bands in cell lysate in Western blot, of which one probably is Sox11 while the other band possibly could be Sox4. However, this needs to be confirmed in future experiments. The affinity of the anti-Sox11 IgG was measured in Biacore and compared to the affinity of its original scFv. This gave a rough estimation of the affinities, but the values are unreliable and the measurements need to be redone. Although more work has to be put into evaluating the potential of the produced IgGs, they compose a promising starting point to an improved understanding and improved diagnosis of MCL.

In this thesis, we address the identification of biomarkers in high-dimensional omics data. The identification of valid biomarkers is especially relevant for personalized medicine that depends on accurate prediction rules. Moreover, biomarkers elucidate the provenance of disease, or molecular changes related to disease. From a statistical point of view the identification of biomarkers is best cast as variable selection. In particular, we refer to variables as the molecular attributes under investigation, e.g. genes, genetic variation, or metabolites; and we refer to observations as the specific samples whose attributes we investigate, e.g. patients and controls. Variable selection in high-dimensional omics data is a complicated challenge due to the characteristic structure of omics data. For one, omics data is high-dimensional, comprising cellular information in unprecedented details. Moreover, there is an intricate correlation structure among the variables due to e.g internal cellular regulation, or external, latent factors. Variable selection for uncorrelated data is well established. In contrast, there is no consensus on how to approach variable selection under correlation. Here, we introduce a multivariate framework for variable selection that explicitly accounts for the correlation among markers. In particular, we present two novel quantities for variable importance: the correlation-adjusted t (CAT) score for classification, and the correlation-adjusted (marginal) correlation (CAR) score for regression. The CAT score is defined as the Mahalanobis-decorrelated t-score vector, and the CAR score as the Mahalanobis-decorrelated correlation between the predictor variables and the outcome. We derive the CAT and CAR score from a predictive point of view in linear discriminant analysis and regression; both quantities assess the weight of a decorrelated and standardized variable on the prediction rule. Furthermore, we discuss properties of both scores and relations to established quantities. Above all, the CAT score decomposes Hotelling’s T 2 and the CAR score the proportion of variance explained. Notably, the decomposition of total variance into explained and unexplained variance in the linear model can be rewritten in terms of CAR scores. To render our approach applicable on high-dimensional omics data we devise an efficient algorithm for shrinkage estimates of the CAT and CAR score. Subsequently, we conduct extensive simulation studies to investigate the performance of our novel approaches in ranking and prediction under correlation. Here, CAT and CAR scores consistently improve over marginal approaches in terms of more true positives selected and a lower model error. Finally, we illustrate the application of CAT and CAR score on real omics data. In particular, we analyze genomics, transcriptomics, and metabolomics data. We ascertain that CAT and CAR score are competitive or outperform state of the art techniques in terms of true positives detected and prediction error.

<p>This thesis is about Bayesian approaches for handling multiplicity. It considers three main kinds of multiple-testing scenarios: tests of exchangeable experimental units, tests for variable inclusion in linear regresson models, and tests for conditional independence in jointly normal vectors. Multiplicity adjustment in these three areas will be seen to have many common structural features. Though the modeling approach throughout is Bayesian, frequentist reasoning regarding error rates will often be employed.</p><p>Chapter 1 frames the issues in the context of historical debates about Bayesian multiplicity adjustment. Chapter 2 confronts the problem of large-scale screening of functional data, where control over Type-I error rates is a crucial issue. Chapter 3 develops new theory for comparing Bayes and empirical-Bayes approaches for multiplicity correction in regression variable selection. Chapters 4 and 5 describe new theoretical and computational tools for Gaussian graphical-model selection, where multiplicity arises in performing many simultaneous tests of pairwise conditional independence. Chapter 6 introduces a new approach to sparse-signal modeling based upon local shrinkage rules. Here the focus is not on multiplicity per se, but rather on using ideas from Bayesian multiple-testing models to motivate a new class of multivariate scale-mixture priors. Finally, Chapter 7 describes some directions for future study, many of which are the subjects of my current research agenda.</p> / Dissertation

<p>In this thesis, we develop some Bayesian sparse learning methods for high dimensional data analysis. There are two important topics that are related to the idea of sparse learning -- variable selection and factor analysis. We start with Bayesian variable selection problem in regression models. One challenge in Bayesian variable selection is to search the huge model space adequately, while identifying high posterior probability regions. In the past decades, the main focus has been on the use of Markov chain Monte Carlo (MCMC) algorithms for these purposes. In the first part of this thesis, instead of using MCMC, we propose a new computational approach based on sequential Monte Carlo (SMC), which we refer to as particle stochastic search (PSS). We illustrate PSS through applications to linear regression and probit models.</p><p>Besides the Bayesian stochastic search algorithms, there is a rich literature on shrinkage and variable selection methods for high dimensional regression and classification with vector-valued parameters, such as lasso (Tibshirani, 1996) and the relevance vector machine (Tipping, 2001). Comparing with the Bayesian stochastic search algorithms, these methods does not account for model uncertainty but are more computationally efficient. In the second part of this thesis, we generalize this type of ideas to matrix valued parameters and focus on developing efficient variable selection method for multivariate regression. We propose a Bayesian shrinkage model (BSM) and an efficient algorithm for learning the associated parameters .</p><p>In the third part of this thesis, we focus on the topic of factor analysis which has been widely used in unsupervised learnings. One central problem in factor analysis is related to the determination of the number of latent factors. We propose some Bayesian model selection criteria for selecting the number of latent factors based on a graphical factor model. As it is illustrated in Chapter 4, our proposed method achieves good performance in correctly selecting the number of factors in several different settings. As for application, we implement the graphical factor model for several different purposes, such as covariance matrix estimation, latent factor regression and classification.</p> / Dissertation

As the world moves towards eliminating lead from consumer products, the microelectronics industry has put effort into developing lead-free solder paste. The major drawback of lead-free solder is the problems caused by its high reflow temperature. Variable frequency microwave (VFM) processing has been shown to allow some materials to be processed at lower temperatures. Issues addressed in this study include using VFM to reduce the solder reflow temperature, comparing the heating rate of different size solder particles, and comparing the reliability of VFM reflowed solder versus conventionally reflowed solder. Results comparing the effect of particle size on the heating rate of solder showed that the differences were negligible. This is due in part to the particle sizes overlapping. Many lead-free solder pastes reflow around 250℃. Results indicate that when using the VFM, lead-free solder paste will reflow at 220℃. The reliability of solder that was reflowed using the VFM at the reduced temperature was found to be comparable to solder reflowed in a conventional manner. Based on these findings, VFM processing can eliminate the major obstacles to making lead-free solder paste a more attractive option for use in the microelectronics industry.

High-tone extensor thrust is common to those with cerebral palsy and those suffering spinal cord injuries. It is a muscle-control phenomenon that causes the body to straighten spastically. One goal of this thesis is to design a dynamic seating system that moves with respect to the wheelchair frame, allowing the seat to move with the user during an extensor thrust and reduce forces. One unique challenge is that the seat needs to remain rigid during normal functional activities and only become dynamic when an involuntary thrust is detected. A second goal of this thesis is to design a control scheme that is able to differentiate between these two types of motion. These design goals are initially investigated with a hinged-seatback system, instrumented with sensors to allow for the detection of thrusts and to actively control seating components. A full seating system is then built to allow for full-body extensor thrusts, involving the seatback, seat bottom, and leg rest of the wheelchair. This system is analyzed for effectiveness of reducing forces on the body during an extensor thrust. Another serious problem for this segment of the population is pressure ulcers. These are caused by prolonged pressure on the skin from weight-bearing bony prominences. Various seating system configurations are known to help with pressure relief. The three standard configurations for a chair are tilt, recline, and standing. The final goal of this thesis is to measure and compare the effectiveness of these three methods for their ability to relieve pressure on the seat bottom. To accomplish this, a powered wheelchair with built-in capabilities for recline and standing is mounted to a tilting mechanism. Test subjects are used to experimentally compare the effectiveness of each method for pressure reduction using pressure mats on all weight-bearing surfaces. A 2D model is also developed and validated with the experimental results.