This thesis describes the use of NetFlow data in the systems for detection of disruptions or anomalies in computer network traffic. Various methods for network data collection are described, focusing especially on the NetFlow protocol. Further, various methods for anomaly detection  in network traffic are discussed and evaluated, and their advantages as well as disadvantages are listed. Based on this analysis one method is chosen. Further, test data set is analyzed using the method. Algorithm for real-time network traffic anomaly detection is designed based on the analysis outcomes. This method was chosen mainly because it enables detection of anomalies even in an unlabelled network traffic. The last part of the thesis describes implementation of the  algorithm, as well as experiments performed using the resulting  application on real NetFlow data.

Master’s thesis assesses the financial and economic situation of the joint-stock company Composite Components. There are theoretical aspects including financial, regression and correlation analysis in the first part of the thesis. Financial indicators are computed and then subjected to the mentioned analyses in order to get prognoses for following years. The created suggestions which are in the end of the thesis are based on the indicators and prognoses.

Background: Statistical and machine learning techniques are now being incorporated into high-dimensional mixture research to overcome issues with traditional methods. Though some methods perform well on specific tasks, no method consistently outperforms all others in complex mixture analyses, largely because different methods were developed to answer different research questions. The research presented here concentrates on answering a single mixtures question: Are there exposure patterns within a mixture corresponding with sources or behaviors that give rise to exposure? Objective: This dissertation details work to design, adapt, and apply pattern recognition methods to environmental mixtures and introduces two methods adapted to specific challenges of environmental health data, (1) Principal Component Pursuit (PCP) and (2) Bayesian non-parametric non-negative matrix factorization (BN²MF). We build on this work to characterize the relationship between identified patterns of in utero endocrine disrupting chemical (EDC) exposure and child neurodevelopment. Methods: PCP---a dimensionality reduction technique in computer vision---decomposes the exposure mixture into a low-rank matrix of consistent patterns and a sparse matrix of unique or extreme exposure events. We incorporated two existing PCP extensions that suit environmental data, (1) a non-convex rank penalty, and (2) a formulation that removes the need for parameter tuning. We further adapted PCP to accommodate environmental mixtures by including (1) a non-negativity constraint, (2) a modified algorithm to allow for missing values, and (3) a separate penalty for measurements below the limit of detection (PCP-LOD). BN²MF decomposes the exposure mixture into three parts, (1) a matrix of chemical loadings on identified patterns, (2) a matrix of individual scores on identified patterns, and (3) and diagonal matrix of pattern weights. It places non-negative continuous priors on pattern loadings, weights, and individual scores and uses a non-parametric sparse prior on the pattern weights to estimate the optimal number. We extended BN²MF to explicitly account for uncertainty in identified patterns by estimating the full distribution of scores and loadings. To test both methods, we simulated data to represent environmental mixtures with various structures, altering the level of complexity in the patterns, the noise level, the number of patterns, the size of the mixture, and the sample size. We evaluated PCP-LOD's performance against principal component analysis (PCA), and we evaluated BN²MF's performance against PCA, factor analysis, and frequentist nonnegative matrix factorization (NMF). For all methods, we compared their solutions with true simulated values to measure performance. We further assessed BN²MF's coverage of true simulated scores. We applied PCP-LOD to an exposure mixture of 21 persistent organic pollutants (POPs) measured in 1,000 U.S. adults from the 2001--2002 National Health and Nutrition Examination Survey (NHANES). We applied BN²MF to an exposure mixture of 17 EDCs measured in 343 pregnant women in the Columbia Center for Children’s Environmental Health's Mothers and Newborns Cohort. Finally, we designed a two-stage Bayesian hierarchical model to estimate health effects of environmental exposure patterns while incorporating the uncertainty of pattern identification. In the first stage, we identified EDC exposure patterns using BN²MF. In the second stage, we included individual pattern scores and their distributions as exposures of interest in a hierarchical regression model, with child IQ as the outcome, adjusting for potential confounders. We present sex-specific results. Results: PCP-LOD recovered the true number of patterns through cross-validation for all simulations; based on an a priori specified criterion, PCA recovered the true number of patterns in 32% of simulations. PCP-LOD achieved lower relative predictive error than PCA for all simulated datasets with up to 50% of the data < LOD. When 75% of values were < LOD, PCP-LOD outperformed PCA only when noise was low. In the POP mixture, PCP-LOD identified a rank three underlying structure. One pattern represented comprehensive exposure to all POPs. The other two patterns grouped chemicals based on known properties such as structure and toxicity. PCP-LOD also separated 6% of values as extreme events. Most participants had no extreme exposures (44%) or only extremely low exposures (18%). BN²MF estimated the true number of patterns for 99% of simulated datasets. BN²MF's variational confidence intervals achieved 95% coverage across all levels of structural complexity with up to 40% added noise. BN²MF performed comparably with frequentist methods in terms of overall prediction and estimation of underlying loadings and scores. We identified two patterns of EDC exposure in pregnant women, corresponding with diet and personal care product use as potentially separate sources or behaviors leading to exposure. The diet pattern expressed exposure to phthalates and BPA. One standard deviation increase in this pattern was associated with a decrease of 3.5 IQ points (95% credible interval: -6.7, -0.3), on average, in female children but not in males. The personal care product pattern represented exposure to phenols, including parabens, and diethyl phthalate. We found no associations between this pattern and child cognition. Conclusion: PCP-LOD and BN^2MF address limitations of existing pattern recognition methods employed in this field such as user-specified pattern number, lack of interpretability of patterns in terms of human understanding, influence of outlying values, and lack of uncertainty quantification. Both methods identified patterns that grouped chemicals based on known sources (e.g., diet), behaviors (e.g., personal care product use), or properties (e.g., structure and toxicity). Phthalates and BPA found in food packaging and can linings formed a BN²MF-identified pattern of EDC exposure negatively associated with female child intelligence in the Mothers and Newborns cohort. Results may be used to inform interventions designed to target modifiable behavior or regulations to act on dietary exposure sources.

<p>Decades and policies come and go, and the ominous problem of corruption remains almost unaltered. Some of the most sought-after policies for corruption deterrence focus on institutional reforms aimed at assuring the right and effective access to information, reinforcing rule of law, tackling impunity, and increasing integrity standards for public servants. The aim of this dissertation is to test whether the impact of these policies over corruption is traceable at the subnational level of mexico and the united states. Seeking to accomplish this purpose, statistics measuring corruption, transparency and relevant variables are analyzed through ols regression and correlation methods. The findings point that spite of the evident benefits of transparency for democratic governance, under the methodology selected and with the ensuing subnational statistics, it is not possible to affirm that corruption is noticeable affected by transparency or integrity variables. Implications of these findings ask for a revision on the manner corruption is measured, and to devise which sort of circumstances bolster or thwart transparency´s prowess to cause a dent over corruption.</p> <p> </p>

This dissertation examines the use of statistical models for prediction. Examples are drawn from public policy and chosen because they represent pressing problems facing U.S. governments at the local, state, and federal level. The first five chapters provide examples where the perfunctory use of linear models, the prediction tool of choice in government, failed to produce reasonable predictions. Methodological flaws are identified, and more accurate models are proposed that draw on advances in statistics, data science, and machine learning. Chapter 1 examines skyscraper construction, where the normality assumption is violated and extreme value analysis is more appropriate. Chapters 2 and 3 examine presidential approval and voting (a leading measure of civic participation), where the non-collinearity assumption is violated and an index model is more appropriate. Chapter 4 examines changes in temperature sensitivity due to global warming, where the linearity assumption is violated and a first-hitting-time model is more appropriate. Chapter 5 examines the crime rate, where the independence assumption is violated and a block model is more appropriate. The last chapter provides an example where simple linear regression was overlooked as providing a sensible solution. Chapter 6 examines traffic fatalities, where the linear assumption provides a better predictor than the more popular non-linear probability model, logistic regression. A theoretical connection is established between the linear probability model, the influence score, and the predictivity.

No description available.

No description available.

In this study we build solutions to three common challenges in the fields of bioinformatics through utilizing statistical methods and developing computational approaches. First, we address a common problem in genome wide association studies, which is linking genotype features within organisms of the same species to their phenotype characteristics. We specifically studied FHA domain genes in Arabidopsis thaliana distributed within Eurasian regions by clustering those plants that share similar genotype characteristics and comparing that to the regions from which they were taken. Second, we also developed a tool for calculating transposable element density within different regions of a genome. The tool is built to utilize the information provided by other transposable element annotation tools and to provide the user with a number of options for calculating the density for various genomic elements such as genes, piRNA and miRNA or for the whole genome. It also provides a detailed calculation of densities for each family and subamily of the transposable elements. Finally, we address the problem of mapping multi reads in the genome and their effects on gene expression. To accomplish this, we implemented methods to determine the statistical significance of expression values within the genes utilizing both a unique and multi-read weighting scheme. We believe this approach provides a much more accurate measure of gene expression than existing methods such as discarding multi reads completely or assigning them randomly to a set of best assignments, while also providing a better estimation of the proper mapping locations of ambiguous reads. Overall, the solutions we built in these studies provide researchers with tools and approaches that aid in solving some of the common challenges that arise in the analysis of high throughput sequence data.

<p>Fundamental changes have been happening in healthcare organizations and delivery in these decades, including more accessible physician information, the low-cost collection and sharing of clinical records, and decision support systems, among others. Emerging information systems and technologies play a signification role in these transformations. To extend the understanding and the implications of information systems on healthcare, my dissertation investigates the influence of information systems on enhancing healthcare operations. The findings reveal the practical value of digitalization in indicating healthcare providers' cognitive behaviors, responding to healthcare crises, and improving medical performance.</p> <p><br></p> <p>The first essay investigates the unrevealed value of a special type of user-generated content in healthcare operations. In today's social media world, individuals are willing to express themselves on various online platforms. This user-generated content posted online help readers get easy assess to individuals' features, including but not limited to personality traits. To study the impact of physicians' personality traits on medicine behaviours and performance, we take a view from the perspective of user generated content posted by their supplier side as well as using physician statements which have been made available in medical review websites. It has been found that a higher openness score leads to lower mortality rates, reduced lab test costs, shorter time usage in hospitals treated by physicians with greater openness scores. Furthermore, taking these personality traits into consideration in an optimization problem of ED scheduling, the estimation of counterfactual analysis shows an average of 11.4%, 18.4%, and 17.8% reduction in in-hospital mortality rates, lab test expenditures, and lengths of stay, respectively. In future operation of healthcare, physicians' personalities should be taken into account when healthcare resources are insufficient in times of healthcare pandemics like COVID-19, as our study indicates that health service providers personality is an actual influence on clinical quality.</p> <p><br></p> <p>In the second essay, we focus on the influences of the most severe healthcare pandemic in these decades, COVID-19, on digital goods consumption and examine whether digital goods consumption is resilient to an individual’s physical restriction induced by the pandemic. Leveraging the enforced quarantine policy during the COVID-19 pandemic as a quasi-experiment, we identify the influence of a specific factor, quarantine policy, on mobile app consumption in every Apple app store category in the short and long terms. In the perspective of better responding in the post-pandemic era, the quantitative findings provide managerial implications to the app industry as well as the stock market for accurately understanding the long-term impact of a significant intervention, quarantine, in the pandemic. Moreover, by using the conditional exogenous quarantine policy to instrument app users’ daily movement patterns, we are able to further investigate the digital resilience of physical mobility in different app categories and quantify the impact of an individual’s physical mobility on human behavior in app usage. For results, we find that the reduction in 10% of one’s physical mobility (measured in the radius of gyration) leads to a 2.68% increase in general app usage and a 5.44% rise in app usage time dispersion, suggesting practitioners should consider users’ physical mobility in future mobile app design, pricing, and marketing.</p> <p><br></p> <p>In the third essay, we investigate the role of an emerging AI-based clinical treatment method, robot-assisted surgery (RAS), in transforming the healthcare delivery. As an advanced technique to help diminish the human physical and intellectual limitations in surgeries, RAS is expected to but has not been empirically proven to improve clinical performance. In this work, we first investigate the effect of RAS on clinical outcomes, controlling physicians' self-selection behavior in choosing whether or not to use RAS treatment methods. In particular, we focus on the accessibility of RAS and explore how physician and patient heterogeneity affect the adoption of the RAS method, including learning RAS and using RAS. Investigating the decision-making process on RAS implementation in both the learning and using stages, we show the synergy of RAS implementation in alleviating healthcare racial disparity. Ultimately, the mechanism analysis will be conducted to reveal the underlying mechanism that induces the enhancement of surgical outcomes. For instance, the estimations tend to reveal that, more than surging clinical performance, RAS tends to increase standardization in time and steps when applying the treatment procedures. </p>

In modern mining industry, new resource efficient and climate resilient methods have been gaining traction. Commissioned efforts to improve the efficiency of European mining is further helping us to such goals. Orexplore AB's X-ray technology for analyzing drill cores is currently involved in two such project. Orexplore AB wishes to incorporate geostatistics (spatial statistics) into their analyzing process in order to further extend the information gained from the mineral data. The geostatistical method implemented here is ordinary kriging which is an interpolation method that, given measured data, predicts intermediate values governed by prior covariance models. Ordinary kriging facilitates prediction of mineral concentrations on a continuous grid in 1-D up to 3-D. Intermediate values are predicted on a Gaussian process regression line, governed by prior covariances. The covariance is modeled by fitting a model to a calculated experimental variogram. Mineral concentrations are available along the lateral surface of the drill core. Ordinary kriging is implemented to sequentially predict mineral concentrations on shorter sections of the drill core, one mineral at a time. Interpolation of mineral concentrations is performed on the data considered in 1-D and 3-D. The validation is performed by calculating the corresponding density at each section that concentrations are predicted on and compare each such value to measured densities. The performance of the model is evaluated by subjective visual evaluation of the fit of the interpolation line, its smoothness, together with the variance. Moreover, the fit is tested through cross-validation using different metrics that evaluates the variance and prediction errors of different models. The results concluded that this method accurately reproduces the measured concentrations while performing well according to the above mentioned metrics, but does not outperform the measured concentrations when evaluated against the measured densities. However, the method was successful in providing information of the minerals in the drill core by producing mineral concentrations on a continuous grid. The method also produced mineral concentrations in 3-D that reproduced the measured densities well. It can be concluded that ordinary kriging implemented according to the methodology described in this report efficiently produces mineral concentrations that can be used to obtain information of the distribution of concentrations in the interior of the drill core. / I den moderna gruvindustrin har nya resurseffektiva och klimatbeständiga metoder ökat i efterfråga. Beställda projekt för att förbättra effektiviteten gällande den europeiska gruvdriften bidrar till denna effekt ytterligare. Orexplore AB:s röntgenteknologi för analys av borrkärnor är för närvarande involverad i två sådana projekt. Orexplore AB vill integrera geostatistik (spatial statistik) i sin analysprocess för att ytterligare vidga informationen från mineraldatan. Den geostatistiska metoden som implementeras här är ordinary kriging, som är en interpolationsmetod som, givet uppmätta data, skattar mellanliggande värden betingade av kovariansmodeller. Ordinary kriging tillåter skattning av mineralkoncentrationer på ett kontinuerligt nät i 1-D upp till 3-D. Mellanliggande värden skattas enligt en Gaussisk process-regressionslinje. Kovariansen modelleras genom att passa en modell till ett beräknat experimentellt variogram. Mineralkoncentrationer är tillgängliga längs borrkärnans mantelyta. Ordinary kriging implementeras för att sekventiellt skatta mineralkoncentrationer på kortare delar av borrkärnan, ett mineral i taget. Interpolering av mineralkoncentrationer utförs på datan betraktad i 1-D och 3-D. Valideringen utförs genom att utifrån de skattade koncentrationerna beräkna den motsvarande densiteten vid varje sektion som koncentrationer skattas på och jämföra varje sådant värde med uppmätta densiteter. Undersökning av modellen utförs genom subjektiv visuell utvärdering av interpolationslinjens passning av datan, dess mjukhet, tillsammans med variansen. Dessutom testas passformen genom korsvalidering med olika mätvärden som utvärderar varians- och skattningsfel för olika modeller. Slutsatsen från resultaten är att denna metod reproducerar de uppmätta koncentrationerna väl samtidigt som den presterar bra enligt de mätvärden som utvärderas, men överträffar ej de uppmätta koncentrationerna vid utvärdering mot de uppmätta densiteterna. Metoden var emellertid framgångsrik med att tillhandahålla information om mineralerna i borrkärnan genom att producera mineralkoncentrationer på ett kontinuerligt rutnät. Metoden producerade också mineralkoncentrationer i 3-D som reproducerade de uppmätta densiteterna väl. Slutsatsen dras att ordinary kriging, implementerad enligt den metod som beskrivs i denna rapport, effektivt skattar mineralkoncentrationer som kan användas för att få information om fördelningen av koncentrationer i det inre av borrkärnan.