Background: A problematic area in today’s large scale distributed systems is the exponential amount of growing log data. Finding anomalies by observing and monitoring this data with manual human inspection methods becomes progressively more challenging, complex and time consuming. This is vital for making these systems available around-the-clock. Aim: The main objective of this study is to determine which are the most suitable Machine Learning (ML) algorithms and if they can live up to needs and requirements regarding optimization and efficiency in the log data monitoring area. Including what specific steps of the overall problem can be improved by using these algorithms for anomaly detection and classification on different real provided data logs. Approach: Initial pre-study is conducted, logs are collected and then preprocessed with log parsing tool Drain and regular expressions. The approach consisted of a combination of K-Means + XGBoost and respectively Principal Component Analysis (PCA) + K-Means + XGBoost. These was trained, tested and with different metrics individually evaluated against two datasets, one being a Server data log and on a HTTP Access log. Results: The results showed that both approaches performed very well on both datasets. Able to with high accuracy, precision and low calculation time classify, detect and make predictions on log data events. It was further shown that when applied without dimensionality reduction, PCA, results of the prediction model is slightly better, by a few percent. As for the prediction time, there was marginally small to no difference for when comparing the prediction time with and without PCA. Conclusions: Overall there are very small differences when comparing the results for with and without PCA. But in essence, it is better to do not use PCA and instead apply the original data on the ML models. The models performance is generally very dependent on the data being applied, it the initial preprocessing steps, size and it is structure, especially affecting the calculation time the most.

No description available.

O presente trabalho tem como objetivo avaliar a influência das variáveis ambientais na mortalidade de idosos (acima de 65 anos) por doenças cardiovasculares (DCV) e respiratórias (RES), nos condados de Los Angeles, Orange e Santa Bárbara na Califórnia, através de análises estatísticas, em particular de componentes principais (ACP). Para isso foram utilizados dados diários das estações meteorológicas dos condados (temperatura, pressão, umidade e velocidade do vento), de poluição do ar (NO2, CO, SO2 e O3) e da mortalidade por DCV e RES, durante o período 1975-2005. Além disso, foram calculados 7 índices de conforto térmico e estabelecidas as zonas de conforto para diferentes graus de percepção térmica a partir do índice de Temperatura Efetiva (TE). Os resultados verificaram tendências diferentes na variação temporal das DCV e RES. Enquanto as doenças RES tiveram um aumento bastante leve em todos os condados, as DCV apresentaram uma ligeira diminuição nos condados de Orange e Santa Bárbara, e uma diminuição constante e acentuada no condado de Los Angeles. Também se verificou uma variação sazonal significativa, com evidente aumento do número de óbitos de ambas as doenças durante o inverno e queda durante o verão. Em relação aos poluentes, foram observadas muitas ultrapassagens dos padrões nacionais de qualidade do ar principalmente para o O3, com 1443 vezes para o condado de Los Angeles, 771 para o Orange e 114 para Santa Bárbara. A partir dos índices de conforto térmico notou-se que o mês mais estressante devido ao calor foi setembro principalmente para os condados de Los Angeles e Santa Bárbara, e julho e agosto para o condado de Orange. Analisando a média de óbitos por sensação térmica observou-se um aumento das doenças nos extremos, principalmente quando a sensação é Muito Frio, com médias por doenças RES de 16,42, 3,31 e 0,81 e por DCV de 77,32, 15,37 e 2,92 nos condados de Los Angeles, Orange e Santa Bárbara, respectivamente. Com base na análise de correlação entre as variáveis e as doenças, observaram-se as melhores correlações no condado de Los Angeles, tanto para as doenças RES quanto DCV, e as correlações mais baixas foram observadas no condado de Santa Bárbara. Os resultados da ACP nos condados de Los Angeles e Orange verificaram o aumento da mortalidade por DCV no inverno, associadas a correlações positivas com os níveis de poluição principalmente em Los Angeles e ao estresse por frio, umidade e vento (via índice de conforto térmico) em Orange. Por outro lado, a mortalidade por doenças RES esteve mais associada às variações meteorológicas do que às concentrações de poluentes, observando-se associações positivas com o índice de calor (IC). Neste sentido, o condado de Orange apresentou maiores correlações com IC do que o condado de Los Angeles, além de uma variância maior (11,36 % contra 9,89 %). Por último, no condado de Santa Bárbara os resultados foram muito diferentes, já que tanto episódios de frio quanto de calor não mostraram impactos claros na população idosa, com respeito a ambos os grupos de doenças. O distintivo foi que os níveis de SO2 tiveram a maior associação positiva à mortalidade principalmente por DCV (0,53), embora tiver o condado as concentrações mais baixas das estudadas (média de 0,29 ppb). Em suma, os condados possuem estruturas sociais diferentes, apesar da semelhança do clima, fazendo com que os resultados sejam bastante dispares entre os mesmos. As perspectivas destes estudos pretendem colocar estes resultados para climas futuros, pois diversos episódios de ondas de calor ocorreram após o ano de 2005. / The objective of this study was to evaluate the influence of environmental variables on the mortality of elderly patients (over 65 years) due to cardiovascular diseases (CD) and respiratory diseases (RD) in the counties of Los Angeles, Orange and Santa Bárbara, California; through statistical analyzes, in particular, principal components (PCA). For this purpose, daily data of the meteorological stations of the counties (temperature, pressure, humidity and wind speed), air pollution (NO2, CO, SO2 and O3) and mortality by CD and RD were used during the period 1975-2005. In addition, seven thermal comfort indexes were calculated and comfort zones were established for different degrees of thermal perception according to the Effective Temperature index (TE). The results showed different trends in the temporal variation of CD and RD. While RD diseases had a fairly mild increase in all counties, CD showed a slight decrease in Orange and Santa Barbara counties and a steady decline in Los Angeles county. There was also a significant seasonal variation, with an evident increase in the number of deaths from both diseases during the winter and fall during the summer. In relation to the pollutants, many national standards for air quality were exceeded, mainly for the O3, with 1443 times for Los Angeles, 771 for Orange and 114 for Santa Barbara. From the thermal comfort indexes it was noted that the most stressful month due to the heat was September, mainly for the counties of Los Angeles and Santa Barbara, and July and August for the county of Orange. Analyzing the average number of deaths due to thermal sensation, there was an increase of the illnesses in extreme, especially when the sensation is Very Cold, with RD averages of 16.42, 3.31 and 0.81 and CD average of 77.32, 15.37 and 2.92 in the counties of Los Angeles, Orange and Santa Barbara, respectively. Based on correlation analysis between variables and diseases, the best correlations were found in Los Angeles for both RD and CD, and the lowest correlations were observed in Santa Barbara. The results of PAC in the counties of Los Angeles and Orange showed the increase in CD mortality in winter associated to positive correlations with the levels of pollution mainly in Los Angeles and stress due to cold, humidity and wind (via thermal comfort index) in Orange. On the other hand, mortality due to RD diseases was more associated with meteorological variations than with pollutant concentrations, with positive associations with heat index (HI). In this sense, Orange County presented higher correlations with HI than the county of Los Angeles, in addition to a greater variance (11.36% vs. 9.89%). Finally, in Santa Barbara the results were very different, since both episodes of cold and heat did not show clear impacts in the elderly population, regarding both groups of diseases. The difference was that SO2 levels had the highest positive association with CD mortality (0.53), although this county had the lowest concentrations studied (mean 0.29 ppb). In sum, counties have different social structures, despite the similarity of the climate, making the results quite different between them. The perspectives of these studies intend to place these results for future climates, since several episodes of heat waves occurred after 2005.

O presente trabalho tem como objetivo avaliar a influência das variáveis ambientais na mortalidade de idosos (acima de 65 anos) por doenças cardiovasculares (DCV) e respiratórias (RES), nos condados de Los Angeles, Orange e Santa Bárbara na Califórnia, através de análises estatísticas, em particular de componentes principais (ACP). Para isso foram utilizados dados diários das estações meteorológicas dos condados (temperatura, pressão, umidade e velocidade do vento), de poluição do ar (NO2, CO, SO2 e O3) e da mortalidade por DCV e RES, durante o período 1975-2005. Além disso, foram calculados 7 índices de conforto térmico e estabelecidas as zonas de conforto para diferentes graus de percepção térmica a partir do índice de Temperatura Efetiva (TE). Os resultados verificaram tendências diferentes na variação temporal das DCV e RES. Enquanto as doenças RES tiveram um aumento bastante leve em todos os condados, as DCV apresentaram uma ligeira diminuição nos condados de Orange e Santa Bárbara, e uma diminuição constante e acentuada no condado de Los Angeles. Também se verificou uma variação sazonal significativa, com evidente aumento do número de óbitos de ambas as doenças durante o inverno e queda durante o verão. Em relação aos poluentes, foram observadas muitas ultrapassagens dos padrões nacionais de qualidade do ar principalmente para o O3, com 1443 vezes para o condado de Los Angeles, 771 para o Orange e 114 para Santa Bárbara. A partir dos índices de conforto térmico notou-se que o mês mais estressante devido ao calor foi setembro principalmente para os condados de Los Angeles e Santa Bárbara, e julho e agosto para o condado de Orange. Analisando a média de óbitos por sensação térmica observou-se um aumento das doenças nos extremos, principalmente quando a sensação é Muito Frio, com médias por doenças RES de 16,42, 3,31 e 0,81 e por DCV de 77,32, 15,37 e 2,92 nos condados de Los Angeles, Orange e Santa Bárbara, respectivamente. Com base na análise de correlação entre as variáveis e as doenças, observaram-se as melhores correlações no condado de Los Angeles, tanto para as doenças RES quanto DCV, e as correlações mais baixas foram observadas no condado de Santa Bárbara. Os resultados da ACP nos condados de Los Angeles e Orange verificaram o aumento da mortalidade por DCV no inverno, associadas a correlações positivas com os níveis de poluição principalmente em Los Angeles e ao estresse por frio, umidade e vento (via índice de conforto térmico) em Orange. Por outro lado, a mortalidade por doenças RES esteve mais associada às variações meteorológicas do que às concentrações de poluentes, observando-se associações positivas com o índice de calor (IC). Neste sentido, o condado de Orange apresentou maiores correlações com IC do que o condado de Los Angeles, além de uma variância maior (11,36 % contra 9,89 %). Por último, no condado de Santa Bárbara os resultados foram muito diferentes, já que tanto episódios de frio quanto de calor não mostraram impactos claros na população idosa, com respeito a ambos os grupos de doenças. O distintivo foi que os níveis de SO2 tiveram a maior associação positiva à mortalidade principalmente por DCV (0,53), embora tiver o condado as concentrações mais baixas das estudadas (média de 0,29 ppb). Em suma, os condados possuem estruturas sociais diferentes, apesar da semelhança do clima, fazendo com que os resultados sejam bastante dispares entre os mesmos. As perspectivas destes estudos pretendem colocar estes resultados para climas futuros, pois diversos episódios de ondas de calor ocorreram após o ano de 2005. / The objective of this study was to evaluate the influence of environmental variables on the mortality of elderly patients (over 65 years) due to cardiovascular diseases (CD) and respiratory diseases (RD) in the counties of Los Angeles, Orange and Santa Bárbara, California; through statistical analyzes, in particular, principal components (PCA). For this purpose, daily data of the meteorological stations of the counties (temperature, pressure, humidity and wind speed), air pollution (NO2, CO, SO2 and O3) and mortality by CD and RD were used during the period 1975-2005. In addition, seven thermal comfort indexes were calculated and comfort zones were established for different degrees of thermal perception according to the Effective Temperature index (TE). The results showed different trends in the temporal variation of CD and RD. While RD diseases had a fairly mild increase in all counties, CD showed a slight decrease in Orange and Santa Barbara counties and a steady decline in Los Angeles county. There was also a significant seasonal variation, with an evident increase in the number of deaths from both diseases during the winter and fall during the summer. In relation to the pollutants, many national standards for air quality were exceeded, mainly for the O3, with 1443 times for Los Angeles, 771 for Orange and 114 for Santa Barbara. From the thermal comfort indexes it was noted that the most stressful month due to the heat was September, mainly for the counties of Los Angeles and Santa Barbara, and July and August for the county of Orange. Analyzing the average number of deaths due to thermal sensation, there was an increase of the illnesses in extreme, especially when the sensation is Very Cold, with RD averages of 16.42, 3.31 and 0.81 and CD average of 77.32, 15.37 and 2.92 in the counties of Los Angeles, Orange and Santa Barbara, respectively. Based on correlation analysis between variables and diseases, the best correlations were found in Los Angeles for both RD and CD, and the lowest correlations were observed in Santa Barbara. The results of PAC in the counties of Los Angeles and Orange showed the increase in CD mortality in winter associated to positive correlations with the levels of pollution mainly in Los Angeles and stress due to cold, humidity and wind (via thermal comfort index) in Orange. On the other hand, mortality due to RD diseases was more associated with meteorological variations than with pollutant concentrations, with positive associations with heat index (HI). In this sense, Orange County presented higher correlations with HI than the county of Los Angeles, in addition to a greater variance (11.36% vs. 9.89%). Finally, in Santa Barbara the results were very different, since both episodes of cold and heat did not show clear impacts in the elderly population, regarding both groups of diseases. The difference was that SO2 levels had the highest positive association with CD mortality (0.53), although this county had the lowest concentrations studied (mean 0.29 ppb). In sum, counties have different social structures, despite the similarity of the climate, making the results quite different between them. The perspectives of these studies intend to place these results for future climates, since several episodes of heat waves occurred after 2005.

Neighboring northern Kaohsiung with a dense population of petrochemical and petroleum industrial complexes included China Petroleum Company (CPC) refinery plant, Renwu and Dazher petrochemical industrial plants. In recent years, although many scholars have conducted regional studies, but are still limited by the lack of relevant information evidences (such as odorous matters identification and VOCs fingerprint database), while unable to clearly identify the causes of poor ambient air quality. By sampling and analyzing VOCs, we will be able to understand the major sources of VOCs in northern Kaohsiung and their contribution, and to provide the air quality management and control countermeasures for local environmental protection administration. In this study, we sampled and analyzed the speciation of VOCs and sulfur-containing odorous matters (SOMs) in the CPC refinery plants, Renwu and Dazher petrochemical complexes simultaneously with stack sampling. The sampling of VOCs and SOMs were conducted on January 7th, 14th, and 19th, 2011 (dry season) and May 6th, 13rd, and 23rd, 2011 (wet season). We established the emission source database, investigated the characteristics of VOC fingerprints, and estimate the emission factor of each stack. It helps us understand the temporal and spatial distribution of VOCs and ascertain major sources and their contribution of VOCs. Major VOCs emitted from the stacks of the CPC refinery plant were toluene and acetone. It showed that petroleum refinery processes had similar VOCs characteristics and fingerprints. The fingerprints of stack emissions at Renwu and Dashe industrial complexes varied with their processes. Hydrogen sulfide was the major sulfur-containing odorous matter in all petrochemical plants. Compared to other petrochemical complexes, Renwu industrial complex emitted a variety of SOMs species as well as relatively high concentrations of sulfur-containing odorous matters. The petrochemical industrial complexes in the industrial ambient of VOCs analysis results showed that isobutane, butane, isopentane, pentane, propane of alkanes, propene of alkenes, toluene, ethylbenzene, xylene, styrene of aromatics, 2-Butanone (MEK), acetone, of carbonyls are major species of VOCs. In addition, ethene+acetylene+ethane (C2), 1,2-dichloroethane, chloromethane, dichloromethane, MTBE were also occasionally found. Sulfur-containing odorous matter (SOMs) analytical results showed that major odorous matters included hydrogen sulfide, methanethiol, dimethyl sulfide, and carbon disulfide. The highest hydrogen sulfide concentration went up to 5.5 ppbv. In this study, the species of VOCs were divided into alkanes, alkenes, aromatics, carbonyls, and others. The temporal and spatial distribution of various types of VOCs strongly correlated with near-surface wind direction. The most obvious contaminants were alkanes, aromatics, and carbonyls of the dispersion to the downwind. Generally, the ambient air surrounding the petrochemical industrial complexes was influenced by various pollutants in the case of high wind speeds. It showed that stack emission and fugitive sources had an important contribution to ambient air quality. TSOMs and hydrogen sulfide emitting mainly from local sources resulted in high concentration of TSOMs and hydrogen sulfide surrounding the petrochemical industrial complex. Principal component analysis (PCA) results showed that the surrounding areas of petrochemical industrial complexes, regardless of dry or wet seasons, were mainly influenced by the process emissions and solvent evaporation. The impact of traffic emission sources ranked the second. Chemical mass balance receptor modeling showed that stack emissions from the CPC refinery plants contributed about 48 %, while fugitive emission sources and mobile sources contributed about 30 % and 11%, respectively. The stack emissions from Renwu industrial complex contributed about 75 %, while fugitive emission sources and mobile sources contributed about 17 % and 5 %, respectively. The stack emissions from Dazher industrial complex contributed about 68 %, while fugitive emission sources and mobile sources contributed about 21 % and 2 %, respectively.

Brain Computer Interfaces (BCIs) are systems developed in order to control devices by using only brain signals. In BCI systems, different mental activities to be performed by the users are associated with different actions on the device to be controlled. Spelling Paradigm is a BCI application which aims to construct the words by finding letters using P300 signals recorded via channel electrodes attached to the diverse points of the scalp. Reducing the letter detection error rates and increasing the speed of letter detection are crucial for Spelling Paradigm. By this way, disabled people can express their needs more easily using this application. In this thesis, two different methods, Support Vector Machine (SVM) and AdaBoost, are used for classification in the analysis. Classification and Regression Trees is used as the weak classifier of the AdaBoost. Time-frequency domain characteristics of P300 evoked potentials are analyzed in addition to time domain characteristics. Wigner-Ville Distribution is used for transforming time domain signals into time-frequency domain. It is observed that classification results are better in time domain. Furthermore, optimum subset of channels that models P300 signals with minimum error rate is searched. A method that uses both SVM and AdaBoost is proposed to select channels. 12 channels are selected in time domain with this method. Also, effect of dimension reduction is analyzed using Principal Component Analysis (PCA) and AdaBoost methods.

Advances in electronics, computer and information technology are fueling major changes in the area of power systems instrumentations. More and more microprocessor based digital instruments are replacing older type of meters. Extensive deployment of digital instruments are generating vast quantities of data which is creating information pressure in Utilities. The legacy SCADA based data management systems do not support management of such huge data. As a result utilities either have to delete or store the metered information in some compact discs, tape drives which are unreliable. Also, at the same time the traditional integrated power industry is going through a deregulation process. The market principle is forcing competition between power utilities, which in turn demands a higher focus on profit and competitive edge. To optimize system operation and planning utilities need better decision making processes which depend on the availability of reliable system information. For utilities it is becoming clear that information is a vital asset. So, the utilities are now keen to store and use as much information as they can. Existing SCADA based systems do not allow to store data of more than a few months. So, in this dissertation effectiveness of compression algorithms in compressing real time operational data has been assessed. Both, lossy and lossless compression schemes are considered. In lossless method two schemes are proposed among which Scheme 1 is based on arithmetic coding and Scheme 2 is based on run length coding. Both the scheme have 2 stages. First stage is common for both the schemes. In this stage the consecutive data elements are decorrelated by using linear predictors. The output from linear predictor, named as residual sequence, is coded by arithmetic coding in Scheme 1 and by run length coding in Scheme 2. Three different types of arithmetic codings are considered in this study : static, decrement and adaptive arithmetic coding. Among them static and decrement codings are two pass methods where the first pass is used to collect symbol statistics while the second is used to code the symbols. The adaptive coding method uses only one pass. In the arithmetic coding based schemes the average compression ratio achieved for voltage data is around 30, for frequency data is around 9, for VAr generation data is around 14, for MW generation data is around 11 and for line flow data is around 14. In scheme 2 Golomb-Rice coding is used for compressing run lengths. In Scheme 2 the average compression ratio achieved for voltage data is around 25, for frequency data is around 7, for VAr generation data is around 10, for MW generation data is around 8 and for line flow data is around 9. The arithmetic coding based method mainly looks at achieving high compression ratio. On the other hand, Golomb-Rice coding based method does not achieve good compression ratio as arithmetic coding but it is computationally very simple in comparison with the arithmetic coding. In lossy method principal component analysis (PCA) based compression method is used. From the data set, a few uncorrelated variables are derived and stored. The range of compression ratio in PCA based compression scheme is around 105-115 for voltage data, around 55-58 for VAr generation data, around 21-23 for MW generation data and around 27-29 for line flow data. This shows that the voltage parameter is amenable for better compression than other parameters. Data of five system parameters - voltage, line flow, frequency, MW generation and MVAr generation - of Souther regional grid of India have been considered for study. One of the aims of this thesis is to argue that collected power system data can be put to other uses as well. In particular we show that, even mining the small amount of practical data (collected from SRLDC) reveals some interesting system behavior patterns. A noteworthy feature of the thesis is that all the studies have been carried out considering data of practical systems. It is believed that the thesis opens up new questions for further investigations.

In this thesis, we would like to present a new way to enhance the “depth map” image which is called as the fusion of depth images. The goal of our thesis is to try to enhance the “depth images” through a fusion of different classification methods. For that, we will use three similar but different methodologies, the Graph-Cut, Super-Pixel and Principal Component Analysis algorithms to solve the enhancement and output of our result. After that, we will compare the effect of the enhancement of our result with the original depth images. This result indicates the effectiveness of our methodology. / Room 401, No.56, Lane 21, Yin Gao Road, Shanghai, China

The task of automatically recognizing targets in IR imagery has a history of approximately 25 years of research and development. ATR is an application of pattern recognition and scene analysis in the field of defense industry and it is still one of the challenging problems. This thesis may be viewed as an exploratory study of ATR problem with encouraging recognition algorithms implemented in the area. The examined algorithms are among the solutions to the ATR problem, which are reported to have good performance in the literature. Throughout the study, PCA, subspace LDA, ICA, nearest mean classifier, K nearest neighbors classifier, nearest neighbor classifier, LVQ classifier are implemented and their performances are compared in the aspect of recognition rate. According to the simulation results, the system, which uses the ICA as the feature extractor and LVQ as the classifier, has the best performing results. The good performance of this system is due to the higher order statistics of the data and the success of LVQ in modifying the decision boundaries.

Because of the increasing demand for the provision of energy worldwide and the numerous damages caused by a major use of fossil sources, the contribution of renewable energies has been increasing significantly in the global energy mix with the aim at moving towards a more sustainable development. In this context, this work aims at the development of a general methodology for designing PV systems based on ecodesign principles and taking into account simultaneously both techno-economic and environmental considerations. In order to evaluate the environmental performance of PV systems, an environmental assessment technique was used based on Life Cycle Assessment (LCA). The environmental model was successfully coupled with the design stage model of a PV grid-connected system (PVGCS). The PVGCS design model was then developed involving the estimation of solar radiation received in a specific geographic location, the calculation of the annual energy generated from the solar radiation received, the characteristics of the different components and the evaluation of the techno-economic criteria through Energy PayBack Time (EPBT) and PayBack Time (PBT). The performance model was then embedded in an outer multi-objective genetic algorithm optimization loop based on a variant of NSGA-II. A set of Pareto solutions was generated representing the optimal trade-off between the objectives considered in the analysis. A multi-variable statistical method (i.e., Principal Componet Analysis, PCA) was then applied to detect and omit redundant objectives that could be left out of the analysis without disturbing the main features of the solution space. Finally, a decision-making tool based on M-TOPSIS was used to select the alternative that provided a better compromise among all the objective functions that have been investigated. The results showed that while the PV modules based on c-Si have a better performance in energy generation, the environmental aspect is what makes them fall to the last positions. TF PV modules present the best trade-off in all scenarios under consideration. A special attention was paid to recycling process of PV module even if there is not yet enough information currently available for all the technologies evaluated. The main cause of this lack of information is the lifetime of PV modules. The data relative to the recycling processes for m-Si and CdTe PV technologies were introduced in the optimization procedure for ecodesign. By considering energy production and EPBT as optimization criteria into a bi-objective optimization cases, the importance of the benefits of PV modules end-of-life management was confirmed. An economic study of the recycling strategy must be investigated in order to have a more comprehensive view for decision making.