1 |
Monitoring and Analysis of Disk throughput and latency in servers running Cassandra databaseKalidindi, Rajeev varma January 2016 (has links)
Context. Light weight process virtualization has been used in the past e.g., Solaris zones, jails in Free BSD and Linux’s containers (LXC). But only since 2013 is there a kernel support for user namespace and process grouping control that make the use of lightweight virtualization interesting to create virtual environments comparable to virtual machines. Telecom providers have to handle the massive growth of information due to the growing number of customers and devices. Traditional databases are not designed to handle such massive data ballooning. NoSQL databases were developed for this purpose. Cassandra, with its high read and write throughputs, is a popular NoSQL database to handle this kind of data. Running the database using operating system virtualization or containerization would offer a significant performance gain when compared to that of virtual machines and also gives the benefits of migration, fast boot up and shut down times, lower latency and less use of physical resources of the servers. Objectives. This thesis aims to investigate the trade-off in performance while loading a Cassandra cluster in bare-metal and containerized environments. A detailed study of the effect of loading the cluster in each individual node in terms of Latency, CPU and Disk throughput will be analyzed. Methods. We implement the physical model of the Cassandra cluster based on realistic and commonly used scenarios or database analysis for our experiment. We generate different load cases on the cluster for bare-metal and Cassandra in docker scenarios and see the values of CPU utilization, Disk throughput and latency using standard tools like sar and iostat. Statistical analysis (Mean value analysis, higher moment analysis, and confidence intervals) are done on measurements on specific interfaces in order to increase the reliability of the results. Results.Experimental results show a quantitative analysis of measurements consisting Latency, CPU and Disk throughput while running a Cassandra cluster in Bare Metal and Container Environments.A statistical analysis summarizing the performance of Cassandra cluster is surveyed. Results.Experimental results show a quantitative analysis of measurements consisting Latency, CPU and Disk throughput while running a Cassandra cluster in Bare Metal and Container Environments.A statistical analysis summarizing the performance of Cassandra cluster is surveyed. Conclusions. With the detailed analysis, the resource utilization of the database was similar in both the bare-metal and container scenarios. Disk throughput is similar in the case of mixed load and containers have a slight overhead in the case of write loads for both the maximum load case and 66% of maximum load case. The latency values inside the container are slightly higher for all the cases. The mean value analysis and higher moment analysis helps us in doing a finer analysis of the results. The confidence intervals calculated show that there is a lot of variation in the disk performance which might be due to compactions happening randomly. Future work in the area can be done on compaction strategies.
|
2 |
Monitoring and Analysis of CPU Utilization, Disk Throughput and Latency in servers running Cassandra database : An Experimental InvestigationChekkilla, Avinash Goud January 2017 (has links)
Context Light weight process virtualization has been used in the past e.g., Solaris zones, jails in Free BSD and Linux’s containers (LXC). But only since 2013 is there a kernel support for user namespace and process grouping control that make the use of lightweight virtualization interesting to create virtual environments comparable to virtual machines. Telecom providers have to handle the massive growth of information due to the growing number of customers and devices. Traditional databases are not designed to handle such massive data ballooning. NoSQL databases were developed for this purpose. Cassandra, with its high read and write throughputs, is a popular NoSQL database to handle this kind of data. Running the database using operating system virtualization or containerization would offer a significant performance gain when compared to that of virtual machines and also gives the benefits of migration, fast boot up and shut down times, lower latency and less use of physical resources of the servers. Objectives This thesis aims to investigate the trade-off in performance while loading a Cassandra cluster in bare-metal and containerized environments. A detailed study of the effect of loading the cluster in each individual node in terms of Latency, CPU and Disk throughput will be analyzed. Method We implement the physical model of the Cassandra cluster based on realistic and commonly used scenarios or database analysis for our experiment. We generate different load cases on the cluster for Bare-Metal and Docker and see the values of CPU utilization, Disk throughput and latency using standard tools like sar and iostat. Statistical analysis (Mean value analysis, higher moment analysis and confidence intervals) are done on measurements on specific interfaces in order to show the reliability of the results. Results Experimental results show a quantitative analysis of measurements consisting Latency, CPU and Disk throughput while running a Cassandra cluster in Bare Metal and Container Environments. A statistical analysis summarizing the performance of Cassandra cluster while running single Cassandra is surveyed. Conclusions With the detailed analysis, the resource utilization of the database was similar in both the bare-metal and container scenarios. From the results the CPU utilization for the bare-metal servers is equivalent in the case of mixed, read and write loads. The latency values inside the container are slightly higher for all the cases. The mean value analysis and higher moment analysis helps us in doing a finer analysis of the results. The confidence intervals calculated show that there is a lot of variation in the disk performance which might be due to compactions happening randomly. Further work can be done by configuring the compaction strategies, memory, read and write rates.
|
3 |
Modeliranje i implementacija sistema za pretragu naučno-istraživačkih podataka / Modeling and implementation of search system for scientific research dataPenca Valentin 26 December 2014 (has links)
<p>Cilj je specifikacija, modeliranje i implementacija sistema pretrage naučno-istraživačkih podataka. Sistem pretrage treba da omogući bogatu i efikasnu pretragu naučno-istraživačkih podataka. Osnovna ideja je da se pretraga zasniva na bibliotečkim standardima SRU/W i CQL kako bi se mogli prihvatati zahtevi za pretragu sa udaljenih računara, a da korisnici sistema mogu da pretraţuju putem veb forme implementirane upotrebom Web 2.0 tehnologije pri čemu se od korisnika ne očekuje poznavanje pomenutih standarda</p><p>Metodologija – Na osnovu pregledane literature došlo se do smernica za analizu sistema koji sadrţe naučno-istraţivače podatke. Na osnovu izvršene analize predloţene su neophodne funkcionalnosti i relevantni set indeksa koji pretraga CRIS sistema treba da podrţi. Predloţen je novi profil za pretragu podataka naučno-istraţivačke delatnosti. Implementiran je standardizovani sistem pretrage za CRIS sisteme baziran na Web 2.0 tehnologijama, SRU/W profilu pretrage i CQL upitnom jeziku. Sistem omogućava i udaljeno pretraţivanje eksternim entitetima zadavanjem adekvatnih zahteva u skladu sa SRU/W standardom. Korišćeni su CASE alati bazirani na objektno-orijentisanoj metodologiji za modeliranje sistema (UML 2.0). Softverska arhitektura je zasnovana na Internet tehnologijama. Za implementaciju je korišćena Java platforma. Dok je za izradu naprednih elemenata interfejsa korišćena se JSF biblioteka i AJAX tehnologije.</p><p>Rezultati – Razvijen je unifikovani profil pretrage za CRIS sisteme i implementiran je standardizovani sistem pretrage naučno-istraživačkih podataka koji se koristi na Univerzitetu u Novom Sadu. Specificiran i implementiran je sistem za pretragu naučno-istraţivačkih rezultata baziran na Web 2.0 tehnologijama i upotrebi CQL upitnog jezika. Korisnički sistem je dovoljno intuitivan da ga mogu koristiti i korisnici koji i ne poznaju standarde na kojima se zasniva sam pretraga. Specificiran i implementiran je servis koji omogućuje pretragu sa udaljenih računara upotrebom SRU/W protokola.</p><p>Praktična primena – Sistem za pretragu naučno-istraživačkih rezultata je implementiran kao deo CRIS UNS sistema koji se koristi na Univerzitetu u Novom Sadu. Čime je obezbeđen javni uvid i jednostavna pretraga podataka o institucijama/organizacijama, istraţivačima i publikovanim naučnim rezultatima unutar novosadskog univerziteta. Podrţani SRU/W standard ga čini potencijalno dostupnim i za eksterne sisteme.</p><p>Vrednost – Dat je profil pretrage koji predstavlja predlog kako da se izvrši standardizacija pretrage podataka iz naučno-istraţivačke oblasti. Posebna prednost pomenutog CRIS profila je nezavisnost istog od implementacije samog sistema pretrage. Sama primena predloţenog profila je vrlo jednostavna jer se njegovi elementi trebaju samo asocirati sa podacima u postojećim sistemima. Jednostavnost primene profila praktično je pokazana u CRIS sistemu Univerziteta u Novom Sadu. Sama aplikacija pretrage implementira SRU/W profil i pruţa potencijalnu inteoperabilnost sa sistemima koji podrţavaju SRU/W standard. Tako su nakon implementacije sistema, podaci iz CRIS UNS postali dostupniji i trasparentniji.</p> / <p>Aim – The aim is specification, modeling and implementation of search of scientific research data. Search system should provide a rich and effective search of scientific research data. The basic idea is to enable search based on bibliographic standards SRU/W and CQL, because system need to be able to process search requests from remote computer, and also to provide searching through the web form implemented using Web 2.0 technology where users do not need to know these standards.</p><p>Methodology - From the related literature are came the guidelines for the analysis of systems that include scientific researchers data. Based on the analysis necessary functionality and relevant set of indexes that searches CRIS system should support are proposed. A new profile for the search data for scientific research was implemented. It was implemented a standardized system search for CRIS systems based on Web 2.0 technologies, SRU/W profile search and CQL Query Language. The system allows a remote search by entering appropriate requests in accordance with SRU/W standard. UML 2.0 CASE tools based on object-oriented methodology were used for modeling the system. The software architecture is relied on a set of Internet technologies. Implementation technologies are based on Java platform. For the development of advanced interface elements JSF library and AJAX technologies were used.<br />Results - A unified search profile for CRIS systems and a standardized system of search of scientific research data were developed, which are booth used at the University of Novi Sad. It was specified and implemented a system for search of scientific research data based on Web 2.0 technologies and the CQL query language. GUI is intuitive enough that it can be used by people who do not know the standards on which the search operation is based. Also it was specified and implemented a service that allows remote computer to search by using the<br />SRU/W protocol.</p><p>Practical application - Search system of scientific research data is implemented as part of CRIS UNS system which used at the University of Novi Sad for a long time. CRIS UNS is now able to provide public access and easy search for data of institutions/ organizations, researchers and published scientific results within the University of Novi Sad. Supported SRU/W standard makes the CRIS UNS potentially interoperable with external systems.</p><p>Value – Proposed search profile is some kind of a suggestion how to standardize the search and retrieval of scientific research data. A major advantage of the CRIS profile is that he has no any affect on the implementation of the search system. The application of the proposed profile is very simple because its elements should only be associated (mapped) with the data in existing systems. Ease of application for the profile is practically demonstrated in the CRIS system of the University of Novi Sad (CRIS UNS). Search application implements SRU/W profile and offers potential interoperability with the systems that support the SRU/W standard. After the implementation of the search system, it is obviously that data from CRIS UNS become more accessible and transparent.</p> / null
|
Page generated in 0.0131 seconds