This thesis deals with creating client - server application. Client part of the application is created with a help of the Compact .NET framework and it is running on the mobile facility (PDA). Server part is written on .NET framework and it is running on the desktop computer. In the first part I characterize PDA facility, it's using and also I discuss communication between SQL Servers and PDA facilities. Next part describes .NET platform and advantages, which this platform provides. Also I tried to show differences between client - server architectures, because of understanding in my working. The last part of this work deals with implementation of the client - server application.

In the era of big data, data is flooding from numerous data sources and many companies have been utilizing different types of tools to load and process data from various sources in a data lake. The major challenges where different companies are facing these days are how to update data into an existing dataset without having to read the entire dataset and overwriting it to accommodate the changes which have a negative impact on the performance. Besides this, finding a way to capture and track changed data in a big data lake as the system gets complex with large amounts of data to maintain and query is another challenge. Web platforms such as Hopsworks are also facing these problems without having an efficient mechanism to modify an existing processed results and pull out only changed data which could be useful to meet the processing needs of an organization. The challenge of accommodating row level changes in an efficient and effective manner is solved by integrating Hudi with Hops. This takes advantage of Hudi’s upsert mechanism which uses Bloom indexing to significantly speed up the ability of looking up records across partitions. Hudi indexing maps a record key into the file id without scanning over every record in the dataset. In addition, each successful data ingestion is stored in Apache Hudi format stamped with commit timeline. This commit timeline is needed for the incremental processing mainly to pull updated rows since a specified instant of time and obtain change logs from a dataset. Hence, incremental pulls are realized through the monotonically increasing commit time line. Similarly, incremental updates are realized over a time column (key expression) that allows Hudi to update rows based on this time column. HoodieDeltaStreamer utility and DataSource API are used for the integration of Hudi with Hops and Feature store. As a result, this provided a fabulous way of ingesting and extracting row level updates where its performance can further be enhanced by the configurations of the shuffle parallelism and other spark parameter configurations since Hudi is a spark based library. / I dag är stora data mängder vanligt förekommande bland företag. Typiskt så flödar datan från många datakällor och det är populärt bland företag att använda olika typer av verktyg för att läsa och bearbeta data i en data lake. En av de stora utmaningarna som företag står inför idag är att kunna uppdatera stora mängder data på ett effektivt sätt. Tidigare lösningar för att uppdatera stora mängder data är baserat på att skriva över datan, vilket är en ineffektiv metod. En ytterligare utmaning med stora data mängder är problemet av att bokföra ändringar till datan på ett effektivt sätt. Hopsworks är en webbplattform som lagrar och bearbetrar stora mängder data och står således inför dessa utmaningar. I denna avhandling så presenteras en lösning på problemet med att uppdatera stora datamängder på ett effektivt sätt. Lösningen är baserad på att integrera Hudi med Hops. Genom att integrera Hudi med Hops så kan Hops utnyttja Hudis mekanism för effektiv uppdatering av data. Mekanismen som används i Hudi för att uppdatera stora mängder dataär baserad på Bloom-indexering samt logg-baserad lagring av data. Hudi indexerar datan för att snabba upp uppdateringsoperationer. Dessutom så stödjer Hudi att varje uppdatering bokförs till en loggfil, vilket i praktiken fungerar som en tidslinje över datan. Genom att använda tidslinjen så stödjer Hudi att läsa stora datamängder inkrementellt, samt att inspektera datan vid specifika tidpunkter. I denna avhandling så beskrivs hur HoodieDeltaStreamer-verktyget samt Hudis DataSource API används för integrera Hudi med Hops Feature Store. Tack vare denna integration så möjligörs en mer effektiv och användbar uppdatering av stora datamängder i Hops.

[pt] A tarefa chamada brevemente de Texto-para-SQL envolve a geração de uma consulta SQL com base em um banco de dados relacional e uma pergunta em linguagem natural. Embora os rankings de benchmarks conhecidos indiquem que Modelos de Linguagem Grandes (LLMs) se destacam nessa tarefa, eles são avaliados em bancos de dados com esquemas bastante simples. Esta dissertação investiga inicialmente o desempenho de modelos Texto-para-SQL baseados em LLMs em um banco de dados disponível ao público (Mondial)com um esquema conceitual complexo e um conjunto de 100 perguntas em Linguagem Natural (NL). Executando sob GPT-3.5 e GPT-4, os resultados deste primeiro experimento mostram que as ferramentas baseadas em LLM têm desempenho significativamente inferior ao relatado nesses benchmarks e enfrentam dificuldades com a vinculação de esquemas e joins, sugerindo que o esquema relacional pode não ser adequado para LLMs. Essa dissertação propõe então o uso de visões e descrições de dados amigáveis ao LLM para melhorara precisão na tarefa Texto-para-SQL. Em um segundo experimento, usando a estratégia com melhor performance, custo e benefício do experimento anterior e outro conjunto com 100 perguntas sobre um banco de dados do mundo real, os resultados mostram que a abordagem proposta é suficiente para melhorar consideravelmente a precisão da estratégia de prompt. Esse trabalho conclui com uma discussão dos resultados obtidos e sugere abordagens adicionais para simplificar a tarefa de Texto-para-SQL. / [en] The Text-to-SQL task involves generating an SQL query based on a given relational database and a Natural Language (NL) question. While the leaderboards of well-known benchmarks indicate that Large Language Models (LLMs) excel in this task, they are evaluated on databases with simpler schemas. This dissertation first investigates the performance of LLM-based Text-to-SQL models on a complex and openly available database (Mondial) with a large schema and a set of 100 NL questions. Running under GPT-3.5 and GPT-4, the results of this first experiment show that the performance of LLM-based tools is significantly less than that reported in the benchmarks and that these tools struggle with schema linking and joins, suggesting that the relational schema may not be suitable for LLMs. This dissertation then proposes using LLM-friendly views and data descriptions for better accuracy in the Text-to-SQL task. In a second experiment, using the strategy with better performance, cost and benefit from the previous experiment and another set with 100 questions over a real-world database, the results show that the proposed approach is sufficient to considerably improve the accuracy of the prompt strategy. This work concludes with a discussion of the results obtained and suggests further approaches to simplify the Text-to-SQL task.

The project involves an extensive database with SQL, developing a backend with Express.js and implementing the frontend with React. The database forms the backbone of the entire application and supports its functionality, while the frontend interface is focused on offering an administration panel for users. The backend is designed to function as a REST API, enabling simple and efficient communication between different parts of the application. Using Express.js, the system uses CRUD (Create, Read, Update, Delete) operations to efficiently manage the database and provide a smooth user experience. Important features such as authentication and registration confirmation via email have also been implemented using SendGrid, ensuring a secure and user-friendly platform. By integrating these elements, the project creates a stable foundation for further development of the application. / Projektet handlar om att skapa en omfattande databas med SQL, utveckla en backend med Express.js och implementera frontend med React. Databasen utgör ryggraden i hela applikationen och stöder dess funktionalitet, medan frontend-gränssnittet är inriktat på att erbjuda en administrationspanel för användare. Backenden är designad för att fungera som ett REST API, vilket möjliggör enkel och effektiv kommunikation mellan olika delar av applikationen. Med hjälp av Express.js använder systemet CRUD-operationer (Create, Read, Update, Delete) för att effektivt hantera databasen och tillhandahålla en smidig användarupplevelse. Viktiga funktioner såsom autentisering och registreringsbekräftelse via e-post har också implementerats med hjälp av SendGrid, vilket säkerställer en trygg och användarvänlig plattform. Genom att integrera dessa element skapar projektet en stabil grund för vidareutveckling av applikationen.

I takt med att datakvantiteter växer och kraven på skalbarhet och tillgänglighet inom molntjänster växer, framhävs behovet av undersökningar kring dess prestanda och kostnadseffektivitet.  Dessa analyser är avgörande för att optimera tjänster och bistå företag med värdefulla rekommendationer för att fatta välgrundade beslut om datalagring i molnet. Detta examensarbete undersöker kostnad samt prestanda hos relationella och icke-relationella datalagringslösningar implementerade på Microsoft Azure och Google Cloud Platform. Verktyget Hyperfine används för att mäta latens och tjänsternas kostnadseffektivitet beräknas baserat på detta resultat samt dess beräknade månadskostnader. Studiens resultat indikerar att för de utvärderade relationella databastjänsterna uppvisar Azure SQL Database initialt en låg latens som sedan ökar proportionellt med datamängden, medan Google Cloud SQL visar en något högre latens vid lägre datamängder men mer konstant latens vid högre datamängder. Azure SQL visar sig vara mer kostnadseffektiv i förhållande till Google Cloud SQL, vilket gör den till ett mer fördelaktigt alternativ för företag som eftersträvar hög prestanda till lägre kostnader. Vid jämförelse mellan de två icke-relationella databastjänsterna Azure Cosmos DB och Google Cloud Datastore uppvisar Azure Cosmos DB genomgående jämförelsevis lägre latens och överlägsen kostnadseffektivitet. Detta gör Azure Cosmos DB till en fördelaktig lösning för företag som prioriterar ekonomisk effektivitet i sin databashantering. / As data volumes grow and the demands for scalability and availability within cloud services increase, the need for studies on their performance and cost-effectiveness is emphasized. These analyses are crucial for optimizing services and providing businesses with valuable recommendations to make well-grounded decisions about cloud data storage. This thesis examines cost and performance for relational and non-relational data storage solutions implemented on Microsoft Azure and Google Cloud Platform. The tool Hyperfine is used to evaluate latency and the cloud services cost efficiency is calculated using this result as well as their monthly cost. The study's results regarding relational data storage indicate that Azure SQL Database initially exhibits low latency, which then increases proportionally with the data volume, while Google Cloud SQL shows slightly higher latency at smaller data volumes but more consistent latency with more data. Azure SQL Database is more cost-effective, making it a more favorable option than Google Cloud SQL for companies seeking high performance at lower costs. Regarding evaluated services for non-relational data storage Azure Cosmos DB consistently demonstrates lower latency and superior cost efficiency compared to Google Cloud Datastore, making it the preferred solution for companies prioritizing economic efficiency in their database management.

This paper deals with data warehouses and OLAP. These technologies are defined and described here. Then an introduction of the architecture of product MS SQL Server and its tools for work with data warehouses and OLAP folow. The knowledge gained is used for creation of sample application.

XML is an emerging standard format for data on the Web as well as in business applications. In order to store and access this information in an efficient manner, database technology must be utilized. A relational database system, the most established and mature technology for query processing and storage, creates a strong foundation for such an XML data management system. However, while relational databases are based on SQL queries, the original user queries are written in XQuery, an XML query language. This XML query language has support for order-sensitive queries as XML is an order-sensitive markup language. A major problem has been discovered with loading XML in a relational database. That problem is the lack of native SQL support for and management of order handling. While XQuery has order and positional support, SQL does not have the same support. For example, individuals who were viewing XML information about music albums would have a hard time querying for the first three songs of a track list from a relational backend. Mapping XML documents to relational backends also proves hard as the data models (hierarchical elements versus flat tables) are so different. For these reasons, and other purposes, the Rainbow System is being developed at WPI as a system that bridges XML data and relational data. This thesis in particular deals with the algebra operators that affect order, order sensitive loading and mapping of XML documents, and the pushdown of order handling into SQL-capable query engines. The contributions of the thesis are the order-sensitive rewrite rules, new XML to relational mappings with different order styles, order-sensitive template-driven SQL generation, and a proposed metadata table for order-sensitive information. A system that implements these proposed techniques with XQuery as the XML query language and Oracle as the backend relational storage system has been developed. Experiments were created to measure execution time based on various factors. First, scalability of the system as backend data set size grows is studied. Second, scalability of the system as results returned from the database grows, and finally, query execution times with different loading types are explored. The experimental results are encouraging. Query execution with the relational backend proves to be much faster than native execution within the Rainbow system. These results confirm the practical utility of our proposed order-sensitive XQuery execution solution over relational data.

<p>När det gäller fakturering av ett företags tjänster har det visat sig att den manuella hanteringen ofta är tidskrävande och att det lätt blir fel. Därför har det tagits fram många faktureringssystem för olika datorsystem. Detta examensarbete går ut på att ta fram en prototyp av ett automatiskt rapportsystem baserat på utvald användarinformation i Active Directory, informationen ska sedan användas som faktureringsunderlag. Informationen sammanställs i en databas där användaren på ett lätt sätt ska kunna ta fram en sammanställning av kundernas användning av diverse tjänster för en specifik domän.</p> / <p>When it comes to invoicing a company’s services it has become evident that the manual handling very often is time-consuming and easily becomes wrong. Therefore many developers have developed different invoicingsystems for different computersystems. This diploma work is about developing a prototype of a fully automatic reportsystem based on Active Directory-information. This information will later on be used as basic data for the invoice. The information will be put together in a database where the user easily can retrieve a compilation about a customer’s usage of different services.</p>

När det gäller fakturering av ett företags tjänster har det visat sig att den manuella hanteringen ofta är tidskrävande och att det lätt blir fel. Därför har det tagits fram många faktureringssystem för olika datorsystem. Detta examensarbete går ut på att ta fram en prototyp av ett automatiskt rapportsystem baserat på utvald användarinformation i Active Directory, informationen ska sedan användas som faktureringsunderlag. Informationen sammanställs i en databas där användaren på ett lätt sätt ska kunna ta fram en sammanställning av kundernas användning av diverse tjänster för en specifik domän. / When it comes to invoicing a company’s services it has become evident that the manual handling very often is time-consuming and easily becomes wrong. Therefore many developers have developed different invoicingsystems for different computersystems. This diploma work is about developing a prototype of a fully automatic reportsystem based on Active Directory-information. This information will later on be used as basic data for the invoice. The information will be put together in a database where the user easily can retrieve a compilation about a customer’s usage of different services.

This thesis is focused on XML and related technologies. The XML language is directly linked to the databases and its support in databases. The overview of the XML support provided by various database products and systems are presented in this work. Support in the MS SQL Server 2008 is discussed in more detail starting with the mapping of relational data to XML and vice versa to support of the XML data type and work with it through XQuery. Also some indexing techniques are briefly presented. Finally, the support in MS SQL Server 2008 is demonstrated by means of a sample application, which verifes the theoretical knowledge in practice.