Autonomy of operations combined with decentralised management of data has given rise to a number of heterogeneous databases or information systems within an enterprise. These systems are often incompatible in structure as well as content and hence difficult to integrate. This thesis investigates the problem of heterogeneous database integration, in order to meet the increasing demand for obtaining meaningful information from multiple databases without disturbing local autonomy. In spite of heterogeneity, the unity of overall purpose within a common application domain, nevertheless, provides a degree of semantic similarity which manifests itself in the form of similar data structures and common usage patterns of existing information systems. This work introduces a conceptual integration approach that exploits the similarity in meta level information in existing systems and performs metadata mining on database objects to discover a set of concepts common to heterogeneous databases within the same application domain. The conceptual integration approach proposed here utilises the background knowledge available in database structures and usage patterns and generates a set of concepts that serve as a domain abstraction and provide a conceptual layer above existing legacy systems. This conceptual layer is further utilised by an information re-engineering framework that customises and packages information to reflect the unique needs of different user groups within the application domain. The architecture of the information re-engineering framework is based on an object-oriented model that represents the discovered concepts as customised application objects for each distinct user group.

Context: Determining attribute correspondence is the most important, time consuming and knowledge intensive part during databases integration. It is also used in other data manipulation applications such as data warehousing, data design, semantic web and e-commerce. Objectives: In this thesis the aim is to investigate how to find the types of correspondence between the attributes of heterogeneous datasets when schema design information of the data sets is unknown. Methods: A literature review was conducted to extract the knowledge related to the approaches that are used to find the correspondence between the attributes of heterogeneous datasets. Extracted knowledge from the literature review is used in developing an instance based approach for finding types of correspondence between the attributes of heterogeneous datasets when schema design information is unknown. To validate the proposed approach an experiment was conducted in the real environment using the data provided by the Telecom Industry (Ericsson) Karlskrona. Evaluation of the results was carried using the well known and mostly used measures from information retrieval field precision, recall and F-measure. Results: To find the types of correspondence between the attributes of heterogeneous datasets, good results depend on the ability of the algorithm to avoid the unmatched pairs of rows during the Row Similarity Phase. An evaluation of proposed approach is performed via experiments. We found 96.7% (average of three experiments) F-measure. Conclusions: The analysis showed that the proposed approach was feasible to be used and it provided users a mean to find the corresponding attributes and the types of correspondence between corresponding attributes, based on the information extracted from the similar pairs of rows from the heterogeneous data sets where their similarity based on the same common primary keys values.

The process of data integration is an important part of the database field when it comes to database migrations and the merging of data. The research in the area has grown with the addition of machine learning approaches in the last 20 years. Due to the complexity of the research field, no go-to solutions have appeared. Instead, a wide variety of ways of enhancing database migrations have emerged. This thesis examines how well a learning-based solution performs for the semantic integration problem in database migrations. Two algorithms are implemented. One that is based on information retrieval theory, with the goal of yielding a matching result that can be used as a benchmark for measuring the performance of the machine learning algorithm. The machine learning approach is based on grouping data with agglomerative hierarchical clustering and then training a neural network to recognize patterns in the data. This allows making predictions about potential data correspondences across two databases. The results show that agglomerative hierarchical clustering performs well in the task of grouping the data into classes. The classes can in turn be used for training a neural network. The matching algorithm gives a high recall of matching tables, but improvements are needed to both receive a high recall and precision. The conclusion is that the proposed learning-based approach, using agglomerative hierarchical clustering and a neural network, works as a solid base to semi-automate the data integration problem seen in this thesis. But the solution needs to be enhanced with scenario specific algorithms and rules, to reach desired performance. / Dataintegrering är en viktig del inom området databaser när det kommer till databasmigreringar och sammanslagning av data. Forskning inom området har ökat i takt med att maskininlärning blivit ett attraktivt tillvägagångssätt under de senaste 20 åren. På grund av komplexiteten av forskningsområdet, har inga optimala lösningar hittats. Istället har flera olika tekniker framställts, som tillsammans kan förbättra databasmigreringar. Denna avhandling undersöker hur bra en lösning baserad på maskininlärning presterar för dataintegreringsproblemet vid databasmigreringar. Två algoritmer har implementerats. En är baserad på informationssökningsteori, som främst används för att ha en prestandamässig utgångspunkt för algoritmen som är baserad på maskininlärning. Den algoritmen består av ett första steg, där data grupperas med hjälp av hierarkisk klustring. Sedan tränas ett artificiellt neuronnät att hitta mönster i dessa grupperingar, för att kunna göra förutsägelser huruvida olika datainstanser har ett samband mellan två databaser. Resultatet visar att agglomerativ hierarkisk klustring presterar väl i uppgiften att klassificera den data som använts. Resultatet av matchningsalgoritmen visar på att en stor mängd av de matchande tabellerna kan hittas. Men förbättringar behöver göras för att både ge hög en hög återkallelse av matchningar och hög precision för de matchningar som hittas. Slutsatsen är att ett inlärningsbaserat tillvägagångssätt, i detta fall att använda agglomerativ hierarkisk klustring och sedan träna ett artificiellt neuronnät, fungerar bra som en basis för att till viss del automatisera ett dataintegreringsproblem likt det som presenterats i denna avhandling. För att få bättre resultat, krävs att lösningen förbättras med mer situationsspecifika algoritmer och regler.

Made available in DSpace on 2015-03-05T13:53:44Z (GMT). No. of bitstreams: 0 Previous issue date: 21 / Nenhuma / Após o advento da Internet, diversas estratégias de desenvolvimento de software foram modificadas para promover maior reuso e interoperabilidade. Design Patterns e Frameworks nos ajudam a criar software e design flexíveis. A idéia de compor aplicações para que trabalhem juntas é bastante atrativa. No entanto, no domínio da saúde, surgem diversos empecilhos para que se realize esta integração. O uso de tecnologias de Agentes em conjunto com Web Services nos permite pensar em uma solução que garanta interoperabilidade, reuso e flexibilidade entre ambientes heterogêneos. Este trabalho descreve a arquitetura de um Agente de Software, chamado WSAgent (que consiste de uma instância de um Framelet para o sub-domínio paciente, no domínio da saúde) e suas estratégias de colaboração e interoperabilidade. Este trabalho também apresenta um estudo de caso com implementação de um protótipo / After the Internet advent, several strategies about software development were changed to promote more reuse and interoperability. Design Patterns and Frameworks help us to create software and design flexible. The idea of glue applications to work together is very attractive. In the health domains, there are many drawbacks to address its goals. The use of agent technologies combined with Web Services allow us to think about the construction of a bind to grant interoperability, reuse and flexibility between heterogeneous environments. This work describes the architecture of a software agent called WSAgent – an instance of a Framelet of Patient subdomain in Health domain – and its strategies of collaborations and interoperability. This work also presents a case study with the implementation of a prototype