Diese Bachelorarbeit beschäftigt sich mit der Entwicklung einer Softwarelösung zur semantischen und syntaktischen Umwandlung natürlicher Sprache in Datenbankabfragesprachen. Ziel ist es, eine benutzerfreundliche Schnittstelle zu schaffen, die auch Nicht-Experten ermöglicht, komplexe Datenbankabfragen durchzuführen. Im Rahmen eines Praktikums bei der OntoChem GmbH wurde zunächst ein regelbasierter Prototyp entwickelt, der natürliche Sprachabfragen in maschinenlesbare Datenbank abfragen transformiert. Anschlieÿend wurde dieser Ansatz mit einem auf Large Language Models (LLMs) basierenden Ansatz, wie beispielsweise ChatGPT, verglichen. Dabei wurden unter anderem die Effizienz, Genauigkeit, Zuverlässigkeit und ökonomischen Kosten beider Ansätze untersucht. Die Arbeit beginnt mit einer Einführung in die Grundlagen der natürlichen Sprachver arbeitung (NLP), regelbasierter Systeme und LLMs. Es folgt eine detaillierte Beschrei bung des Praktikumsprojekts, einschlieÿlich der eingesetzten Technologien und Tools. In den darauf folgenden Kapiteln werden der regelbasierte Ansatz und der LLM-Ansatz zur Umwandlung natürlicher Sprache in Datenbankabfragen vorgestellt, implementiert und getestet. Die Vergleichsanalyse zeigt, dass der regelbasierte Ansatz durch hohe Geschwindigkeit und Datenkontrolle besticht, jedoch in seiner Flexibilität und Genauigkeit limitiert ist. Der LLM-Ansatz bietet hingegen eine höhere Genauigkeit und Flexibilität bei der Interpretation natürlicher Sprache, weist jedoch längere Antwortzeiten und höhere Betriebskosten auf. Abschließend werden Empfehlungen für die Praxis gegeben und zukünftige Forschungsrichtungen aufgezeigt, wie etwa die Kombination beider Ansätze oder das Training eines eigenen Modells. Die Ergebnisse dieser Arbeit tragen dazu bei, die Interaktion zwischen natürlicher Sprache und Datenbanksystemen zu verbessern und bieten praktische Lösungen für die semantische Transformation von Benutzeranfragen.:1 Einleitung 2 1.1 Motivation 2 1.2 Zielsetzung der Arbeit 3 1.3 Aufbau der Arbeit 4 2 Hintergrund und theoretische Grundlagen 6 2.1 Natürliche Sprachverarbeitung (NLP) 6 2.1.1 Grundlagen der NLP 6 2.1.2 Modelle und Algorithmen 7 2.1.3 Anwendungsbereiche 8 2.2 Regelbasierte Systeme 9 2.2.1 Definition und Funktionsweise 9 2.2.2 Beispiele und Anwendungen 10 2.3 Large Language Models (LLMs) 10 2.3.1 Funktionsweise und Architektur 10 2.3.2 Entwicklung und Technologien 14 2.3.3 Training und Datenbasis 15 2.3.4 Anwendungsbereiche 15 2.3.5 Limitationen von GPT-Modellen 16 3 Praktikumsprojekt bei OntoChem GmbH 18 3.1 Unternehmensvorstellung 18 3.1.1 Überblick und Geschichte 18 3.1.2 Produkte und Technologien 19 3.2 Projektbeschreibung 21 3.2.1 Ziel des Projekts 21 3.2.2 Aufgabenstellung 26 3.3 Technologie-Stack und Tools 27 3.3.1 Programmiersprache und Umgebung 27 3.3.2 Bibliotheken 28 4 Regelbasierter Ansatz zur Umwandlung natürlicher Sprache in Datenbankabfragen 29 4.1 API-Design 29 4.1.1 Methodik und Konzeption 29 4.1.2 structFromNaturalSearch 29 4.1.3 queryFromSearchStructure 35 4.2 Implementierung 37 4.2.1 Funktion: SearchStructureFromString 37 4.2.2 Integration OC-Technologien 38 4.2.3 Algorithmen und Regeln 40 4.2.4 Herausforderungen 43 5 LLM-Ansatz zur Umwandlung natürlicher Sprache in Datenbankabfragen 45 5.1 Einführung in den LLM-Ansatz 45 5.1.1 Grundlagen 45 5.1.2 Vergleich mit Regelbasierten Systemen 46 5.2 Prompting in LLMs (z.B. ChatGPT) 46 5.2.1 Prinzipien des Promptings 46 5.2.2 Design effektiver Prompts 47 5.3 Tests und Evaluierung 50 5.3.1 Beschreibung der Tests 50 5.3.2 Ergebnisse und Analyse 52 6 Vergleich der Ansätze 58 6.1 Methodik 58 6.2 Ergebnisse 58 6.3 Diskussion 61 7 Evaluation und Ausblick 62 7.1 Kritische Betrachtung 62 7.2 Limitationen und Fehlerquellen 62 7.3 Fazit und Implikationen 63 7.4 Zukünftige Forschung 63 Literaturverzeichnis I Abbildungsverzeichnis IV Daten- und Codeverzeichnis V

<p>The thesis describes steps taken towards using graphic representation and visual modelling support for the knowledge acquisition process in knowledge-based systems – a process commonly regarded as difficult. The performance of the systems depends on the quality of the embedded knowledge, which makes the knowledge acquisition phase particularly significant. During the acquisition phase, a main obstacle to proper extraction of information is the absence of effective modelling techniques.</p><p>The contributions of the thesis are: introducing a methodology for user-centred knowledge modelling, enhancing transparency to support the modelling of content and of the reasoning strategy, incorporating conceptualisation to simplify the grasp of the contents and to support assimilation of the domain knowledge, and supplying a visual compositional logic programming language for adding and modifying functionality.</p><p>The user-centred knowledge acquisition model, proposed in this thesis, applies a combination of different approaches to knowledge modelling. The aim is to bridge the gap between the users (i.e., knowledge engineers, domain experts and end users) and the system in transferring knowledge, by supporting the users through graphics and visualisation. Visualisation supports the users by providing several different views of the contents of the system.</p><p>The Unified Modelling Language (UML) is employed as a modelling language. A benefit of utilising UML is that the knowledge base can be modified, and the reasoning strategy and the functionality can be changed directly in the model. To make the knowledge base more comprehensible and expressive, we incorporated visual conceptualisation into UML’s diagrams to describe the contents. Visual conceptualisation of the knowledge can also facilitate assimilation in a hypermedia system through visual libraries.</p><p>Visualisation of functionality is applied to a programming paradigm, namely relational programming, often employed in artificial intelligence systems. This approach employs Venn-Euler diagrams as a graphic interface to a compositional operator based relational programming language. </p><p>The concrete result of the research is the development of a graphic representation and visual modelling approach to support the knowledge acquisition process. This approach has been evaluated for two different knowledge bases, one built for hydropower development and river regulation and the other for diagnosing childhood diseases.</p>

The thesis describes steps taken towards using graphic representation and visual modelling support for the knowledge acquisition process in knowledge-based systems – a process commonly regarded as difficult. The performance of the systems depends on the quality of the embedded knowledge, which makes the knowledge acquisition phase particularly significant. During the acquisition phase, a main obstacle to proper extraction of information is the absence of effective modelling techniques. The contributions of the thesis are: introducing a methodology for user-centred knowledge modelling, enhancing transparency to support the modelling of content and of the reasoning strategy, incorporating conceptualisation to simplify the grasp of the contents and to support assimilation of the domain knowledge, and supplying a visual compositional logic programming language for adding and modifying functionality. The user-centred knowledge acquisition model, proposed in this thesis, applies a combination of different approaches to knowledge modelling. The aim is to bridge the gap between the users (i.e., knowledge engineers, domain experts and end users) and the system in transferring knowledge, by supporting the users through graphics and visualisation. Visualisation supports the users by providing several different views of the contents of the system. The Unified Modelling Language (UML) is employed as a modelling language. A benefit of utilising UML is that the knowledge base can be modified, and the reasoning strategy and the functionality can be changed directly in the model. To make the knowledge base more comprehensible and expressive, we incorporated visual conceptualisation into UML’s diagrams to describe the contents. Visual conceptualisation of the knowledge can also facilitate assimilation in a hypermedia system through visual libraries. Visualisation of functionality is applied to a programming paradigm, namely relational programming, often employed in artificial intelligence systems. This approach employs Venn-Euler diagrams as a graphic interface to a compositional operator based relational programming language. The concrete result of the research is the development of a graphic representation and visual modelling approach to support the knowledge acquisition process. This approach has been evaluated for two different knowledge bases, one built for hydropower development and river regulation and the other for diagnosing childhood diseases.

Verification of knowledge-bases is a critical step to ensure the quality of a knowledge-based system. The success of these systems depends heavily on how qualitative the knowledge is. Manual verification is however cumbersome and error prone, especially for large knowledge-bases. This thesis provides a design theory, based upon the suggested framework by Gregor and Jones (2007). The theory proposes a general design of automated verification tools, which have the abilities of verifying heuristic knowledge in rule-based systems utilizing certainty factors. Included is a verification of completeness and consistency technique customized to this class of knowledge-based systems. The design theory is instantiated in a real-world verification tool development project at Uppsala University. Considerable attention is given to the design and implementation of this artifact – uncovering issues and considerations involved in the development process. For the knowledge management practitioner, this thesis offers guidance and recommendations for automated verification tool development projects. For the IS research community, the thesis contributes with extensions of existing design theory, and reveals some of the complexity involved with verification of a specific rule-based system utilizing certainty factors.

The paradigm of Decision Support Systems (DSS) is to support decision-making, while an Expert System’s (ES) major objective is to provide expert advice in specialised situations. Knowledge-Based DSS (KB-DSS), also called Intelligent Decision Support Systems (IDSS), integrate traditional DSS with the advances of ES. A KB-DSS’ knowledge base usually contains knowledge expressed by an expert and captured by a knowledge engineer. The indirect transfer between the domain expert and the knowledge base through a knowledge engineer may lead to a long and inefficient knowledge acquisition process. This thesis compares 11 DSS packages in search of a (KB-) DSS generator where domain experts can specify and maintain a Specific Decision Support System (SDSS) to assist users in making decisions. The proposed (KB-) DSS-generator is tested with a university and study-program prototype. Since course and study plan programs change intermittently, the (KB-) DSS’ knowledge base enables domain experts to set and maintain their course and study plan rules without the assistance of a knowledge engineer. Criteria are set to govern the (KB-) DSS generator search process. Example knowledge base rules are inspected to determine if domain experts will be able to maintain a set of production rules used in a student registration advice system. By developing a prototype and inspecting knowledge base rules, it was found that domain experts would be able to maintain their knowledge in the decentralised knowledge base, on condition that the objects and attributes used in the rule base were first specified by a builder/programmer. / Dissertation (MSc Computer Science)--University of Pretoria, 2005. / Computer Science / unrestricted