Die Mensch-Computer-Interaktion wird dank neuer Eingabemöglichkeiten jenseits von Tastatur und Maus reicher, vielseitiger und intuitiver. Durch den Verzicht auf zusätzliche Geräte beim Umgang mit Computern geht seitens der Eingabeverarbeitung jedoch eine erhöhte Komplexität einher: Die Programmierung gestischer Eingabe für Multitouch-Systeme ist in derzeitigen Frameworks abgesehen von den verfügbaren Standard-Gesten mit hohem Aufwand verbunden. Die entwickelte Gestenformalisierung für Multitouch (GeForMT) definiert eine domänenspezifische Sprache zur Beschreibung von Multitouch-Gesten. Statt wie verwandte Formalisierungsansätze detaillierte Filter für die Rohdaten zu definieren, bedient sich GeForMT eines bildhaften Ansatzes, um Gesten zu beschreiben. Die Konzeption von Gesten wird unterstützt, indem beispielsweise in einem frühen Stadium der Entwicklung Konflikte zwischen ähnlichen Gesten aufgedeckt werden. Die formalisierten Gesten lassen sich direkt in den Code einbetten und vereinfachen damit die Programmierung. Das zugrundeliegende Framework sorgt für die Verbindung zu den Algorithmen der Gestenerkennung. Die Übertragung des semiotischen Ansatzes zur Formalisierung auf andere Formen gestischer Eingabe wird abschließend diskutiert.:1 Einleitung 1.1 Motivation 1.2 Zielstellung und Abgrenzung 1.3 Aufbau der Arbeit 2 Interdisziplinäre Grundlagenbetrachtung 2.1 Semiotik 2.1.1 Begriffe und Zeichenklassen 2.1.2 Linguistik 2.1.3 Graphische Semiologie 2.1.4 Formgestaltung und Produktsprache 2.1.5 Interfacegestaltung 2.2 Gestenforschung 2.2.1 Kendons Kontinuum für Gesten 2.2.2 Taxonomien 2.2.3 Einordnung 2.3 Gestische Eingabe in der Mensch-Computer-Interaktion 2.3.1 Historische Entwicklung von Ein- und Ausgabetechnologien 2.3.2 Begreifbare Interaktion 2.3.3 Domänenspezifische Modellierung 2.4 Zusammenfassung 3 Verwandte Formalisierungsansätze 3.1 Räumliche Gesten 3.1.1 XML-Beschreibung mit der Behaviour Markup Language 3.1.2 Detektornetze in multimodalen Umgebungen 3.1.3 Gestenvektoren zur Annotation von Videos 3.1.4 Vergleich 3.2 Gesten im Sketching 3.2.1 Gestenfunktionen für Korrekturzeichen 3.2.2 Sketch Language zur Beschreibung von Skizzen 3.2.3 Domänenspezifische Skizzen mit LADDER 3.2.4 Vergleich 3.3 Flächige Gesten 3.3.1 Regelbasierte Definition mit Midas 3.3.2 Gesture Definition Language als Beschreibungssprache 3.3.3 Reguläre Ausdrücke von Proton 3.3.4 Gesture Interface Specification Language 3.3.5 Logische Formeln mit Framous 3.3.6 Gesture Definition Markup Language 3.3.7 Vergleich 3.4 Zusammenfassung 4 Semiotisches Modell zur Formalisierung 4.1 Phasen gestischer Eingabe 4.2 Syntax gestischer Eingabe 4.3 Semantik gestischer Eingabe 4.4 Pragmatik gestischer Eingabe 4.5 Zusammenfassung 5 Gestenformalisierung für Multitouch 5.1 Ausgangslage für die Konzeption 5.1.1 Ikonographische Einordnung flächiger Gesten 5.1.2 Voruntersuchung zur Programmierung flächiger Gesten 5.1.3 Anforderungskatalog für die Formalisierung 5.2 Semiotische Analyse flächiger Gesten 5.2.1 Syntax flächiger Gesten 5.2.2 Semantik flächiger Gesten 5.2.3 Pragmatik flächiger Gesten 5.3 Präzedenzfälle für die Formalisierung 5.3.1 Geschicklichkeit bei der Multitouch-Interaktion 5.3.2 Präzision bei flächigen Gesten 5.3.3 Kooperation in Multitouch-Anwendungen 5.4 Evaluation und Diskussion 5.4.1 Vergleich der Zeichenanzahl 5.4.2 Evaluation der Beschreibungsfähigkeit 5.4.3 Limitierungen und Erweiterungen 6 Referenzarchitektur 6.1 Analyse existierender Multitouch-Frameworks 6.2 Grundlegende Architekturkomponenten 6.2.1 Parser 6.2.2 Datenmodell 6.2.3 Gestenerkennung und Matching 6.2.4 Programmierschnittstelle 6.3 Referenzimplementierung für JavaScript 6.3.1 Komponenten der Bibliothek 6.3.2 Praktischer Einsatz 6.3.3 Gesteneditor zur bildhaften Programmierung 7 Praxisbeispiele 7.1 Analyse prototypischer Anwendungen 7.1.1 Workshop zur schöpferischen Zerstörung 7.1.2 Workshop zu semantischen Dimensionen 7.1.3 Vergleich 7.2 Abbildung von Maus-Interaktion auf flächige Gesten in DelViz 7.2.1 Datengrundlage und Suchkonzept 7.2.2 Silverlight-Implementierung von GeForMT 7.3 Flächige Gesten im 3D-Framework Bildsprache LiveLab 7.3.1 Komponentenarchitektur 7.3.2 Implementierung von GeForMT mit C++ 7.4 Statistik und Zusammenfassung 8 Weiterentwicklung der Formalisierung 8.1 Räumliche Gesten 8.1.1 Verwandte Arbeiten 8.1.2 Prototypischer Aufbau 8.1.3 Formalisierungsansatz 8.2 Substanzen des Alltags 8.2.1 Verwandte Arbeiten 8.2.2 Experimente mit dem Explore Table 8.2.3 Formalisierungsansatz 8.3 Elastische Oberflächen 8.3.1 Verwandte Arbeiten 8.3.2 Der Prototyp DepthTouch 8.3.3 Formalisierungsansatz 9 Zusammenfassung 9.1 Kapitelzusammenfassungen und Beiträge der Arbeit 9.2 Diskussion und Bewertung 9.3 Ausblick und zukünftige Arbeiten Anhang Vergleichsmaterial Formalisierungsansätze Fragebogen Nachbefragung Ablaufplan studentischer Workshops Grammatikdefinitionen Statistische Auswertung Gestensets Literatur Webreferenzen Eigene Veröffentlichungen Betreute studentische Arbeiten Abbildungsverzeichnis Tabellen Verzeichnis der Code-Beispiele

The use of Audience Response Systems (ARSs) for tech-enhanced learning scenarios has proven to address issues occurring within higher education, e.g. the missing interaction between the lecturer and the students. Since the majority of these systems relies on a single supported didactic concept and therefore has a limited set of provided functions, ARSs are currently restricted to support classic content-based as well as enquiry-based learning. The support of more advanced didactic concepts in order to investigate studio-based learning is currently not possible due to the lacking collaborative and cooperative functionality. This paper presents a unified (meta-)model which is able to express various scenarios, targeting the holistic support of content-based, enquiry-based and studio-based learning. The created model is evaluated within a user study to reason about the applicability of its underlying concept as well as the defined function blocks. In addition, this paper purposes ideas for a future graphical editor, which will support the modeling process, and provides concrete details for a possible implementation of a system on top of the (meta-)model. Index Terms—audience response systems, collaborative learning, adaptability, meta-model, domain-specific language, highereducation, technology-enhanced learning

Experimentieren in der vorliegenden Arbeit bedeutet, Experimente auf der Basis von computerbasierten Modellen durchzuführen, wobei diese Modelle Struktur, Verhalten und Umgebung eines Systems abstrahiert beschreiben. Aus verschiedenen Gründen untersucht man stellvertretend für das System ein Modell dieses Systems. Systematisches Experimentieren bei Variation der Modelleingabeparameterbelegung führt in der Regel zu sehr vielen, potentiell lang andauernden Experimenten, die geplant, dokumentiert, automatisiert ausgeführt, überwacht und ausgewertet werden müssen. Häufig besteht dabei das Problem, dass dem Experimentator (der üblicherweise kein Informatiker ist) adäquate Ausdrucksmittel fehlen, um seine Experimentier-Prozesse formal zu beschreiben, so dass sie von einem Computersystem automatisiert ausgeführt werden können. Dabei müssen Verständlichkeit, Nachnutzbarkeit und Reproduzierbarkeit gewahrt werden. Der neue Ansatz besteht darin, generelle Experimentier-Workflow-Konzepte als Spezialisierung von Scientific-Workflows zu identifizieren und diese als eine metamodellbasierte Domain-Specific-Language (DSL) zu formalisieren, die hier als Experimentation-Language (ExpL) bezeichnet wird. ExpL beinhaltet allgemeine Workflow-Konzepte und erlaubt das Modellieren von Experimentier-Workflows auf einer frameworkunabhängigen, konzeptuellen Ebene. Dadurch werden die Nachnutzbarkeit und das Publizieren von Experimentier-Workflows nicht mehr durch die Gebundenheit an ein spezielles Framework behindert. ExpL wird immer in einer konkreten Experimentierdomäne benutzt, die spezifische Anforderungen an Konfigurations- und Auswertemethoden aufweist. Um mit dieser Domänenspezifik umzugehen, wird in dieser Arbeit gezeigt, diese beiden Aspekte separat in zwei weiteren, abhängigen Domain-Specific-Languages (DSLs) zu behandeln: für Konfiguration und Auswertung. / Experimentation in my work means performing experiments based on computer-based models, which describe system structure and behaviour abstractly. Instead of the system itself models of the system will be explored due to several reasons. Systematic experimentation using model input parameter variation assignments leads to lots of possibly long-running experiments that must be planned, documented, automated executed, monitored and evaluated. The problem is, that experimenters (who are usually not computer scientists) miss the proper means of expressions (e. g., to express variations of parameter assignments) to describe experimentation processes formally in a way, that allows their automatic execution by a computer system while preserving reproducibility, re-usability and comprehension. My approach is to identify general experimentation workflow concepts as a specialization of a scientific workflow and formalize them as a meta-model-based domain-specific language (DSL) that I call experimentation language (ExpL). experimentation language (ExpL) includes general workflow concepts like control flow and the composition of activities, and some new declarative language elements. It allows modeling of experimentation workflows on a framework-independent, conceptional level. Hence, re-using and sharing the experimentation workflow with other scientists is not limited to a particular framework anymore. ExpL is always being used in a specific experimentation domain that has certain specifics in configuration and evaluation methods. Addressing this, I propose to separate the concerns and use two other, dependent domain-specific languages (DSLs) additionally for configuration and evaluation.

The benefits of Model-Driven Software Development (MDSD) and Domain-Specific Languages (DSLs) wrt. efficiency and quality in software engineering increase the demand for custom languages and the need for efficient methods for language engineering. This motivated the introduction of language families that aim at further reducing the development costs and the maintenance effort for custom languages. The basic idea is to exploit the commonalities and provide means to enable systematic variation among a set of related languages. Current techniques and methodologies for language engineering are not prepared to deal with the particular challenges of language families. First, language engineering processes lack means for a systematic analysis, specification and management of variability as found in language families. Second, technical approaches for a modular specification and realisation of languages suffer from insufficient modularity properties. They lack means for information hiding, for explicit module interfaces, for loose coupling, and for flexible module integration. Our first contribution, Feature-Oriented Language Family Engineering (LFE), adapts methods from Software Product Line Engineering to the domain of language engineering. It extends Feature-Oriented Software Development to support metamodelling approaches used for language engineering and replaces state-of-the-art processes by a variability- and reuse-oriented LFE process. Feature-oriented techniques are used as means for systematic variability analysis, variability management, language variant specification, and the automatic derivation of custom language variants. Our second contribution, Integrative Role-Based Language Composition, extends existing metamodelling approaches with roles. Role models introduce enhanced modularity for object-oriented specifications like abstract syntax metamodels. We introduce a role-based language for the specification of language components, a role-based composition language, and an extensible composition system to evaluate role-based language composition programs. The composition system introduces integrative, grey-box composition techniques for language syntax and semantics that realise the statics and dynamics of role composition, respectively. To evaluate the introduced approaches and to show their applicability, we apply them in three major case studies. First, we use feature-oriented LFE to implement a language family for the ontology language OWL. Second, we employ role-based language composition to realise a component-based version of the language OCL. Third, we apply both approaches in combination for the development of SumUp, a family of languages for mathematical equations.

The benefits of Model-Driven Software Development (MDSD) and Domain-Specific Languages (DSLs) wrt. efficiency and quality in software engineering increase the demand for custom languages and the need for efficient methods for language engineering. This motivated the introduction of language families that aim at further reducing the development costs and the maintenance effort for custom languages. The basic idea is to exploit the commonalities and provide means to enable systematic variation among a set of related languages. Current techniques and methodologies for language engineering are not prepared to deal with the particular challenges of language families. First, language engineering processes lack means for a systematic analysis, specification and management of variability as found in language families. Second, technical approaches for a modular specification and realisation of languages suffer from insufficient modularity properties. They lack means for information hiding, for explicit module interfaces, for loose coupling, and for flexible module integration. Our first contribution, Feature-Oriented Language Family Engineering (LFE), adapts methods from Software Product Line Engineering to the domain of language engineering. It extends Feature-Oriented Software Development to support metamodelling approaches used for language engineering and replaces state-of-the-art processes by a variability- and reuse-oriented LFE process. Feature-oriented techniques are used as means for systematic variability analysis, variability management, language variant specification, and the automatic derivation of custom language variants. Our second contribution, Integrative Role-Based Language Composition, extends existing metamodelling approaches with roles. Role models introduce enhanced modularity for object-oriented specifications like abstract syntax metamodels. We introduce a role-based language for the specification of language components, a role-based composition language, and an extensible composition system to evaluate role-based language composition programs. The composition system introduces integrative, grey-box composition techniques for language syntax and semantics that realise the statics and dynamics of role composition, respectively. To evaluate the introduced approaches and to show their applicability, we apply them in three major case studies. First, we use feature-oriented LFE to implement a language family for the ontology language OWL. Second, we employ role-based language composition to realise a component-based version of the language OCL. Third, we apply both approaches in combination for the development of SumUp, a family of languages for mathematical equations.:1. Introduction 1.1. The Omnipresence of Language Families 1.2. Challenges for Language Family Engineering 1.3. Language Family Engineering with Features and Role-Based Composition 2. Review of Current Language Engineering 2.1. Language Engineering Processes 2.1.1. Analysis Phase 2.1.2. Design Phase 2.1.3. Implementation Phase 2.1.4. Applicability in Language Family Engineering 2.1.5. Requirements for an Enhanced LFE Process 2.2. Technical Approaches in Language Engineering 2.2.1. Specification of Abstract Syntax 2.2.2. Specification of Concrete Syntax 2.2.3. Specification of Semantics 2.2.4. Requirements for an Enhanced LFE Technique 3. Feature-Oriented Language Family Engineering 3.1. Foundations of Feature-Oriented SPLE 3.1.1. Introduction to SPLE 3.1.2. Feature-Oriented Software Development 3.2. Feature-Oriented Language Family Engineering 3.2.1. Variability and Variant Specification in LFE 3.2.2. Product-Line Realisation, Mapping and Variant Derivation for LFE 3.3. Case Study: Scalability in Ontology Specification, Evaluation and Application 3.3.1. Review of Evolution, Customisation and Combination in the OWL LanguageFamily 3.3.2. Application of Feature-Oriented Language Family Engineering for OWL 3.4. Discussion 3.4.1. Contributions 3.4.2. Related Work. 3.4.3. Conclusion 4. Integrative, Role-Based Composition for Language Family Engineering 4.1. Foundations of Role-Based Modelling. 4.1.1. Information Hiding and Interface Specification in Role Models 4.1.2. Loose Coupling and Flexible Integration in Role Composition 4.2. The LanGems Language Composition System 4.2.1. The Language Component Specification Language . 4.2.2. TheLanguageCompositionLanguage 4.2.3. TechniquesofLanguageComposition 4.3. Case Study: Component-based OCL 4.3.1. Role-Based OCL Modularisation 4.3.2. Role-Based OCL Composition 4.4. Discussion 4.4.1. Contributions 4.4.2. Related Work 4.4.3. Conclusion 5. LFE with Integrative, Role-Based Syntax and Semantics Composition 5.1. Integrating Features and Roles 5.2. SumUp Case Study 5.2.1. Motivation 5.2.2. Feature-Oriented Variability and Variant Specification 5.2.3. Role-Based Component Realisation 5.2.4. Feature-Oriented Variability and Variant Evolution 5.2.5. Model-driven Concrete Syntax Realisation 5.2.6. Model-driven Semantics Realisation 5.2.7. Role-Based Composition and Feature Mapping 5.2.8. Language Variant Derivation 5.3. Conclusion 6. Conclusion 6.1. Contributions 6.2. Outlook 6.2.1. Co-Evolution in Language Families 6.2.2. Role-Based Tool Integration. 6.2.3. Automatic Modularisation of Existing Language Families 6.2.4. Language Component Library Appendix A Appendix B Bibliography

Software components provide essential means to structure and organize software effectively. However, frequently, required component abstractions are not available in a programming language or system, or are not adequately combinable with each other. Invasive software composition (ISC) is a general approach to software composition that unifies component-like abstractions such as templates, aspects and macros. ISC is based on fragment composition, and composes programs and other software artifacts at the level of syntax trees. Therefore, a unifying fragment component model is related to the context-free grammar of a language to identify extension and variation points in syntax trees as well as valid component types. By doing so, fragment components can be composed by transformations at respective extension and variation points so that always valid composition results regarding the underlying context-free grammar are yielded. However, given a language’s context-free grammar, the composition result may still be incorrect. Context-sensitive constraints such as type constraints may be violated so that the program cannot be compiled and/or interpreted correctly. While a compiler can detect such errors after composition, it is difficult to relate them back to the original transformation step in the composition system, especially in the case of complex compositions with several hundreds of such steps. To tackle this problem, this thesis proposes well-formed ISC—an extension to ISC that uses reference attribute grammars (RAGs) to specify fragment component models and fragment contracts to guard compositions with context-sensitive constraints. Additionally, well-formed ISC provides composition strategies as a means to configure composition algorithms and handle interferences between composition steps. Developing ISC systems for complex languages such as programming languages is a complex undertaking. Composition-system developers need to supply or develop adequate language and parser specifications that can be processed by an ISC composition engine. Moreover, the specifications may need to be extended with rules for the intended composition abstractions. Current approaches to ISC require complete grammars to be able to compose fragments in the respective languages. Hence, the specifications need to be developed exhaustively before any component model can be supplied. To tackle this problem, this thesis introduces scalable ISC—a variant of ISC that uses island component models as a means to define component models for partially specified languages while still the whole language is supported. Additionally, a scalable workflow for agile composition-system development is proposed which supports a development of ISC systems in small increments using modular extensions. All theoretical concepts introduced in this thesis are implemented in the Skeletons and Application Templates framework SkAT. It supports “classic”, well-formed and scalable ISC by leveraging RAGs as its main specification and implementation language. Moreover, several composition systems based on SkAT are discussed, e.g., a well-formed composition system for Java and a C preprocessor-like macro language. In turn, those composition systems are used as composers in several example applications such as a library of parallel algorithmic skeletons.

Software components provide essential means to structure and organize software effectively. However, frequently, required component abstractions are not available in a programming language or system, or are not adequately combinable with each other. Invasive software composition (ISC) is a general approach to software composition that unifies component-like abstractions such as templates, aspects and macros. ISC is based on fragment composition, and composes programs and other software artifacts at the level of syntax trees. Therefore, a unifying fragment component model is related to the context-free grammar of a language to identify extension and variation points in syntax trees as well as valid component types. By doing so, fragment components can be composed by transformations at respective extension and variation points so that always valid composition results regarding the underlying context-free grammar are yielded. However, given a language’s context-free grammar, the composition result may still be incorrect. Context-sensitive constraints such as type constraints may be violated so that the program cannot be compiled and/or interpreted correctly. While a compiler can detect such errors after composition, it is difficult to relate them back to the original transformation step in the composition system, especially in the case of complex compositions with several hundreds of such steps. To tackle this problem, this thesis proposes well-formed ISC—an extension to ISC that uses reference attribute grammars (RAGs) to specify fragment component models and fragment contracts to guard compositions with context-sensitive constraints. Additionally, well-formed ISC provides composition strategies as a means to configure composition algorithms and handle interferences between composition steps. Developing ISC systems for complex languages such as programming languages is a complex undertaking. Composition-system developers need to supply or develop adequate language and parser specifications that can be processed by an ISC composition engine. Moreover, the specifications may need to be extended with rules for the intended composition abstractions. Current approaches to ISC require complete grammars to be able to compose fragments in the respective languages. Hence, the specifications need to be developed exhaustively before any component model can be supplied. To tackle this problem, this thesis introduces scalable ISC—a variant of ISC that uses island component models as a means to define component models for partially specified languages while still the whole language is supported. Additionally, a scalable workflow for agile composition-system development is proposed which supports a development of ISC systems in small increments using modular extensions. All theoretical concepts introduced in this thesis are implemented in the Skeletons and Application Templates framework SkAT. It supports “classic”, well-formed and scalable ISC by leveraging RAGs as its main specification and implementation language. Moreover, several composition systems based on SkAT are discussed, e.g., a well-formed composition system for Java and a C preprocessor-like macro language. In turn, those composition systems are used as composers in several example applications such as a library of parallel algorithmic skeletons.