No description available.

No description available.

Deep neural language models like GPT-2 is undoubtedly strong at text generation, but often requires special decoding strategies to prevent producing degenerate output - namely repetition. The use of maximum likelihood training objective results in a peaked probability distribution, leading to the over-confidence of neural networks. In this thesis, we explore entropy regularization for a neural language model that can easily smooth peaked output distribution during the fine-tuning process employing GPT-2. We first define the models in three ways: (1) Out of-the box model without fine-tuning process, (2) Fine-tuned model without entropy regularization, and (3) Fine-tuned model with entropy regularization. To investigate the effect of domains on the model, we also divide the dataset into three ways: (1) fine-tuned on heterogeneous dataset, tested on heterogeneous dataset, (2) fine-tuned on homogeneous dataset, tested on homogeneous dataset, and (3) fine-tuned on heterogeneous dataset, tested on homogeneous dataset. In terms of entropy regularization, we experiment controlling the entropy strength parameter (𝛽) in the range of [0.5, 1.0, 2.0, 4.0, 6.0] and annealing the parameter during fine-tuning process. Our findings prove that the entropy-based regularization during fine-tuning process improve the text generation models by significantly reducing the repetition rate without tuning the decoding strategies. As a result of comparing the probabilities of human-generated sentence tokens, it was observed that entropy regularization compensates for the shortcomings of the deterministic decoding method (Beam search) that mostly selects few high-probability words. Various studies have explored entropy regularization in the cold-start training process of neural networks. However, there are not many studies covering the effect of the fine-tuning stage of text generation tasks when employing large scale pre-trained language models. Our findings present strong evidence that one can achieve significant improvement in text generation by way of utilizing entropy regularization, a highly cost-effective approach, during the fine-tuning process.

LibX is a browser extension that provides direct access to library resources. LibX enables users to add additional features to a webpage, such as placing a tutorial video on a digital library homepage. LibX achieves this ability of enhancing web pages through library applications, called LibApps. A LibApp examines a webpage, extracts and processes information of the page, and modifies the web content. It is possible to build an unlimited number of LibApps and enhance web pages in numerous ways. The developers of LibX team cannot build all possible LibApps by themselves. Hence, we decided to create an environment that allows users to create and share LibApps, thereby creating an eco-system of library applications. We developed the LibApp Builder, a cloud-based end-user programming tool that assists users in creating customized library applications with minimal effort. We designed an easy-to-understand meta-design language model with modularized, reusable components. The LibApp language model is designed to hide the complex programming details from the target audiences who are mostly non-technical users, primarily librarians. The LibApp Builder is a web-based editor that allows users to build and test LibApps in an executable environment. A built-in publishing mechanism allows users to group LibApps into packages and publish them in AtomPub format. Any user can directly reuse or adapt published components as required. Two error checking mechanisms have been built into the LibApp Builder viz., type checking and semantic checking to enhance user experience and reduce debugging effort. Additionally, the web interface displays help tooltips to guide users through the process of building a LibApp. We adhered to good software engineering practices such as the agile development model and the model-view-controller design paradigm. The LibApp Builder is built with the ZK AJAX framework and provides a rich interactive user interface. The LibApp Builder is integrated with an optimized full-text, fuzzy search engine and facilitates faceted search by exploiting the BaseX XML database system and XPath/XQuery processor. Users can locate and reuse existing language components through the search interface. To summarize, the LibApp Builder is a community platform for librarians to create, adapt and share LibApps. / Master of Science

This report outlines the research focused on finding the system design for Happymaker AI, a large language model with a mission to promote well-being at workplaces through daily interactions. The study includes a market analysis of relevant system components, such as database, cloud storage, cloud computing service and large language model, as well as the development of a prototype. Despite facing challenges including limited training data and resource constraints, the prototype was developed using the Llama 2 13B model which was quantized to 8-bits and fine-tuned using LoRA. Through research and prototyping of Happymaker AI, recommendations for the system design were established. These findings provide a foundation for the further development of an ethical AI system, specifically tailored for user data security and scalability. The findings also introduce a new perspective on empathy and personal well-being within the AI field, emphasizing the importance of integrating human-centric values into technological advancements. / Denna rapport skildrar forskningen som fokuserade på att hitta systemdesignen för Happymaker AI, en stor språkmodell med uppdraget att främja välmående på arbetsplatser genom dagliga interaktioner. Studien inkluderar en marknadsanalys av relevanta systemkomponenter såsom databas, molnlagring, molntjänster och en stor språkmodell, samt utvecklingen av en prototyp. Trots utmaningar, inklusive begränsad träningsdata och resursbegränsningar utvecklades prototypen med modellen Llama 2 13B som kvantiserades till 8-bit och tränades med LoRA. Genom forskning och prototypframtagning av Happymaker AI fastställdes rekommendationer för systemdesignen. Resultaten av studien ger en grund för vidareutveckling av ett etiskt AI-system som är anpassat för användardatasäkerhet och skalbarhet. Samtidigt introduceras ett nytt perspektiv på empati och personligt välmående inom AI-fältet, vilket betonar vikten av att integrera människocentrerade värderingar i teknologiska framsteg.

Introduction and Purpose – Emerging technologies such as GenAI are revolutionizing the business landscape and drastically changing the way organizations operate. As digital transformation accelerates, more and more organizations are using GenAI to streamline operations and strengthen their competitive position. Therefore, this study explores the enabling factors and challenges when implementing GenAI in the organizational settings. Furthermore, it also examines the driving factors and leveraging benefits of GenAI in digital transformation efforts.  Methodology – The study has an explorative qualitative research design with semi-structured interviews to gather data from different industries, and business areas to collect insights into the practical applications and challenges of GenAI. This approach allowed the authors to conduct an in-depth understanding of the context and complex phenomena, GenAI. Moreover, a theoretical framework was adapted and developed from the literature review that further guided the findings and analysis.  Findings and Analysis – The findings and analysis identified enabling factors for a successful implementation; Technological, Organizational and Employees, and challenges concerning; Ethics, Regulations and Skill Gaps. Hence, these factors can be both enablers and challenges, resonating with the findings that emphasize adaptability and responsiveness in digital transformation efforts. Moreover, responsible AI is still an uncertainty due to the rapid evolvement of the technology, which means that regulatory compliance does not keep up and can act as a barrier, or enabler. It is clear that GenAI is not a straightforward path, as several enabling factors need to be in place before scaling the technology into the organizational settings. However, organizations face challenges with technological infrastructure, data management, change management, and skill gaps. Lastly, the driving factors and leveraging benefits of GenAI stems from increased business value, divided into; Efficiency and Productivity Enhancements, Innovative Product and Service Development, Knowledge Management, Personal Assistant, and Data-Driven Insights.  Discussion and Conclusion – The discussion is central to this study, where the authors integrate theory and empirical findings to generate valuable contributions. Therefore, the most central elements merges and are further discussed; Technological Readiness, Organizational Dynamics, and Responsible AI, which resulted in the creation of a new framework that further guides the academic and practical discourse. Although GenAI facilitates significant value creation, efficiency and competitive advantage, organizations are often hampered by the lack of these factors in the pursuit of digital transformation. In conclusion, this study underlines the importance of understanding that there is not one single enabling factor that needs to be in place before an implementation, rather they need to coexist with each other for a successful integration, emphasizing the transformation where technological advances meet human skills. Additionally, the human interaction and monitoring is also crucial, by setting organizational policies and standards in the quest to adapt to new regulations and ethical standards.

<p dir="ltr">Deep learning models have demonstrated remarkable success across diverse domains, including computer vision and natural language processing. These models heavily rely on resources, encompassing annotated data, computational power, and storage. However, mobile devices, particularly in scenarios like medical or multilingual contexts, often face constraints with computing power, making ample data annotation prohibitively expensive. Developing deep learning models for such resource-constrained scenarios presents a formidable challenge. Our primary goal is to enhance the efficiency of state-of-the-art neural network models tailored for resource-limited scenarios. Our commitment lies in crafting algorithms that not only mitigate annotation requirements but also reduce computational complexity and alleviate storage demands. Our dissertation focuses on two key areas: Parameter-efficient Learning and Data-efficient Learning. In Part 1, we present our studies on parameter-efficient learning. This approach targets the creation of lightweight models for efficient storage or inference. The proposed solutions are tailored for diverse tasks, including text generation, text classification, and text/image retrieval. In Part 2, we showcase our proposed methods for data-efficient learning, concentrating on cross-lingual and multi-lingual text classification applications. </p>

An important challenge in natural language surface realization is the generation of grammatical sentences from incomplete sentence plans. Realization can be broken into a two-stage process consisting of an over-generating rule-based module followed by a ranker that outputs the most probable candidate sentence based on a statistical language model. Thus far, an n-gram language model has been evaluated in this context. More sophisticated syntactic knowledge is expected to improve such a ranker. In this thesis, a new language model based on featurized functional dependency syntax was developed and evaluated. Generation accuracies and cross-entropy for the new language model did not beat the comparison bigram language model.

Påskyndat av teknik går utvecklingen snabbare än någonsin. Generativ AI har blivit tillgänglig för allmänheten. Det ger möjligheter för verksamheter att nyttja AI-teknik utan större insatser och kunskap. Detta skiftar förutsättningarna inom digital innovation. Denna nya aktör skapar gap i litteraturen, där tidigare forskning behöver omvärderas. Ett viktigt forskningsområde är hur användningen av generativ AI påverkar digital innovation. En annan aspekt är hur innovatörer kan nyttja, och förhålla sig till generativ AI inom innovationsprocessen. För att undersöka detta har en kvalitativ studie genomförts, där empiri har samlats in genom åtta intervjuer. Studien har resulterat i en tematisk modell med följande teman: Generativ AI som en kollega; Generativ AI som resurs för digital innovation; Generativ AI ökar tillgängligheten till AI-teknik; Känslor gällande generativ AI; Problematik gällande generativ AI; Spridd och differentierad syn på digital innovation. Studien visar att generativ AI kan påverka digital innovation genom de resulterande temana. Vidare relateras dessa teman till innovationsprocessen, där en modifierad processmodell för innovation har tagits fram. Då användningen av generativ AI är ett relativt nytt fenomen är det sannolikt att innovatörer framöver kommer att öka sin användning av verktyget, vilket medför att fynden från denna studie riskerar att snabbt bli utdaterade. Vidare forskning bör därför utföra liknande studier med jämna mellanrum, för att fånga upp nya erfarenheter som uppstår av den ökade användningen. / Accelerated by technology, development is progressing faster than ever. Generative AI has become accessible to the general public. It provides opportunities for businesses to leverage AI technology without significant efforts and expertise. This shifts the conditions within digital innovation. This new actor creates gaps in the literature, where previous research needs to be reevaluated. An important research area is how the use of generative AI affects digital innovation. Another aspect is how innovators can utilize and engage with generative AI in the innovation process. To investigate this, a qualitative study has been conducted, where empirical data has been collected through eight interviews. The study has resulted in a thematic model with the following themes: Generative AI as a colleague; Generative AI as resource for digital innovation; Generative AI increases accessibility to AI technology; Emotions regarding generative AI; Challenges regarding generative AI; Diverse and differentiated views on digital innovation. The study shows that generative AI can affect digital innovation through the resulting themes. Furthermore, these themes were related to the innovation process, where a modified process model for innovation has been developed. Since the use of generative AI is a relatively new phenomenon, it is likely that innovators will increase their use of the tool in the future. This may render the findings from this study quickly outdated. Further research should therefore conduct similar studies at regular intervals to capture new experiences arising from increased usage.

<p>Brain-computer interfaces (BCI) have the potential to restore communication or control abilities in individuals with severe neuromuscular limitations, such as those with amyotrophic lateral sclerosis (ALS). The role of a BCI is to extract and decode relevant information that conveys a user's intent directly from brain electro-physiological signals and translate this information into executable commands to control external devices. However, the BCI decision-making process is error-prone due to noisy electro-physiological data, representing the classic problem of efficiently transmitting and receiving information via a noisy communication channel. </p><p>This research focuses on P300-based BCIs which rely predominantly on event-related potentials (ERP) that are elicited as a function of a user's uncertainty regarding stimulus events, in either an acoustic or a visual oddball recognition task. The P300-based BCI system enables users to communicate messages from a set of choices by selecting a target character or icon that conveys a desired intent or action. P300-based BCIs have been widely researched as a communication alternative, especially in individuals with ALS who represent a target BCI user population. For the P300-based BCI, repeated data measurements are required to enhance the low signal-to-noise ratio of the elicited ERPs embedded in electroencephalography (EEG) data, in order to improve the accuracy of the target character estimation process. As a result, BCIs have relatively slower speeds when compared to other commercial assistive communication devices, and this limits BCI adoption by their target user population. The goal of this research is to develop algorithms that take into account the physical limitations of the target BCI population to improve the efficiency of ERP-based spellers for real-world communication. </p><p>In this work, it is hypothesised that building adaptive capabilities into the BCI framework can potentially give the BCI system the flexibility to improve performance by adjusting system parameters in response to changing user inputs. The research in this work addresses three potential areas for improvement within the P300 speller framework: information optimisation, target character estimation and error correction. The visual interface and its operation control the method by which the ERPs are elicited through the presentation of stimulus events. The parameters of the stimulus presentation paradigm can be modified to modulate and enhance the elicited ERPs. A new stimulus presentation paradigm is developed in order to maximise the information content that is presented to the user by tuning stimulus paradigm parameters to positively affect performance. Internally, the BCI system determines the amount of data to collect and the method by which these data are processed to estimate the user's target character. Algorithms that exploit language information are developed to enhance the target character estimation process and to correct erroneous BCI selections. In addition, a new model-based method to predict BCI performance is developed, an approach which is independent of stimulus presentation paradigm and accounts for dynamic data collection. The studies presented in this work provide evidence that the proposed methods for incorporating adaptive strategies in the three areas have the potential to significantly improve BCI communication rates, and the proposed method for predicting BCI performance provides a reliable means to pre-assess BCI performance without extensive online testing.</p> / Dissertation