In real-world information retrieval (IR) tasks, the searched items and/or the users' queries often have implicit information associated with them -- information that describes unspecified aspects of the items or queries. For example, in web search tasks, web pages are often pointed to by hyperlinks (known as anchors) from other pages, and thus have human-generated succinct descriptions of their content (anchor text) associated with them. This indirectly available information has been shown to improve search effectiveness for different retrieval tasks. However, in many real-world IR challenges this information is sparse in the data; i.e., it is incomplete or missing in a large portion of the data. In this work, we explore how to discover and use implicit information in large amounts of data in the context of IR. We present a general perspective for discovering implicit information and demonstrate how to use the discovered data in four specific IR challenges: (1) finding relevant records in semi-structured databases where many records contain incomplete or empty fields; (2) searching web pages that have little or no associated anchor text; (3) using click-through records in web query logs to help search pages that have no or very few clicks; and (4) discovering plausible geographic locations for web queries that contain no explicit geographic information. The intuition behind our approach is that data similar in some aspects are often similar in other aspects. Thus we can (a) use the observed information of queries/documents to find similar queries/documents, and then (b) utilize those similar queries/documents to reconstruct plausible implicit information for the original queries/documents. We develop language modeling based techniques to effectively use content similarity among data for our work. Using the four different search tasks on large-scale noisy datasets, we empirically demonstrate the effectiveness of our approach. We further discuss the advantages and weaknesses of two complementary approaches within our general perspective of handling implicit information for retrieval purpose. Taken together, we describe a general perspective that uses contextual similarity among data to discover implicit information for IR challenges. Using this general perspective, we formally present two language modeling based information discovery approaches. We empirically evaluate our approaches using different IR challenges. Our research shows that supporting information discovery tailored to different search tasks can enhance IR systems' search performance and improve users' search experience.

No description available.

No description available.

Deep neural language models like GPT-2 is undoubtedly strong at text generation, but often requires special decoding strategies to prevent producing degenerate output - namely repetition. The use of maximum likelihood training objective results in a peaked probability distribution, leading to the over-confidence of neural networks. In this thesis, we explore entropy regularization for a neural language model that can easily smooth peaked output distribution during the fine-tuning process employing GPT-2. We first define the models in three ways: (1) Out of-the box model without fine-tuning process, (2) Fine-tuned model without entropy regularization, and (3) Fine-tuned model with entropy regularization. To investigate the effect of domains on the model, we also divide the dataset into three ways: (1) fine-tuned on heterogeneous dataset, tested on heterogeneous dataset, (2) fine-tuned on homogeneous dataset, tested on homogeneous dataset, and (3) fine-tuned on heterogeneous dataset, tested on homogeneous dataset. In terms of entropy regularization, we experiment controlling the entropy strength parameter (𝛽) in the range of [0.5, 1.0, 2.0, 4.0, 6.0] and annealing the parameter during fine-tuning process. Our findings prove that the entropy-based regularization during fine-tuning process improve the text generation models by significantly reducing the repetition rate without tuning the decoding strategies. As a result of comparing the probabilities of human-generated sentence tokens, it was observed that entropy regularization compensates for the shortcomings of the deterministic decoding method (Beam search) that mostly selects few high-probability words. Various studies have explored entropy regularization in the cold-start training process of neural networks. However, there are not many studies covering the effect of the fine-tuning stage of text generation tasks when employing large scale pre-trained language models. Our findings present strong evidence that one can achieve significant improvement in text generation by way of utilizing entropy regularization, a highly cost-effective approach, during the fine-tuning process.

LibX is a browser extension that provides direct access to library resources. LibX enables users to add additional features to a webpage, such as placing a tutorial video on a digital library homepage. LibX achieves this ability of enhancing web pages through library applications, called LibApps. A LibApp examines a webpage, extracts and processes information of the page, and modifies the web content. It is possible to build an unlimited number of LibApps and enhance web pages in numerous ways. The developers of LibX team cannot build all possible LibApps by themselves. Hence, we decided to create an environment that allows users to create and share LibApps, thereby creating an eco-system of library applications. We developed the LibApp Builder, a cloud-based end-user programming tool that assists users in creating customized library applications with minimal effort. We designed an easy-to-understand meta-design language model with modularized, reusable components. The LibApp language model is designed to hide the complex programming details from the target audiences who are mostly non-technical users, primarily librarians. The LibApp Builder is a web-based editor that allows users to build and test LibApps in an executable environment. A built-in publishing mechanism allows users to group LibApps into packages and publish them in AtomPub format. Any user can directly reuse or adapt published components as required. Two error checking mechanisms have been built into the LibApp Builder viz., type checking and semantic checking to enhance user experience and reduce debugging effort. Additionally, the web interface displays help tooltips to guide users through the process of building a LibApp. We adhered to good software engineering practices such as the agile development model and the model-view-controller design paradigm. The LibApp Builder is built with the ZK AJAX framework and provides a rich interactive user interface. The LibApp Builder is integrated with an optimized full-text, fuzzy search engine and facilitates faceted search by exploiting the BaseX XML database system and XPath/XQuery processor. Users can locate and reuse existing language components through the search interface. To summarize, the LibApp Builder is a community platform for librarians to create, adapt and share LibApps. / Master of Science

<p dir="ltr">Deep learning models have demonstrated remarkable success across diverse domains, including computer vision and natural language processing. These models heavily rely on resources, encompassing annotated data, computational power, and storage. However, mobile devices, particularly in scenarios like medical or multilingual contexts, often face constraints with computing power, making ample data annotation prohibitively expensive. Developing deep learning models for such resource-constrained scenarios presents a formidable challenge. Our primary goal is to enhance the efficiency of state-of-the-art neural network models tailored for resource-limited scenarios. Our commitment lies in crafting algorithms that not only mitigate annotation requirements but also reduce computational complexity and alleviate storage demands. Our dissertation focuses on two key areas: Parameter-efficient Learning and Data-efficient Learning. In Part 1, we present our studies on parameter-efficient learning. This approach targets the creation of lightweight models for efficient storage or inference. The proposed solutions are tailored for diverse tasks, including text generation, text classification, and text/image retrieval. In Part 2, we showcase our proposed methods for data-efficient learning, concentrating on cross-lingual and multi-lingual text classification applications. </p>

Introduction and Purpose – Emerging technologies such as GenAI are revolutionizing the business landscape and drastically changing the way organizations operate. As digital transformation accelerates, more and more organizations are using GenAI to streamline operations and strengthen their competitive position. Therefore, this study explores the enabling factors and challenges when implementing GenAI in the organizational settings. Furthermore, it also examines the driving factors and leveraging benefits of GenAI in digital transformation efforts.  Methodology – The study has an explorative qualitative research design with semi-structured interviews to gather data from different industries, and business areas to collect insights into the practical applications and challenges of GenAI. This approach allowed the authors to conduct an in-depth understanding of the context and complex phenomena, GenAI. Moreover, a theoretical framework was adapted and developed from the literature review that further guided the findings and analysis.  Findings and Analysis – The findings and analysis identified enabling factors for a successful implementation; Technological, Organizational and Employees, and challenges concerning; Ethics, Regulations and Skill Gaps. Hence, these factors can be both enablers and challenges, resonating with the findings that emphasize adaptability and responsiveness in digital transformation efforts. Moreover, responsible AI is still an uncertainty due to the rapid evolvement of the technology, which means that regulatory compliance does not keep up and can act as a barrier, or enabler. It is clear that GenAI is not a straightforward path, as several enabling factors need to be in place before scaling the technology into the organizational settings. However, organizations face challenges with technological infrastructure, data management, change management, and skill gaps. Lastly, the driving factors and leveraging benefits of GenAI stems from increased business value, divided into; Efficiency and Productivity Enhancements, Innovative Product and Service Development, Knowledge Management, Personal Assistant, and Data-Driven Insights.  Discussion and Conclusion – The discussion is central to this study, where the authors integrate theory and empirical findings to generate valuable contributions. Therefore, the most central elements merges and are further discussed; Technological Readiness, Organizational Dynamics, and Responsible AI, which resulted in the creation of a new framework that further guides the academic and practical discourse. Although GenAI facilitates significant value creation, efficiency and competitive advantage, organizations are often hampered by the lack of these factors in the pursuit of digital transformation. In conclusion, this study underlines the importance of understanding that there is not one single enabling factor that needs to be in place before an implementation, rather they need to coexist with each other for a successful integration, emphasizing the transformation where technological advances meet human skills. Additionally, the human interaction and monitoring is also crucial, by setting organizational policies and standards in the quest to adapt to new regulations and ethical standards.

This report outlines the research focused on finding the system design for Happymaker AI, a large language model with a mission to promote well-being at workplaces through daily interactions. The study includes a market analysis of relevant system components, such as database, cloud storage, cloud computing service and large language model, as well as the development of a prototype. Despite facing challenges including limited training data and resource constraints, the prototype was developed using the Llama 2 13B model which was quantized to 8-bits and fine-tuned using LoRA. Through research and prototyping of Happymaker AI, recommendations for the system design were established. These findings provide a foundation for the further development of an ethical AI system, specifically tailored for user data security and scalability. The findings also introduce a new perspective on empathy and personal well-being within the AI field, emphasizing the importance of integrating human-centric values into technological advancements. / Denna rapport skildrar forskningen som fokuserade på att hitta systemdesignen för Happymaker AI, en stor språkmodell med uppdraget att främja välmående på arbetsplatser genom dagliga interaktioner. Studien inkluderar en marknadsanalys av relevanta systemkomponenter såsom databas, molnlagring, molntjänster och en stor språkmodell, samt utvecklingen av en prototyp. Trots utmaningar, inklusive begränsad träningsdata och resursbegränsningar utvecklades prototypen med modellen Llama 2 13B som kvantiserades till 8-bit och tränades med LoRA. Genom forskning och prototypframtagning av Happymaker AI fastställdes rekommendationer för systemdesignen. Resultaten av studien ger en grund för vidareutveckling av ett etiskt AI-system som är anpassat för användardatasäkerhet och skalbarhet. Samtidigt introduceras ett nytt perspektiv på empati och personligt välmående inom AI-fältet, vilket betonar vikten av att integrera människocentrerade värderingar i teknologiska framsteg.

In recent years, the technological space experienced the proliferation of Generative AI models. A prominent type of this model is a language model-based chatbot. The primary function of these models is to generate answers to a question from an extensive database and create a stream of conversation at various levels of complexity. The database of these models encompasses diverse data type of text (e.g., ChatGPT), audio (e.g., PlayHT), or images (e.g., DALLE-2). The intricate process involves neural networks, which undergoes pre-training from the database, building result from architecture neural networks, refined tuning to creating coherent result, probability estimation to produce the correct context result, and generating and refinement as improvement to generated answers. This proposal aims to delve deep into the probability estimation process of the generative AI model. A specific focus is to predict an airborne object's trajectory to create an understanding of how to adapt and adjust robotic limbs and enable them to intercept and capture the thing with some degree of precision.

An important challenge in natural language surface realization is the generation of grammatical sentences from incomplete sentence plans. Realization can be broken into a two-stage process consisting of an over-generating rule-based module followed by a ranker that outputs the most probable candidate sentence based on a statistical language model. Thus far, an n-gram language model has been evaluated in this context. More sophisticated syntactic knowledge is expected to improve such a ranker. In this thesis, a new language model based on featurized functional dependency syntax was developed and evaluated. Generation accuracies and cross-entropy for the new language model did not beat the comparison bigram language model.