{"id":{"repo_id":"csusb","oai_identifier":"oai:scholarworks.lib.csusb.edu:etd-3118"},"canonical_url":"https://search.ndltd.org/etd/csusb/oai:scholarworks.lib.csusb.edu:etd-3118","repository":{"repo_id":"csusb","name":"CSUniversity San Bernardino","base_url":"https://scholarworks.lib.csusb.edu/do/oai/"},"display":{"title":"AN EMPIRICAL STUDY ON THE EFFICACY OF LLM-POWERED CHATBOTS IN BASIC INFORMATION RETRIEVAL TASKS","abstract":"The rise of conversational user interfaces (CUIs) powered by large language models (LLMs) is transforming human-computer interaction. This study evaluates the efficacy of LLM-powered chatbots, trained on website data, compared to browsing websites for finding information about organizations across diverse sectors. A within-subjects experiment with 165 participants was conducted, involving similar information retrieval (IR) tasks using both websites (GUIs) and chatbots (CUIs). The research questions are: (Q1) Which interface helps users find information faster: LLM chatbots or websites? (Q2) Which interface helps users find more accurate information: LLM chatbots or websites?. The findings are: (Q1) Participants found information significantly faster using LLM-chatbots, Q2. Participants found more accurate information using LLM-chatbots. The conclusions are: (Q1) LLM-chatbots are highly efficient, and (Q2). LLM-chatbots are highly reliable for information lookup tasks. These findings highlight the potential of LLM-powered CUIs to revolutionize user experience and advocate for integrating advanced AI capabilities in future interface design. Future research should investigate: 1. LLM-chatbot interaction speed over time to measure efficiency especially with more complex questions, 2. The precision of these models over larger knowledge bases and complex questions, 3. Improvements in chatbot’s usability and its impact on user experience and human computer interaction (HCI), and 4. Gauge user preference over prolonged interactions over more complex questions.","abstract_html":"The rise of conversational user interfaces (CUIs) powered by large language models (LLMs) is transforming human-computer interaction. This study evaluates the efficacy of LLM-powered chatbots, trained on website data, compared to browsing websites for finding information about organizations across diverse sectors. A within-subjects experiment with 165 participants was conducted, involving similar information retrieval (IR) tasks using both websites (GUIs) and chatbots (CUIs). The research questions are: (Q1) Which interface helps users find information faster: LLM chatbots or websites? (Q2) Which interface helps users find more accurate information: LLM chatbots or websites?. The findings are: (Q1) Participants found information significantly faster using LLM-chatbots, Q2. Participants found more accurate information using LLM-chatbots. The conclusions are: (Q1) LLM-chatbots are highly efficient, and (Q2). LLM-chatbots are highly reliable for information lookup tasks. These findings highlight the potential of LLM-powered CUIs to revolutionize user experience and advocate for integrating advanced AI capabilities in future interface design. Future research should investigate: 1. LLM-chatbot interaction speed over time to measure efficiency especially with more complex questions, 2. The precision of these models over larger knowledge bases and complex questions, 3. Improvements in chatbot’s usability and its impact on user experience and human computer interaction (HCI), and 4. Gauge user preference over prolonged interactions over more complex questions.","abstract_has_math":false,"creators":["Faysal, Naja"],"institution":null,"degree_name":"Master of Science in Information Systems and Technology","degree_level":"Project","degree_discipline":"Information and Decision Sciences","degree_department":null,"school":null,"contributors":["Shayo, Conrad"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:09:38Z","subjects":["Large Language Models (LLMs)","Conversational User Interfaces (CUIs)","Information Retrieval (IR)","User Experience (UX)","Graphical User Interfaces (GUIs)","Human-Computer Interaction (HCI)","Artificial Intelligence and Robotics","Databases and Information Systems","Graphic Design","Graphics and Human Computer Interfaces","Industrial and Product Design","Interactive Arts"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://scholarworks.lib.csusb.edu/etd/1938","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://scholarworks.lib.csusb.edu/etd/1938","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://scholarworks.lib.csusb.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Ascholarworks.lib.csusb.edu%3Aetd-3118","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Shayo, Conrad"]},{"key":"dc:creator","label":"Author","values":["Faysal, Naja"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2024-04-15T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2024-05-01T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Information and Decision Sciences"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Project"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science in Information Systems and Technology"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Large Language Models (LLMs)","Conversational User Interfaces (CUIs)","Information Retrieval (IR)","User Experience (UX)","Graphical User Interfaces (GUIs)","Human-Computer Interaction (HCI)","Artificial Intelligence and Robotics","Databases and Information Systems","Graphic Design","Graphics and Human Computer Interfaces","Industrial and Product Design","Interactive Arts"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://scholarworks.lib.csusb.edu/etd/1938"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The rise of conversational user interfaces (CUIs) powered by large language models (LLMs) is transforming human-computer interaction. This study evaluates the efficacy of LLM-powered chatbots, trained on website data, compared to browsing websites for finding information about organizations across diverse sectors. A within-subjects experiment with 165 participants was conducted, involving similar information retrieval (IR) tasks using both websites (GUIs) and chatbots (CUIs). The research questions are: (Q1) Which interface helps users find information faster: LLM chatbots or websites? (Q2) Which interface helps users find more accurate information: LLM chatbots or websites?. The findings are: (Q1) Participants found information significantly faster using LLM-chatbots, Q2. Participants found more accurate information using LLM-chatbots. The conclusions are: (Q1) LLM-chatbots are highly efficient, and (Q2). LLM-chatbots are highly reliable for information lookup tasks. These findings highlight the potential of LLM-powered CUIs to revolutionize user experience and advocate for integrating advanced AI capabilities in future interface design. Future research should investigate: 1. LLM-chatbot interaction speed over time to measure efficiency especially with more complex questions, 2. The precision of these models over larger knowledge bases and complex questions, 3. Improvements in chatbot’s usability and its impact on user experience and human computer interaction (HCI), and 4. Gauge user preference over prolonged interactions over more complex questions."]},{"key":"dc:title","label":"Title","values":["AN EMPIRICAL STUDY ON THE EFFICACY OF LLM-POWERED CHATBOTS IN BASIC INFORMATION RETRIEVAL TASKS"]}]}],"canonical_facts":{"dc:contributor":["Shayo, Conrad"],"dc:creator":["Faysal, Naja"],"dc:date.available":["2024-04-15T07:00:00Z"],"dc:date.created":["2024-05-01T07:00:00Z"],"dc:description.abstract":["The rise of conversational user interfaces (CUIs) powered by large language models (LLMs) is transforming human-computer interaction. This study evaluates the efficacy of LLM-powered chatbots, trained on website data, compared to browsing websites for finding information about organizations across diverse sectors. A within-subjects experiment with 165 participants was conducted, involving similar information retrieval (IR) tasks using both websites (GUIs) and chatbots (CUIs). The research questions are: (Q1) Which interface helps users find information faster: LLM chatbots or websites? (Q2) Which interface helps users find more accurate information: LLM chatbots or websites?. The findings are: (Q1) Participants found information significantly faster using LLM-chatbots, Q2. Participants found more accurate information using LLM-chatbots. The conclusions are: (Q1) LLM-chatbots are highly efficient, and (Q2). LLM-chatbots are highly reliable for information lookup tasks. These findings highlight the potential of LLM-powered CUIs to revolutionize user experience and advocate for integrating advanced AI capabilities in future interface design. Future research should investigate: 1. LLM-chatbot interaction speed over time to measure efficiency especially with more complex questions, 2. The precision of these models over larger knowledge bases and complex questions, 3. Improvements in chatbot’s usability and its impact on user experience and human computer interaction (HCI), and 4. Gauge user preference over prolonged interactions over more complex questions."],"dc:identifier":["https://scholarworks.lib.csusb.edu/etd/1938"],"dc:subject":["Large Language Models (LLMs)","Conversational User Interfaces (CUIs)","Information Retrieval (IR)","User Experience (UX)","Graphical User Interfaces (GUIs)","Human-Computer Interaction (HCI)","Artificial Intelligence and Robotics","Databases and Information Systems","Graphic Design","Graphics and Human Computer Interfaces","Industrial and Product Design","Interactive Arts"],"dc:title":["AN EMPIRICAL STUDY ON THE EFFICACY OF LLM-POWERED CHATBOTS IN BASIC INFORMATION RETRIEVAL TASKS"],"thesis:degree_discipline":["Information and Decision Sciences"],"thesis:degree_level":["Project"],"thesis:degree_name":["Master of Science in Information Systems and Technology"]},"updated_at":"2026-11-19T08:09:38Z"}