{"id":{"repo_id":"calpoly","oai_identifier":"oai:digitalcommons.calpoly.edu:theses-4914"},"canonical_url":"https://search.ndltd.org/etd/calpoly/oai:digitalcommons.calpoly.edu:theses-4914","repository":{"repo_id":"calpoly","name":"Cal Poly","base_url":"https://digitalcommons.calpoly.edu/do/oai/"},"display":{"title":"Designing for Trust in Chat-Based Question Answering Systems: An Exchange-Based Retrieval Approach","abstract":"Community chat platforms such as Discord and Slack support spontaneous, collaborative communication but make it difficult to retrieve previously discussed information. As conversations accumulate, valuable exchanges become buried, leading to repeated questions and sustained burden on experienced community members. This work contributes a set of design requirements for question-answering systems operating over unstructured chat data, a Discord bot prototype implementing those requirements named Echo, and an empirical evaluation of how such a system affects user trust. Rather than encoding discrete question-answer pairs or generating synthetic responses with a language model, Echo indexes conversation topics for semantic retrieval and presents results as constructed exchanges of original messages attributed to their authors. This design preserves conversational context and grounds responses in source material. A user study evaluated how interaction with Echo affected trust-related perceptions of automated question-answering systems. Ten participants performed information-seeking tasks within a synthetic Discord server and completed trust surveys before and after interaction. All participants exhibited increased composite trust scores following use of the system, with a mean increase of 8.4 points. Findings suggest high perceived value in semantic retrieval over keyword search and in exchange-based responses, as well as the importance of source attribution in building user trust.","abstract_html":"Community chat platforms such as Discord and Slack support spontaneous, collaborative communication but make it difficult to retrieve previously discussed information. As conversations accumulate, valuable exchanges become buried, leading to repeated questions and sustained burden on experienced community members. This work contributes a set of design requirements for question-answering systems operating over unstructured chat data, a Discord bot prototype implementing those requirements named Echo, and an empirical evaluation of how such a system affects user trust. Rather than encoding discrete question-answer pairs or generating synthetic responses with a language model, Echo indexes conversation topics for semantic retrieval and presents results as constructed exchanges of original messages attributed to their authors. This design preserves conversational context and grounds responses in source material. A user study evaluated how interaction with Echo affected trust-related perceptions of automated question-answering systems. Ten participants performed information-seeking tasks within a synthetic Discord server and completed trust surveys before and after interaction. All participants exhibited increased composite trust scores following use of the system, with a mean increase of 8.4 points. Findings suggest high perceived value in semantic retrieval over keyword search and in exchange-based responses, as well as the importance of source attribution in building user trust.","abstract_has_math":false,"creators":["McCutchen, Nathan"],"institution":null,"degree_name":"MS in Computer Science","degree_level":null,"degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Ayaan Kazerouni","Computer Science","College of Engineering"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T07:12:51Z","subjects":["Question-answering","community chat platforms","Discord","knowledge retrieval","user trust","AI","Graphics and Human Computer Interfaces","Software Engineering"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.calpoly.edu/theses/3245","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.calpoly.edu/theses/3245","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.calpoly.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.calpoly.edu%3Atheses-4914","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Ayaan Kazerouni","Computer Science","College of Engineering"]},{"key":"dc:creator","label":"Author","values":["McCutchen, Nathan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2026-03-20T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2026-03-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Computer Science"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Question-answering","community chat platforms","Discord","knowledge retrieval","user trust","AI","Graphics and Human Computer Interfaces","Software Engineering"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.calpoly.edu/theses/3245"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Community chat platforms such as Discord and Slack support spontaneous, collaborative communication but make it difficult to retrieve previously discussed information. As conversations accumulate, valuable exchanges become buried, leading to repeated questions and sustained burden on experienced community members. This work contributes a set of design requirements for question-answering systems operating over unstructured chat data, a Discord bot prototype implementing those requirements named Echo, and an empirical evaluation of how such a system affects user trust. Rather than encoding discrete question-answer pairs or generating synthetic responses with a language model, Echo indexes conversation topics for semantic retrieval and presents results as constructed exchanges of original messages attributed to their authors. This design preserves conversational context and grounds responses in source material. A user study evaluated how interaction with Echo affected trust-related perceptions of automated question-answering systems. Ten participants performed information-seeking tasks within a synthetic Discord server and completed trust surveys before and after interaction. All participants exhibited increased composite trust scores following use of the system, with a mean increase of 8.4 points. Findings suggest high perceived value in semantic retrieval over keyword search and in exchange-based responses, as well as the importance of source attribution in building user trust."]},{"key":"dc:title","label":"Title","values":["Designing for Trust in Chat-Based Question Answering Systems: An Exchange-Based Retrieval Approach"]}]}],"canonical_facts":{"dc:contributor":["Ayaan Kazerouni","Computer Science","College of Engineering"],"dc:creator":["McCutchen, Nathan"],"dc:date.available":["2026-03-20T07:00:00Z"],"dc:date.created":["2026-03-01T08:00:00Z"],"dc:description.abstract":["Community chat platforms such as Discord and Slack support spontaneous, collaborative communication but make it difficult to retrieve previously discussed information. As conversations accumulate, valuable exchanges become buried, leading to repeated questions and sustained burden on experienced community members. This work contributes a set of design requirements for question-answering systems operating over unstructured chat data, a Discord bot prototype implementing those requirements named Echo, and an empirical evaluation of how such a system affects user trust. Rather than encoding discrete question-answer pairs or generating synthetic responses with a language model, Echo indexes conversation topics for semantic retrieval and presents results as constructed exchanges of original messages attributed to their authors. This design preserves conversational context and grounds responses in source material. A user study evaluated how interaction with Echo affected trust-related perceptions of automated question-answering systems. Ten participants performed information-seeking tasks within a synthetic Discord server and completed trust surveys before and after interaction. All participants exhibited increased composite trust scores following use of the system, with a mean increase of 8.4 points. Findings suggest high perceived value in semantic retrieval over keyword search and in exchange-based responses, as well as the importance of source attribution in building user trust."],"dc:identifier":["https://digitalcommons.calpoly.edu/theses/3245"],"dc:subject":["Question-answering","community chat platforms","Discord","knowledge retrieval","user trust","AI","Graphics and Human Computer Interfaces","Software Engineering"],"dc:title":["Designing for Trust in Chat-Based Question Answering Systems: An Exchange-Based Retrieval Approach"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_name":["MS in Computer Science"]},"updated_at":"2026-11-19T07:12:51Z"}