A strong capability of large language models is Chain-of-Thought reasoning. Prompting a model to ‘think step-by-step’ has led to great performance improvements in solving problems such as planning and question answering, and with the extended output it provides some evidence about the rationale behind an answer or decision. In search of better, more robust, and interpretable language model behavior, this work investigates self-reflection in large language models. Here, self-reflection consists of feedback from large language models to medical question-answering and whether the feedback can be used to accurately distinguish between correct and incorrect answers. GPT-3.5-Turbo and GPT-4 provide zero-shot feedback scores to Chain-of-Thought reasoning on the MedQA (medical questionanswering) dataset. The question-answering is evaluated on traits such as being structured, relevant and consistent. We test whether the feedback scores are different for questions that were either correctly or incorrectly answered by Chain-of-Thought reasoning. The potential differences in feedback scores are statistically tested with the Mann-Whitney U test. Graphical visualization and logistic regressions are performed to preliminarily determine whether the feedback scores are indicative to whether the Chain-of-Thought reasoning leads to the right answer. The results indicate that among the reasoning objectives, the feedback models assign higher feedback scores to questions that were answered correctly than those that were answered incorrectly. Graphical visualization shows potential for reviewing questions with low feedback scores, although logistic regressions that aimed to predict whether or not questions were answered correctly mostly defaulted to the majority class. Nonetheless, there seems to be a possibility for more robust output from self-reflecting language systems. / En stark förmåga hos stora språkmodeller är Chain-of-Thought-resonerande. Att prompta en modell att tänka stegvis har lett till stora prestandaförbättringar vid lösandet av problem som planering och frågebesvarande, och med den utökade outputen ger det en del bevis rörande logiken bakom ett svar eller beslut. I sökandet efter bättre, mer robust och tolk bart beteende hos språkmodeller undersöker detta arbete självreflektion i stora språkmodeller. Forskningsfrågan är: I vilken utsträckning kan feedback från stora språkmodeller, såsom GPT-3.5-Turbo och GPT-4, på ett korrekt sätt skilja mellan korrekta och inkorrekta svar i medicinska frågebesvarande uppgifter genom användningen av Chainof-Thought-resonerande? Här ger GPT-3.5-Turbo och GPT-4 zero-shot feedback-poäng till Chain-ofThought-resonerande på datasetet för MedQA (medicinskt frågebesvarande). Frågebesvarandet bör vara strukturerat, relevant och konsekvent. Feedbackpoängen jämförs mellan två grupper av frågor, baserat på om dessa besvarades korrekt eller felaktigt i första hand. Statistisk testning genomförs på skillnaden i feedback-poäng med Mann-Whitney U-testet. Grafisk visualisering och logistiska regressioner utförs för att preliminärt avgöra om feedbackpoängen är indikativa för huruvida Chainof-Thought-resonerande leder till rätt svar. Resultaten indikerar att bland resonemangsmålen tilldelar feedbackmodellerna fler positiva feedbackpoäng till frågor som besvarats korrekt än de som besvarats felaktigt. Grafisk visualisering visar potential för granskandet av frågor med låga feedbackpoäng, även om logistiska regressioner som syftade till att förutsäga om frågorna besvarades korrekt eller inte för det mesta majoritetsklassen. Icke desto mindre verkar det finnas potential för robustare från självreflekterande språksystem.

<p>Abstract</p><p>Chained Thoughts Broken by Chains of Thought</p><p>An Analysis of the Narrative Style Used in Virginia Woolf’s A Room of One’s Own</p><p>The purpose of this essay is to analyse the narrative style used in Virginia Woolf’s A Room of One’s Own in order to show in which ways it supports and reinforces the author’s arguments in her quest for a more equal society. One of the most prominent stylistic means applied by Woolf is her ‘train of thought’, linking one reflection to another like wagons in a railway convoy or like loops in a chain (therefore also sometimes referred to as ‘chain of thought’ in dictionaries). By examining how different rhetorical devices are applied within this train or chain of thought and in which ways these strategies are linked to the main elements of persuasion (ethos, pathos and logos) in Aristotelian Rhetoric, I have found that one of Woolf’s central themes - the resentment against confinement and the advocacy of androgyny or mixed-gendered thinking - is mirrored in her style. It reflects the author’s call to resist society’s restrictions by its unrestricted combination of different rhetorical strategies; this mixture of stylistic, partly gender-neutral devices helps her to create a common ground where she can reach and appeal to both genders in a very effective and innovative way, thus enabling her chain of thoughts to break some of our chained thoughts.</p><p>Ellen Johansson</p><p>Engelska C</p>

Abstract Chained Thoughts Broken by Chains of Thought An Analysis of the Narrative Style Used in Virginia Woolf’s A Room of One’s Own The purpose of this essay is to analyse the narrative style used in Virginia Woolf’s A Room of One’s Own in order to show in which ways it supports and reinforces the author’s arguments in her quest for a more equal society. One of the most prominent stylistic means applied by Woolf is her ‘train of thought’, linking one reflection to another like wagons in a railway convoy or like loops in a chain (therefore also sometimes referred to as ‘chain of thought’ in dictionaries). By examining how different rhetorical devices are applied within this train or chain of thought and in which ways these strategies are linked to the main elements of persuasion (ethos, pathos and logos) in Aristotelian Rhetoric, I have found that one of Woolf’s central themes - the resentment against confinement and the advocacy of androgyny or mixed-gendered thinking - is mirrored in her style. It reflects the author’s call to resist society’s restrictions by its unrestricted combination of different rhetorical strategies; this mixture of stylistic, partly gender-neutral devices helps her to create a common ground where she can reach and appeal to both genders in a very effective and innovative way, thus enabling her chain of thoughts to break some of our chained thoughts. Ellen Johansson Engelska C

Teaching AI to play Tabletop Roleplaying Games (TRPG) is a difficult challenge due their negotiable rules and open-ended nature. This is further exacerbated when considering the act of roleplaying, where players do not play or act as themselves, but as a character in a fantasy setting. Previous studies attempting to teach LLMs to play TRPGs do not explicitly discuss role-play in their work, highlighting an absence of a definition in current research. This thesis endeavours in introducing role-playing to AI through developing a prompting method called Control Chain of Thoughts, aimed at teaching it the Dungeons and Dragons alignment system. The prompting method is evaluated through an ablation study where GPT-3.5 is tasked to guess the alignment of characters based on exctracts from D&amp;D gaming sessions. The results indicate a small improvement in GPT’s predictions. Further work needs to be done to evaluate if its alignments help LLMs understand roleplaying.