This study is an investigation into the science literacy of college genetics students who were given a modified curriculum to address specific teaching and learning problems from a previous class. This study arose out of an interest by the professor and researcher to determine how well students in the class Human Genetics in the 21st Century responded to a reorganized curriculum to address misconceptions that were prevalent after direct instruction in the previous year's class. One of the components to the revised curriculum was the addition of a multimodal representation requirement as part of their normal writing assignments. How well students performed in these writing assignments and the relationship they had to student learning the rest of the class formed the principle research interest of this study. Improving science literacy has been a consistent goal of science educators and policy makers for over 50 years (DeBoer, 2000). This study uses the conceptualization of Norris and Phillips (2003) in which science literacy can be organized into both the fundamental sense (reading and writing) and the derived sense (experience and knowledge) of science literacy. The fundamental sense of science literacy was investigated in the students' ability to understand and use multimodal representations as part of their homework writing assignments. The derived sense of science literacy was investigated in how well students were able to apply their previous learning to class assessments found in quizzes and exams. This study uses a mixed-methods correlational design to investigate the relationship that existed between students' writing assignment experiences connected to multimodal representations and their academic performance in classroom assessments. Multimodal representations are pervasive in science literature and communication. These are the figures, diagrams, tables, pictures, mathematical equations, and any other form of content in which scientists and science educators are communicating ideas and concepts to their audience with more than simple text. A focused holistic rubric was designed in this study to score how well students in this class were able to incorporate aspects of multimodality into their writing assignments. Using these scores and factors within the rubric (ex. Number of original modes created) they were correlated with classroom performance scores to determine the strength and direction of the relationship. Classroom observations of lectures and discussion sections along with personal interviews with students and teaching assistants aided the interpretation of the results. The results from the study were surprisingly complex to interpret given the background of literature which suggested a strong relationship between multimodal representations and science learning (Lemke, 2000). There were significant positive correlations between student multimodal representations and quiz scores but not exam scores. This study was also confounded by significant differences between sections at the beginning of the study which may have led to learning effects later. The dissimilarity between the tasks of writing during their homework and working on exams may be the reason for no significant correlations with exams. The power to interpret these results was limited by the number of the participants, the number of modal experiences by the students, and the operationalization of multimodal knowledge through the holistic rubric. These results do show that a relationship does exist between the similar tasks within science writing and quizzes. Students may also gain derived science literacy benefits from modal experiences on distal tasks in exams as well. This study shows that there is still much more research to be known about the interconnectedness of multimodal representational knowledge and use to the development of science literacy.

Knowledge graphs (KGs) store relational information in a flexible triplet schema and have become ubiquitous for information storage in domains such as web search, e-commerce, social networks, and biology. Retrieval of information from KGs is generally achieved through logical reasoning, but this process can be computationally expensive and has limited performance due to the large size and complexity of relationships within the KGs. Furthermore, to extend the usage of KGs to non-expert users, retrieval over them cannot solely rely on logical reasoning but also needs to consider text-based search. This creates a need for multi-modal representations that capture both the semantic and structural features from the KGs. The primary objective of the proposed work is to extend the accessibility of KGs to non-expert users/institutions by enabling them to utilize non-technical textual queries to search over the vast amount of information stored in KGs. To achieve this objective, the research aims to solve four limitations: (i) develop a framework for logical reasoning over KGs that can learn representations to capture hierarchical dependencies between entities, (ii) design an architecture that can effectively learn the logic flow of queries from natural language text, (iii) create a multi-modal architecture that can capture inherent semantic and structural features from the entities and KGs, respectively, and (iv) introduce a novel hyperbolic learning framework to enable the scalability of hyperbolic neural networks over large graphs using meta-learning. The proposed work is distinct from current research because it models the logical flow of textual queries in hyperbolic space and uses it to perform complex reasoning over large KGs. The models developed in this work are evaluated on both the standard research setting of logical reasoning, as well as, real-world scenarios of query matching and search, specifically, in the e-commerce domain. In summary, the proposed work aims to extend the accessibility of KGs to non-expert users by enabling them to use non-technical textual queries to search vast amounts of information stored in KGs. To achieve this objective, the work proposes the use of multi-modal representations that capture both semantic and structural features from the KGs, and a novel hyperbolic learning framework to enable scalability of hyperbolic neural networks over large graphs. The work also models the logical flow of textual queries in hyperbolic space to perform complex reasoning over large KGs. The models developed in this work are evaluated on both the standard research setting of logical reasoning and real-world scenarios in the e-commerce domain. / Doctor of Philosophy / Knowledge graphs (KGs) are databases that store information in a way that allows computers to easily identify relationships between different pieces of data. They are widely used in domains such as web search, e-commerce, social networks, and biology. However, retrieving information from KGs can be computationally expensive, and relying solely on logical reasoning can limit their accessibility to non-expert users. This is where the proposed work comes in. The primary objective is to make KGs more accessible to non-experts by enabling them to use natural language queries to search the vast amounts of information stored in KGs. To achieve this objective, the research aims to address four limitations. Firstly, a framework for logical reasoning over KGs that can learn representations to capture hierarchical dependencies between entities is developed. Secondly, an architecture is designed that can effectively learn the logic flow of queries from natural language text. Thirdly, a multi-modal architecture is created that can capture inherent semantic and structural features from the entities and KGs, respectively. Finally, a novel hyperbolic learning framework is introduced to enable the scalability of hyperbolic neural networks over large graphs using meta-learning. The proposed work is unique because it models the logical flow of textual queries in hyperbolic space and uses it to perform complex reasoning over large KGs. The models developed in this work are evaluated on both the standard research setting of logical reasoning, as well as, real-world scenarios of query matching and search, specifically, in the e-commerce domain. In summary, the proposed work aims to make KGs more accessible to non-experts by enabling them to use natural language queries to search vast amounts of information stored in KGs. To achieve this objective, the work proposes the use of multi-modal representations that capture both semantic and structural features from the KGs, and a novel hyperbolic learning framework to enable scalability of hyperbolic neural networks over large graphs. The work also models the logical flow of textual queries in hyperbolic space to perform complex reasoning over large KGs. The results of this work have significant implications for the field of information retrieval, as it provides a more efficient and accessible way to retrieve information from KGs. Additionally, the multi-modal approach taken in this work has potential applications in other areas of machine learning, such as image recognition and natural language processing. The work also contributes to the development of hyperbolic geometry as a tool for modeling complex networks, which has implications for fields such as network science and social network analysis. Overall, this work represents an important step towards making the vast amounts of information stored in KGs more accessible and useful to a wider audience.

Consisting of both theoretical and empirical inquires, this study examines the primary functions of narrative and the relationship between narrative and mental imagery. The study proposes a new framework to interpret semiotic resources. Combining this with the linguistic functional theory of Halliday (1978), a functional method to empirically investigate semiotic representations was also developed. In the empirical inquiry, the study developed a latent construct method to empirically test the effects of narrative in a real learning situation. This study is the first to investigate the functional relationship between narrative and mental imagery, and among the first to suggest a theory and empirically investigate representations of a multimodal nature. The study is also among the first to use latent constructs to investigate the learning experience in a real educational setting. Data were collected from 190 library professionals who enrolled in three sections (two in narrative and one in plain text) of an online course administered through Vista 4.0 and who completed the course and responded to several instruments. Essay data (n = 82 x 2) were analyzed using content analysis based on the narrative analysis framework developed. Quantitative data analysis methods include univariate data analysis, factor analysis, and structural equation modeling that tests the proposed model and verifies the relationships between the latent variables. Overall, the findings support the hypotheses about the functional effects of narrative identified, and narrative is found to provide a favorable and positive learning context which is tested by the proposed model of learning experience measured by several latent constructs (X2 = 31.67, df = 47, p = .9577, RMSEA = .00, SRMR = .047, NNFI = 1.05, CFI = 1.00, and GFI = .94). The results indicate that participants who enrolled in the narrative sections of the course gained higher creative scores and showed better results in performance-based and attribution-based experiences. The model testing results indicate that even though more time spent during learning led to better outcome and performance in both groups, more time spent means more satisfaction for the individuals in the narrative group, but led to less satisfaction for the individuals in the non-narrative group.