TITLE: Concept maps as an assessment tool in education SUMMARY: This thesis thesis is concerned with the types of assessment of concept maps, creation on the bases of studying professional literature. It points out different views and perceptions of assessment methods and offers its own assessment methods. By way of research evaluates its own methods for assessing concept maps, and then explores the possibilities of evaluation of concept maps (in contrast with the written exam, objectivity of assessing methods) for students of upper primary school in the three selected subjects: geography, citizenship education,science. KEYWORDS: Concept map, assessment

The purpose of this study was to examine two scaffolding methods on the performance of students in computer-based concept linking and retention of comprehension. After training and practice in concept mapping and CmapTools--a computer-based concept mapping program, 116 undergraduate students were randomly assigned to one of four treatment groups to work on a computer-based concept mapping task. Students in the no scaffolding (NS) group did not receive any scaffolding. Students in the linking phrase scaffolding (PS) group received linking words or phrases as scaffolding. Students in the articulation hint scaffolding (AS) group received a hint question as scaffolding, which asked them to elaborate on relationships between concepts in full sentences. Students in the linking phrase and articulation hint scaffolding (PAS) group received both scaffolding while working on the computer-based concept mapping task. One week after the treatment, students took a concept linking posttest, in which they constructed a concept map in CmapTools based on a web-based instruction on the human heart. After another week, they took another posttest on retention of comprehension about the heart. Two 2 X 2 factorial Analysis of Variance (ANOVA) were conducted to examine the main effects of linking phrase scaffolding and articulation hint scaffolding and any interaction effect between them on the performance of students in computer-based concept linking and retention of comprehension. The results showed no significant difference in the performance of students in both tests. However, the Pearson's correlation analysis showed that there was a positive correlation between students' performance in computer-based concept linking and retention of comprehension (γ = 0.447, p < 0.01). / Ph. D.

The problems existent within the built environment are inherently complex due to the interactions between different stakeholders, structures, and systems. The reductionist approach vastly utilized by engineers is not appropriate for dealing with this complexity. Engineers need to be trained to think in systems in order to fully explore the design problem space and therefore identify appropriate design solutions. The study here presented investigates the possibility of the use of concept mapping as an intervention to prime engineering students to think in systems. In the study, 66 engineering students were given two problem framing tasks. Half of the sample received the priming intervention before each task. The control and the intervention group were compared across different metrics. The time spent on the task and length of responses were used as measures of cognitive effort. The number of systems mentioned and the semantic distance between words used in each response were the metrics used for exploration of the design space. Results of the analysis for one of the tasks were significant. The findings suggest that the participants who received the concept mapping priming intervention were able to sustain cognitive effort longer and explore a wider design problem space. / Master of Science / The problems existent within the built environment present interdependencies that need to be identified before suitable solutions can be designed. Engineers need to be able to identify and understand these complex relationships. However, engineers are instead trained and prompted to apply a reductionist approach to problem solving, which isolates parts of a system in order to reduce complexity and facilitate the design process. Concept maps, a graphical tool utilized to display the relationships between concepts and ideas in a hierarchical form, could be used to assist engineers on applying a more holistic approach to problem solving. This research investigates if concept mapping activities can affect engineering students' ability to think in systems and consider all the variables behind a design problem. Participants in the study had to identify and describe everything that could be improved about two different systems familiar to Virginia Tech students. Half of the participants were asked to draw a concept map about each system before each task. All responses were compared between the group that did the concept mapping activity and the group that did not do it. The length in time and words of the responses, the number of systems mentioned, and the originality of the words used by each participant were the metrics utilized to compare the groups. Results suggest that concept mapping can be used as a tool to assist engineering students explore the design problem space more fully.

Grand challenges in engineering are complex and require engineers to be cognizant of different systems associated with each problem. The approach to think about these systems is called systems thinking. Systems thinking provides engineers with a lens to identify relationships between multiple components which helps them develop new ideas about the problem. Concept maps are a tool that enables systems thinking by helping engineers organize ideas and the relationship between ideas, graphically. The research presented in this thesis uses concept maps, as an intervention to help engineering students think in systems and, in turn, shape how they frame their design problem. The aim of the research was to understand the neurocognitive effects of engineering students thinking in systems. The effects of systems thinking on neurocognition is not well understood. Sixty-six engineering students were randomly chosen to either draw concept maps about a design problem or not. They were then asked to develop design problem statements for two design problems. Functional near-infrared spectroscopy (fNIRS) was used to measure changes in oxy-hemoglobin (oxy-Hb) in the prefrontal cortex (PFC) of students while they developed their design problem statements. A lower average oxy-Hb was observed in the group that was first asked to develop concept maps. The lower activation was observed in their left PFC. The group of students who first developed concept maps also demonstrated lower network connections between brain regions in the prefrontal cortex, which is a proxy for functional coordination. Using concept maps changed activation in students' brains, reducing the average neuro-cognition in the left PFC and reducing the need for functional coordination between brain regions. / Master of Science / Engineering challenges require engineers to think "outside the box". Concept mapping is a tool that encourages out of the box thinking. Concept mapping is the process of representing components of the problem and the relationship between components graphically. How the process of concept mapping changes the way engineers think is not well understood. Exploring various interconnected system components and their relationships may give rise to new ideas and this may be expressed differently in the brain. The research presented in this thesis explores how concept maps change engineering students' brain behavior. Sixty-six students participated in the study. Half of the participants (the intervention group) were required to draw concept maps before developing two engineering problem statements. The other half (the control group) were given the same two tasks to develop engineering problem statements but without being asked to first develop concept maps. A neuroimaging tool, called functional near-infrared spectroscopy, was used to measure change in the engineering students' prefrontal cortex (PFC) when they were developing problem statements. The PFC is generally associated with executive functions like planning, design, and creative thinking. The results indicate that concept mapping significantly changed brain behavior when developing problem statements. It reduced brain activation in the left PFC, a region generally associated with making analytical judgments and goal-directed planning. It also reduced the network complexity in the PFC, which is a proxy for functional connectivity. These results demonstrate how concept mapping can shape brain behavior when designing and lays the groundwork for future studies to explore how other interventions similar to concept mapping can help shape design thinking.

The development of spatial thinking proficiency has been increasingly demanded in Geographic Information System (GIS) education. Despite this educational trend, there is little empirical research on college students' spatial concept knowledge, which critically affects the quality of spatial thinking. This study addressed the following three research questions: 1) What differences exist between students' understandings of spatial concepts at the beginning, middle, and end of an introductory-level GIS course?, 2) What spatial misconceptions students may possess while taking an introductory-level GIS course?, and 3) Which spatial concepts are easy or hard for undergraduate students to understand? The researcher asked twelve participants who were taking an introductory-level GIS course to create concept maps about space and revised their concept maps in three experiment sessions. For the first question, the researcher scored the sixty obtained concept maps and statistically analyzed those scores to examine if there is any significant difference among the scores of the three experiment sessions. For the second question, the researcher examined participants' misconceptions by analyzing the incorrect statements of distortion, map projection, and scale. For the third question, the researcher statistically analyzed concept-based scores to examine if there is any significant difference among the scores of three different complexity levels. A main finding for the first question was that there was a significant difference among the scores of the concept maps created in the first session and the scores of the concept maps revised in the second and third sessions. This implied that participants could successfully revise their own original concept maps in the middle of a semester. The result of the study of the second question indicated that a half of participants misunderstood the concepts of map projections and scale. This result suggested that some undergraduate students may have difficulty shifting from scientifically inappropriate spatial concept knowledge to appropriate knowledge. Analysis of the third question resulted that the concept-based scores of simple spatial concepts are significantly higher than the scores of complicated spatial concepts. This result inferred that participants' scores decreased as the complexity of the concepts increased.

Made available in DSpace on 2018-08-02T00:03:48Z (GMT). No. of bitstreams: 1 tese_11160_CamilaZacche_dissertacao_final.pdf: 5437260 bytes, checksum: 0c96c6b2cce9c15ea234627fad78ac9a (MD5) Previous issue date: 2017-03-31 / 8 Resumo Os mapas conceituais são ferramentas gráficas para a representação e construção do conhecimento. Conceitos e relações formam a base para o aprendizado e, portanto, os mapas conceituais têm sido amplamente utilizados em diferentes situações e para diferentes propósitos na educação, sendo uma delas a represent ação do texto escrito. Mes mo um gramá tico e complexo texto pode ser representado por um mapa conceitual contendo apenas conceitos e relações que represente m o que foi expresso de uma forma mais complicada. No entanto, a construção manual de um mapa conceit ual exige bastante tempo e esforço na identificação e estruturação do conhecimento, especialmente quando o mapa não deve representar os conceitos da estrutura cognitiva do autor. Em vez disso, o mapa deve representar os conceitos expressos em um texto. Ass im, várias abordagens tecnológicas foram propostas para facilitar o processo de construção de mapas conceituais a partir de textos. Portanto, esta dissertação propõe uma nova abordagem para a construção automática de mapas conceituais como sumarização de t extos científicos. A sumarização pretende produzir um mapa conceitual como uma representação resumida do texto, mantendo suas diversas e mais importantes características. A sumarização pode facilitar a compreensão dos textos, uma vez que os alunos estão te ntando lidar com a sobrecarga cognitiva causada pela crescente quantidade de informação textual disponível atualmente. Este crescimento também pode ser prejudicial à construção do conhecimento. Assim, consideramos a hipótese de que a sumarização de um text o representado por um mapa conceitual pode atribuir características importantes para assimilar o conhecimento do texto, bem como diminuir a sua complexidade e o tempo necessário para processá - lo. Neste contexto, realizamos uma revisão da literatura entre o s anos de 1994 e 2016 sobre as abordagens que visam a construção automática de mapas conceituais a partir de textos. A partir disso, construímos uma categorização para melhor identificar e analisar os recursos e as características dessas abordagens tecnoló gicas. Além disso, buscamos identificar as limitações e reunir as melhores características dos trabalhos relacionados para propor nossa abordagem. 9 Ademais, apresentamos um processo Concept Map Mining elaborado seguindo quatro dimensões : Descrição da Fonte de Dados, Definição do Domínio, Identificação de Elementos e Visualização do Mapa. Com o intuito de desenvolver uma arquitetura computacional para construir automaticamente mapas conceituais como sumarização de textos acadêmicos, esta pesquisa resultou na ferramenta pública CMBuilder , uma ferramenta online para a construção automática de mapas conceituais a partir de textos, bem como uma api java chamada ExtroutNLP , que contém bibliotecas para extração de informações e serviços públicos. Para alcançar o objetivo proposto, direcionados esforços para áreas de processamento de linguagem natural e recuperação de informação. Ressaltamos que a principal tarefa para alcançar nosso objetivo é extrair do texto as proposições do tipo ( conceito, rela ção, conceito ). Sob essa premissa, a pesquisa introduz um pipeline que compreende: regras gramaticais e busca em profundidade para a extração de conceitos e relações a partir do texto; mapeamento de preposição, resolução de anáforas e exploração de entidad es nomeadas para a rotulação de conceitos; ranking de conceitos baseado na análise de frequência de elementos e na topologia do mapa; e sumarização de proposição baseada na topologia do grafo. Além disso, a abordagem também propõe o uso de técnicas de apre ndizagem supervisionada de clusterização e classificação associadas ao uso de um tesauro para a definição do domínio do texto e construção de um vocabulário conceitual de domínios. Finalmente, uma análise objetiva para validar a exatidão da biblioteca Extr outNLP é executada e apresenta 0.65 precision sobre o corpus . Além disso, uma análise subjetiva para validar a qualidade do mapa conceitual construído pela ferramenta CMBuilder é realizada , apresentando 0.75/0.45 para precision / recall de conceitos e 0.57/ 0.23 para precision/ recall de relações em idioma inglês e apresenta ndo 0.68/ 0.38 para precision/ recall de conceitos e 0.41/ 0.19 para precision/ recall de relações em idioma português. Ademais , um experimento para verificar se o mapa conceitual sumarizado pe lo CMBuilder tem influência para a compreensão do assunto abordado em um texto é realizado , atingindo 60% de acertos para mapas extraídos de pequenos textos com questões de múltipla escolha e 77% de acertos para m apas extraídos de textos extensos com quest ões discursivas

Epidemiological studies are rich in information that could improve the understanding of concept complexity of a health problem, and are important sources for evidence based medicine. However, epidemiologists experience difficulties in recognising and aggregating key characteristics in related research due to an increasing number of published articles. The main aim of this dissertation is to explore how text mining techniques can assist epidemiologists to identify important pieces of information and detect and integrate key knowledge for further research and exploration via concept maps. Concept maps are widely used in medicine for exploration and representation as a relatively formal, easy to design and understand knowledge representation model. To support this aim, we have developed a methodology for the extraction of key epidemiological characteristics from all types of epidemiological research articles in order to visualise, explore and aggregate concepts related to a health care problem. A generic rule-based approach was designed and implemented for the identification of mentions of six key characteristics, including study design, population, exposure, outcome, covariate and effect size. The system also relies on automatic term recognition and biomedical dictionaries to identify concepts of interests. In order to facilitate knowledge integration and aggregation, extracted characteristics are further normalized and mapped to existing resources. Study design mentions are mapped to an expanded version of the Ontology of Clinical Research (OCRe), whereas exposure, outcome and covariate mentions are mapped to Unified Medical Language System (UMLS) semantic groups and categories. Population mentions are mapped to age groups, gender and nationality/ethnicity, and effect size mentions are normalised with the regards to the used metric and confidence interval and related concept. The evaluation has shown reliable results, with an average micro F-score of 87% for recognition of epidemiological mentions and 91% for normalisation. Normalised concepts are further organised in an automatically generated concept map, which has three sections for exposures, outcomes and covariates. To demonstrate the potential of the developed methodology, it was applied to a large-scale corpus of epidemiological research abstracts related to obesity. Obesity was chosen as a case study since it has emerged as one of the most important global health problems of the 21st century. Using the concepts extracted from the corpus, we have built a searchable database of key epidemiological characteristics explored in obesity and an automatically generated concept map represented the normalized exposures, outcomes and covariates. An epidemiological workbench (EpiTeM) was designed to enable further exploration and inspection of the normalized extracted data, with direct links to the literature. The generated results also allow exploration of trends in obesity research and can facilitate understanding of its concept complexity. For example, we have noted the most frequent concepts and the most common pairs of characteristics that have been studied in obesity epidemiology. Finally, this thesis also discusses a number of challenges for text mining of epidemiological literature and suggests various opportunities for future work.

The purpose of this study is to explore the feasibility and effectiveness of a scalable concept map based navigation system for a digital textbook. A literature review has been conducted to identify possible methods to promote conceptual understanding in the context of a digital textbook, and these hypothesized solutions will be evaluated through a prototype tool. The primary method that has been selected for this study to promote conceptual understanding in textbooks is the concept map. When concept maps are used as advance organizers or navigation aids for hypermedia documents, they have been shown to promote conceptual understanding. Issues with scalability exist, however. When maps become too large or complicated, a phenomenon labeled "map-shock" occurs. Map-shock is a result of cognitive overload that nullifies the positive effects the concept map has on learning. In order to eliminate map-shock, one needs to manage the cognitive load imposed on the learner. This project proposes using information visualization techniques leveraged from the computer science domain to develop an interactive concept map based navigation system that will retain the positive effects of concept maps, and also present the visuals in a way that does not cognitively overload the user. This study seeks to answer the research question: "How can a large-scale concept map visualization tools be realized in a way that promotes conceptual understanding and manages cognitive load?" To answer the research question, a prototype tool, labeled the "Adaptive Map tool", was developed and populated with engineering statics content. This prototype contains content that is similar to the material in a traditional statics textbook, but the information is accessed through the proposed adaptive concept map visualization. The tool was then given to students in engineering statics class to be used as a supplemental textbook. The effects of the Adaptive Map tool were evaluated through a multiple case study approach that allowed researchers to understand how this tool fit into the larger learning context of a class. Results indicated that some students did integrate the Adaptive Map tool into the learning process, and furthermore that the tool did promote meaningful learning behaviors that lead to better conceptual understanding of the material. / Ph. D.

A elaboração de textos por alunos em resposta a questões dissertativas dentro do contexto escolar parece ser a forma mais convencional de se tentar avaliar o que os alunos sabem. O texto que um indivíduo produz procura refletir de forma aproximada a sua estrutura de conhecimentos sobre determinado tema. A leitura e análise de textos são tarefas que exigem um tempo considerável no dia-a-dia de um professor ou pesquisador e, quando há o interesse em conhecer as ideias mais relevantes sobre determinado tema para um grupo de alunos, a tarefa é ainda mais trabalhosa. O principal objetivo desta pesquisa consiste em desenvolver uma metodologia que utiliza ferramentas computacionais para transformar textos escritos por alunos em estruturas gráficas como mapas e redes de conceitos. A utilidade desta metodologia aparece tanto no contexto da pesquisa em ensino quanto na própria prática docente, já que o produto final de sua aplicação pode permitir estabelecer inferências quanto à estrutura de conhecimentos de um grupo de alunos. A investigação ocorreu a partir de dados coletados em duas disciplinas distintas de cursos de graduação do IQ-USP. Os dados coletados referem-se a produções textuais de 42 estudantes em resposta a uma questão que fornecia alguns conceitos pré-estabelecidos. A partir das respostas dos alunos foram realizados testes: (i) com dois softwares de análise de textos para a quantificação das relações entre conceitos; (ii) para verificar a influência na quantificação das relações entre conceitos partindo-se do texto como foi escrito e das proposições extraídas do mesmo e (iii) para a obtenção de diferentes tipos de estruturas gráficas. A partir dos testes realizados, foi possível concluir que o programa Hamlet® é mais eficiente e prático do que o programa ALA-Reader® para os objetivos da presente pesquisa. Além disso, a matriz gerada pelo Hamlet® para quantificar as relações entre conceitos depende essencialmente da estrutura do texto em questão - texto original ou texto modificado. Os três tipos de estruturas gráficas construídos apresentam diferentes focos, porém, podem ser considerados complementares. As redes V+P se mostraram interessantes para análises centradas nos conceitos pré-estabelecidos e fornecidos na questão que originou os textos dos alunos. As redes a partir de corte percentual apresentaram-se como representações bastante úteis para investigações interessadas em fazer um recorte ou destacar os aspectos considerados mais relevantes pelos alunos sobre determinado tema. Os mapas conceituais construídos neste trabalho mostraram-se como representações extremamente valiosas para conhecer a aproximada estrutura de conhecimentos dos grupos de alunos, uma vez que explicitam a natureza das relações proposicionais entre os conceitos. A construção de mapas conceituais partindo-se tanto dos textos originais quanto dos textos modificados permitiu concluir que as estruturas gráficas obtidas dos dois modos se aproximam bastante uma da outra, apresentando alta semelhança. Esta semelhança sugere que a utilização do programa Hamlet® para a obtenção de matrizes que quantificam relações entre conceitos presentes em um texto na forma como foi escrito é eficiente quando comparada ao processo manual e mais demorado de se extraírem proposições de um texto para obter uma matriz. / The drafting of essays by students in response to essay questions in the school context seems to be the most conventional way to assess the students` knowledge. The essay produced by a student seeks to approximately reflect his/her knowledge structure about a certain domain. The reading and the analysis of essays are tasks that require a considerable time in a teacher\'s or researcher\'s routine and, when the interest on knowing the most important ideas about a certain topic is verified in a group of students, the task is even harder. The main objective of this research is to develop a methodology that uses computational tools in order to transform written essays in graphic structures such as concept maps and networks graphs. This methodology could be useful not only for teaching research purposes but also for teaching practice, since the final product of its application may lead to inferences about the knowledge structure of a group of students. The investigation developed herein was based on data collected from two distinct matters of undergraduate IQ-USP. That data refer to written essays of 42 students in response to an essay question provided of some pre-established concepts. From the students\' responses, tests were performed: (i) with two softwares for text analysis with a view of quantifying the relationships between concepts, (ii) to investigate the influence on the quantification of relationships between concepts, from the original text and from the propositions extracted from this original text and (iii) to obtain different types of graphic structures. From the tests that were done, it was possible to conclude that Hamlet® consists in a more efficient and convenient program than ALA-Reader® to the objectives of this research. Furthermore, the array generated by Hamlet® program to quantify the relationships between concepts depends essentially on the structure of the essay - either the original text or the modified text. The three types of graphic structures that were built present different focuses, however, these graphic structures may be considered complementaries. The (V+P) network graphs can be thought as interesting representations that focuses on pre-established concepts that were provided on the essay questions. Network graphs from cutting percentage can be thought as representations that are more useful for investigations interested in making a cut or in highlighting the most relevant aspects of a subject by the students. The concept maps constructed in this paper can be thought as extremely valuable representations to know the approximate knowledge structure of the students groups, since they make clear the nature of the propositional relationships between concepts. The construction of concept maps starting from the original texts and also from the modified texts proved that both graphic structures obtained are very close to each other, being highly similar. This similarity suggests that the use of the Hamlet® program to obtain arrays that quantify relationship between concepts found in an original text is more effective in comparison to manual and time-consuming process of extracting propositions from the original text to obtain an array.

Neste trabalho utilizou-se o mapa conceitual como ferramenta para analisar a estrutura conceitual de alunos de um curso de graduação em química de uma universidade pública do estado de São Paulo, sobre o tema equilíbrio químico em dois momentos distintos, o primeiro ao ingressarem na universidade e o segundo após um ano de curso. Os mapas foram analisados utilizando uma adaptação do método de Análise Estrutural de Mapas Conceituais (AEMC adaptada). Teve-se como meta do trabalho: a) Avaliar, através de mapas conceituais elaborados pelos alunos a estrutura conceitual dos mesmos referente ao conceito equilíbrio químico; b) Descrever e analisar a evolução das relações entre os principais conceitos envolvidos no tema feitas por alunos no seu ingresso no curso de graduação e após um ano de curso, quando são submetidos pela primeira vez a um estudo sistemático do tema. Pretendeu-se assim, verificar quais foram às mudanças mais significativas ocorridas em decorrência das disciplinas ministradas neste período e c) Estabelecer uma relação entre estas mudança com o processo de ensino ao qual o aluno foi submetido, ou seja, como a estrutura conceitual dos estudantes a respeito do tema mudou em função do processo de aprendizado a que foram submetidos. Concluiu-se que a ferramenta desenvolvida durante a pesquisa foi muito útil na análise proposta; onde os alunos ingressantes definem o Equilíbrio químico como o estado onde as velocidades das reações se igualam sem relacionar a conceitos da termodinâmica, estes também, demonstraram muitos conhecimentos oriundos de como o conceito é apresentado no ensino médio. Verificou-se também que os estudantes ao serem submetidos à disciplinas relacionadas ao tópico equilíbrio químico modificaram seu entendimento sobre o tema. / In this work it was used the Concept Map as a tool for analyzing the conceptual structure of students in an undergraduate degree in Chemistry from a public university in the state of São Paulo, on the subject of Chemical Equilibrium at two different times, the first at the moment when entered in the university and second after a year. The maps were analyzed using an adaptation of the method of Structural Analysis of Concept Maps (AEMC adaptada). The objectives of work were: a) Explore and modify a method of analysis of Concept Maps in order to obtain a Concept Map that shows the main conceptual connection of a group of students in order to graphically represent their conceptual structure; b) Evaluate, through Concept Maps prepared by students their conceptual framework for the Chemical Equilibrium concept; c) Describe and analyze the evolution of relations between the main concepts involved on the subject made by students in their admission to the undergraduate course and after one year course, when they are first subjected to a systematic study of the subject. Thus the intention was to verify which were most significant changes occurred as a result of the matter taught in this period and d) Establish a connection between these changes with the teaching process to which the student was submitted, ie, how the conceptual framework of students regarding the subject changed as a function of the learning process they have submitted. It was concluded that the tool developed during the research was very useful in analyzing the proposal, which the new students define the Chemical Equilibrium as state where the velocities of reactions are equal without relating the concepts of thermodynamics, they also showed a lot of knowledge from as the concept is presented in high school. It was also found that students be subjected to the disciplines related to the topic Chemical Equilibrium changed their understanding of the subject, therefore, provide a greater number of connections between the different concepts involved on the matter.