Reading comprehension assessments often vary from one measure to another related to the response format required, passage length and other variables. Yet, these measures purport to assess the same skill of reading comprehension, and they are often used interchangeably. Over the last decade some reading researchers have raised concerns that the variability in reading comprehension assessments may mean these measures are assessing different components of reading comprehension instead of consistently assessing one well-defined, complex skill. This study compared two commonly used reading comprehension assessments: the Wechsler Individual Achievement Test- Reading Comprehension (WIAT-RC) subtest, which requires examinees to read longer passages and answer open ended questions, and the Woodcock Johnson Passage Comprehension test (WJ-PC), which requires examinees to read very short passages and fill in the one word that is missing. This study was designed to test whether performance on these two measures is predicted by the same underlying language and executive functioning skills (a form of convergent validity) and whether these two measures are commensurate in predicting performance on a ‘real-world’ reading comprehension task (GRE verbal questions), with the latter being a test of ecological validity. Results suggested that the two measures varied in their ability to predict performance on the GRE verbal questions, where the WJ-PC significantly predicted performance on these questions, while the WIAT-RC did not. None of the executive functioning skills included in analyses significantly predicted performance on either the WJ-PC or the WIAT-RC, so no difference was found there; however, there were differences in how the language skills predicted performance on the two measures. Both vocabulary and word reading predicted significantly more variance in performance on the WJ-PC than the WIAT-RC. Overall, these results suggest there are important differences between these two reading comprehension measures, related to which underlying skills influence performance on each measure and the two tests’ ecological validity. These differences raise concerns about how well the reading comprehension construct has been defined and how consistently that complex skill is assessed across different measures. In addition, the differences between these two tests should be considered by clinicians who use and interpret scores on these measures in clinical settings.

The present study explored experimenters’ looking behavior in a gaze-following task as a function of infant temperament. Two experienced female experimenters ran 62 15-month-olds through a six-trial gaze-following procedure in which infants were not distracted on the first three trials, but were distracted on the latter three trials by an Elmo video playing in the background. Although experimenters were trained to look at target objects for eight seconds per trial and were blind to infant temperamental status, both experimenters looked significantly longer during the non-distracted trials when infants were rated by their caregivers as high on effortful control or surgency. These results suggest that even experienced experimenters are susceptible to infant-driven influences. More importantly, these results highlight the importance of conceptualizing lab-based infant research involving human experimenters as, in Bronfenbrenner's (1977) terms, representing the totality of a functional social system that does not exclude the experimenters.

Commonly employed executive function measures may be of limited use due to their modest ecological validity. A novel task was developed - the Multitasking in the City Test (MCT) - in an attempt to improve ecological validity. The MCT involves task demands that resemble the demands of everyday activities. In study one, healthy participants were recruited in order to explore ‘normal’ performance on the MCT and its relationship with other cognitive measures. The MCT showed poor associations with executive tests and significant correlations with non-executive tests. This suggested the MCT may evaluate executive functioning in a different way from other executive measures such that it does not simply measure component executive processes but the integration of these components into meaningful behaviour. Patients with stroke and traumatic brain injury were recruited for study two to further explore the ecological validity and MCT performance characteristics. Only the MCT and a semantic fluency task demonstrated good ecological validity via significant relationships with a behavioural rating scale. Patients and normals made qualitatively similar types of errors although patients made these errors more frequently. Patients demonstrated better planning ability but completed fewer tasks than normals on the MCT. This discrepancy was attributed to impaired initiation. In study three, the MCT and verbal fluency tasks were administered to brain-injured individuals both pre- and post-executive function rehabilitation to evaluate their utility as treatment outcome measures and to assess ecological validity via a different behavioural rating scale from the one used in study two. Strategies trained during treatment generalized to MCT but not verbal fluency performance. Both MCT and semantic fluency performance were found to have good ecological validity. Overall, the findings from this research project suggest the MCT and semantic fluency tasks have good ecological validity. They further suggest that several common executive function measures lack adequate ecological validity and may not be predictive of real world behaviour. Moreover, these results support the concept of an executive function ‘system’ that can be fractionated into a variety of executive processes and that impairments in one process (e.g., initiation) can exist alongside intact functioning in other processes (e.g., planning).

Commonly employed executive function measures may be of limited use due to their modest ecological validity. A novel task was developed - the Multitasking in the City Test (MCT) - in an attempt to improve ecological validity. The MCT involves task demands that resemble the demands of everyday activities. In study one, healthy participants were recruited in order to explore ‘normal’ performance on the MCT and its relationship with other cognitive measures. The MCT showed poor associations with executive tests and significant correlations with non-executive tests. This suggested the MCT may evaluate executive functioning in a different way from other executive measures such that it does not simply measure component executive processes but the integration of these components into meaningful behaviour. Patients with stroke and traumatic brain injury were recruited for study two to further explore the ecological validity and MCT performance characteristics. Only the MCT and a semantic fluency task demonstrated good ecological validity via significant relationships with a behavioural rating scale. Patients and normals made qualitatively similar types of errors although patients made these errors more frequently. Patients demonstrated better planning ability but completed fewer tasks than normals on the MCT. This discrepancy was attributed to impaired initiation. In study three, the MCT and verbal fluency tasks were administered to brain-injured individuals both pre- and post-executive function rehabilitation to evaluate their utility as treatment outcome measures and to assess ecological validity via a different behavioural rating scale from the one used in study two. Strategies trained during treatment generalized to MCT but not verbal fluency performance. Both MCT and semantic fluency performance were found to have good ecological validity. Overall, the findings from this research project suggest the MCT and semantic fluency tasks have good ecological validity. They further suggest that several common executive function measures lack adequate ecological validity and may not be predictive of real world behaviour. Moreover, these results support the concept of an executive function ‘system’ that can be fractionated into a variety of executive processes and that impairments in one process (e.g., initiation) can exist alongside intact functioning in other processes (e.g., planning).

Preference is commonly incorporated into educational interventions for individuals with developmental disabilities. Preference assessments have a solid research base indicating that they are more reliable tools for finding preference than the subjective opinions of parents and teachers. As evidence-based practices have been emphasized, the preference assessment has been a regular component of interventions and instructional programs for the population. Along with the utility, research regarding the assessment and relevant variables has also increased. However, many questions still exist and wait for more inquiry. One of the practical issues is the occurrence of challenging behaviors of individuals with disabilities during preference assessments. Highly occurring challenging behavior during an assessment may interrupt the procedure and lead to inaccurate results about the individual’s preference. That may ultimately affect the effectiveness of the intervention or instructional program. Using a procedure that does not evoke challenging behavior is necessary for accurate results as well as ethically responsible. Therefore this study examined the relation between functions of challenging behavior and three commonly used preference assessment procedures: Paired-Stimulus (PS), Multiple-Stimulus without Replacement (MSWO), and Free-Operant (FO). This study had two phases: Functional analyses and preference assessments. First, functional analyses were conducted to identify the function of challenging behaviors. The participants were nine children with developmental disabilities whose functional analysis results indicated their challenging behavior was maintained by access to tangible items (5), attention (2), and escape (2) reinforcers. After identifying the behaviors’ functions, preference assessments were implemented to compare the rates of the challenging behaviors. Each preference assessment format was conducted 5 times, in a random order for each participant. The results of the study demonstrate that the occurrence of challenging behavior with different functions was different depending on procedure formats. This suggests that there would be a relation between functions of challenging behavior and preference assessment formats. In other words, depending on the function of challenging behavior, the assessment procedure may act as a trigger evoking the challenging behavior. This study discussed practical guidance to prevent challenging behavior during preference assessments. / text

Virtual-reality neuropsychological tests offer a novel way to assess real-life executive functioning within the context of standardized test conditions whilst maximizing ecological validity. Given the limited empirical evidence base for many of these virtual-reality tests, the current study aimed to further examine the relative ecological validity, sensitivity to brain-injury and diagnostic accuracy of the virtual-reality based Jansari assessment of Executive Functions (JEF). To do this, the test was compared to seven standard tests of executive function, in a New Zealand sample of 27 brain-injured and 28 non brain-injured participants matched for age, gender and education. The JEF’s ecological validity was supported, with JEF total percent scores exhibiting large correlations with participants’ independently reported levels of everyday functioning, as measured by the Dysexecutive Questionnaire: Independent-Rater (DEX-IR) and Patient Competency Rating Scale-Relative (PCRS-R) (both rs = 0.606, p < 0.001). Compared to the standard executive function test variables included in the current study, the JEF’s associations with the everyday functioning measures were the largest, but only significantly larger than the respective correlations of some standard executive function test variables. These results indicated that the JEF is trending towards being relatively more ecologically valid than most of the standard executive function tests included in the study. The JEF also exhibited good sensitivity to brain-injury and diagnostic accuracy for discriminating brain-injured from non brain-injured participants, which was generally consistent with most of the standard tests, but only significantly better than some. These results provide empirical support for the JEF’s clinical utility, particularly in the assessment of brain-injured persons’ functional abilities. It also suggests that both virtual-reality and standard executive function tests have a place in the routine neuropsychological test batteries used to assess brain-injured persons.

Little is known about infants’ ability to deal with commonly encountered situations in which speech from one individual occurs simultaneously with that of others. Previous research has shown that while age of the infant and intensity of the background matter, so does the number of background speakers. The read-aloud multi-talker speech used in previous studies is perceptually different from conversational speech typically encountered by infants. To test generalizability, this study used a background of ecologically valid multi-talker speech. Using the head-turn preference procedure, infants were presented with passages of background noise with and without target infant-directed speech at a 10 dB SNR. Results show that while 9-month-olds prefer passages containing target speech with a white noise background, both 9- and 12-month-olds failed to show a preference with a multi-talker background. This inability to segregate speech streams under ecologically valid conditions demonstrates the adversity infants face to learn their community’s language.

Working memory (WM) has been closely linked to learning and achievement in children (Gathercole et al., 2004). The Forchelli Following Directions Task (FFDT) is a 15-item group-administered screener designed to assess working memory ability in school-aged children. The FFDT was developed to address the need for early identification of children with working memory difficulty. It specifically focuses on the need for easily administered and ecologically valid assessment. The FFDT was developed based on tasks cited in research to assess WM. The measure was developed across three iterations after receiving continual review from research experts in working memory and a group of three elementary school teachers. It also was piloted by three elementary school children to assess group-administration considerations. Participants in the validation study were 70 elementary school students 5 to 10 years of age spanning kindergarten to third grade were recruited from schools in the greater Philadelphia area. Participants were administered the group-administered working memory screener and completed individually administered measures of working memory, the WISC-IV Digit Span and Spatial Span, for comparison. Parents and teachers also completed behavior rating scales (i.e., BRIEF) measuring working memory. The FFDT demonstrated a sufficient Alpha's coefficient, indicating internal consistency. Significant Pearson correlations were found between existing measures of WM and the FFDT, indicating that the FFDT measures WM ability to a similar extent. The FFDT demonstrated good sensitivity to age and grade, as well. Further, the results of a ROC analysis comparing the identification of WM difficulty on the FFDT to existing measures of WM demonstrated a low to moderate effect. Overall, results indicate that the FFDT exhibited good reliability and validity. The anecdotal support of elementary school teachers and time efficiency of the task compared to existing WM measures also suggests good ecological validity. This study also demonstrated the utility of the FDDT in populations within a Response to Intervention (RtI) framework. Further research will be challenged to investigate the FFDT further scrutinize the construct validity and demonstrate significance in a larger, more representative sample of students. / School Psychology

Assessment of executive functioning in neurodevelopmental disorders (e.g., autism) is a crucial aspect of neuropsychological evaluations. The executive functions are accomplished by the supervisory attentional system (SAS) and include such processes as inhibition, switching, and planning. Autism spectrum disorder (ASD) tends to present similarly to other neurodevelopmental disorders (e.g., ADHD). For example, ASD and ADHD may share similar etiological underpinnings in the frontostriatal system of the frontal lobe. Research on executive functioning in ASD has been mixed, thus the precise nature of executive functioning deficits in ASD remains equivocal. In recent years, simulation technologies have emerged as an avenue to assess neurocognitive functioning in individuals with neurodevelopmental disorders impacting frontostriatal function. Simulation technology enables neuropsychologists to assess neurocognitive functioning within a testing environment that replicates environments in which the subject is likely to be in everyday life, as well as present controlled, real-world distractions, which may be better able to tap “hot” executive functions. A Virtual Classroom Continuous Performance Test (CPT) has been used successfully to assess attention in individuals with neurodevelopmental disorders impacting frontostriatal function. The current study aimed to investigate executive functioning in individuals with high functioning ASD using a new construct driven Stroop assessment embedded into the Virtual Classroom. Group differences were found in the Virtual Classroom with distractions condition, indicating individuals with ASD may be more vulnerable to external interference control than neurotypical individuals.

The present study aimed to examine how physical exertion influences cognitive performance.Orienteers (n = 23) competing at primarily a national to international level were recruited toparticipate. A randomised control trial using a cross-over research design required participantsto complete two trials measuring performance over a duration of 35 minutes. Specifically, inone trial participants undertook a cognitive testing protocol divided into five blocks comprisedof three separate cognitive tests that each lasted approximately one minute, between each testparticipants rested for one minute (i.e., one minute testing, followed by one minute resting fora duration of 35 minutes). The other trial required participants to complete the same cognitivetesting protocol whilst simultaneous completing a cycling time trial (i.e., aiming to cycle amaximum distance within the time of 35 minutes). Analyses revealed participants performedsignificantly worse on the cognitive tests whilst simultaneously completing the cycling timetrial. Upon closer scrutiny of the individual tests, designed to measure the cognitive functionsof decision-making, working memory, and updating, a similar trend in performance wasobserved although it was not found to be statistically significant. The findings of the presentstudy highlights implications for athletes, coaches, and sports psychologists in attempts tooptimise sport performance and minimize cognitive impairments during physical exertion. Thestudy supports the need for greater ecological validity in the investigation of cognitiveperformance in sport science research. The findings of the present study indicate that appliedsport psychology research may be enhanced by the use of research designs comprised ofcognitive tests that more closely replicate the cognitive demands of competition settings. / Denna studie syftade till att undersöka hur fysisk ansträngning påverkar kognitivprestation. Urvalet bestod av orienterare tävlandes på huvudsakligen nationell tillinternationell nivå (n = 23). Studien var en randomiserad korsstudie där deltagarna genomfördekognitiva tester vid två försökstillfällen på 35 minuter vardera. Under det ena försökstillfälletgenomförde deltagarna ett kognitivt testprotokoll, indelat i fem block innehållande tre testervardera, där varje test varade i ungefär en minut, följt av en minuts vila mellan testerna (dvs.en minuts testning, följt av en minuts vila med en varaktighet på 35 minuter). Det andraförsökstillfället bestod av att deltagarna genomförde samma kognitiva testprotokoll samtidigtsom de cyklade, med målet att komma så lågt som möjligt i distans under 35 minuter.Resultaten visade på signifikant lägre kognitiv prestation när deltagarna cyklade samtidigt.Genom närmare analys av varje enskilt test (som ämnade mäta de kognitiva funktionernabeslutsfattande, arbetsminne och uppdatering) hittades samma trend, men utan signifikans.Fynden kan vara av vikt för idrottare, tränare och idrottspsykologer i deras arbete för attoptimera prestation och minimera kognitiv försämring under fysisk ansträngning. De stödjeräven behovet av ekologisk validitet i studier ämnade att undersöka kognitiv prestation inomidrott. Vidare indikerar fynden att tillämpad idrottspsykologisk forskning kan förbättras genomatt använda en forskningsdesign innehållande kognitiva test som är mer jämförbara med dekognitiva utmaningar idrottare ställs inför under tävlingssammanhang.