Back to search

University of British Columbia

Universal psychosocial screening for post-secondary students with HEARTSMAP-U : evidence for inter-rater reliability

Abstract

dc:description

Post-secondary students are typically assuming greater life responsibilities and independence, which can become emotionally overwhelming, resulting in a high prevalence of psychosocial issues. At a time when many young adults are moving away from their support networks and transitioning from pediatric to adult health care systems, there is a perceived lack of resources and numerous barriers to mental health treatment. A digital psychosocial self-assessment and guidance tool for post-secondary students (HEARTSMAP-U) was adapted to address this growing concern. HEARTSMAP-U has already undergone comprehensive adaptation and evaluation work among both a broad population of clinicians who support young adults in post-secondary education, as well as among post-secondary students themselves. Through a multiphasic study, this research aimed to 1) evaluate the inter-rater reliability of HEARTSMAP-U among young adults pursuing post-secondary education, as applied to a set of fictional cases, and 2) evaluate, on the same set of fictional cases, the scoring agreement between student and clinician assessors. In Phase 1 (n = 15), an iterative process was used to evaluate the fictional vignettes for comprehensiveness and clarity. Feedback received was reviewed and incorporated into the next version of the vignette and this process was conducted four times until all comments were positive and saturated. In Phase 2 (n = 34), HEARTSMAP-U’s inter-rater reliability was evaluated among post-secondary students. Students displayed substantial to near perfect inter-rater scoring agreement in applying HEARTSMAP-U to the finalized fictional clinical vignettes, with weighted kappas on tool domains ranging from 0.72 (Student Health; 95% CI: 0.71, 0.73) to 0.81 (Psychiatry; 95% CI: 0.80, 0.82). In Phase 3, a clinician applied HEARTSMAP-U to the same vignettes and a large proportion of scoring agreement was found between student and clinician responses (median range 0.82-0.85) on concern severity and on whether the individual described in the vignette had resources in place (97%). Together, these results indicate that HEARTSMAP-U can be consistently interpreted by young adults pursuing post-secondary education. These study results will add to HEARTSMAP-U’s ongoing evaluation in which HEARTSMAP-U’s predictive validity will be assessed.

Degree

thesis:*
Name thesis:degree_name
Master of Science - MSc
Level thesis:degree_level
master's
Discipline thesis:degree_discipline
Population and Public Health
Grantor dc:publisher
University of British Columbia
Year dc:date
2021

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Burt, Heather
Contributors dc:contributor
  • Doan, Quynh
  • Gadermann, Anne

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial-NoDerivatives 4.0 International
Language dc:language
eng

Identifiers

dc:identifier.*
Handle dc:identifier
http://hdl.handle.net/2429/79483

Chain of custody

source
Harvested from
University of British Columbia
Base URL
circle.library.ubc.ca/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Burt, Heather. Universal psychosocial screening for post-secondary students with HEARTSMAP-U : evidence for inter-rater reliability. master's thesis, University of British Columbia, 2021. http://hdl.handle.net/2429/79483