Objective: The purpose of this study is to determine whether a novel, user-friendly rating system, visual sort and rate (VSR) provides comparable ratings to the currently used direct magnitude estimation (DME) rating system for rating perceptions of audible nasal emission (ANE) and hypernasality in cleft palate speakers. Methods: Twelve naïve listeners rated 152 speech samples of speakers with cleft palate across four conditions: rating hypernasality and ANE using either a VSR or DME rating scale. Raters were provided with a short training session, prior to rating each day. Inter- and intra-rater reliabilities, as well the line of best fit between scores using VSR and scores using DME was calculated to determine usability of VSR as a novel rating system. Results: Direct magnitude estimation resulted in the highest levels of inter-rater reliability, when rating hypernasality (DME r= .48; VSR r=.14), as well as ANE (DME r= .27; VSR r=.15). Most raters demonstrated high intra-rater reliabilities across conditions. A curvilinear line of best fit most accurately captured the relationship between DME and VSR scores when rating hypernasality (r=.64) and ANE (r=.66). Conclusions: A curvilinear relationship between ratings suggests that both variables are prothetic, and therefore, best captured using a DME rating scale (Eadie & Doyle, 2002). The use of DME is supported for continued use rating hypernasality, even amongst naïve listeners given a training session. Rating ANE was difficult, as ratings yielded low inter-rater reliabilities, regardless of the scale used. Further research regarding perceptions of audible nasal emission is warranted.