英語字彙的知識和能力是學好英語的關鍵因素之一，同時亦是影響閱讀流暢度的主要原因。但英語字彙教學常受限於時間及語言學習環境的限制，致使學習成效有限。隨著資訊科技的進步，許多研究著手於發展科技輔助英語字彙學習系統來輔助學習者進行英語字彙學習，希望能將英語字彙學習擴展到課程之外的時間。過去閱讀標註系統已被成功應用在提升學生的閱讀及字彙能力，然而相關的研究大都以使用文字、圖片和影片單一標註功能為主，鮮少研究探討多媒體閱讀標註對於字彙學習的研究。此外，閱讀標註部份也以教師的標註輔助學習為主，未有透過學生合作標註輔以英語字彙學習的研究。鑒於多媒體標註功能對教學的助益，本研究探討以具多媒體標註功能的數位閱讀標註系統來輔助英語字彙學習，不同多媒體標註組合對於英語字彙學習的影響差異。 據此，本研究探討在英文閱讀文本上讓國小學童採用文字及聲音(學生錄音)、圖片及聲音(學生錄音)與文字、圖片及聲音(學生錄音)等三種不同多媒體閱讀標註輔以英語字彙學習，對於兒童英語字彙學習成效、英語學習動機、學習滿意度以及記憶保留的影響差異。此外，也進一步探討採用上述三種不同多媒體閱讀標註輔以英語字彙學習，對於不同認知風格、不同英語能力及不同性別學童的英語字彙學習成效、英語學習動機、學習滿意度，以及記憶保留的影響差異，希望能發展出最有助於兒童基於多媒體閱讀標註輔以英語字彙學習的有效閱讀標註輔助學習模式。 研究結果發現：（1）採用語音標註搭配文字及圖片標註的學童在英語字彙學習的成效上，顯著優於採用語音標註搭配圖片標註的學童，而採用語音標註搭配圖片標註的學童之成效又顯著優於採用與語音標註搭配文字標註的學童；（2）採用語音標註搭配文字及圖片標註以及採用語音標註搭配圖片標註的學童在英語字彙記憶保留的成效上顯著優於採用語音標註搭配文字標註的學童；（3）採用三種不同多媒體搭配進行合作式閱讀標註的學童，其英語學習動機除了使用語音標註搭配文字標註的學童在「動機強度」向度的表現顯著優於使用語音標註搭配圖片標註的學童外，其餘向度未達顯著差異；而學習滿意度均無顯著差異；（4）採用語音標註搭配文字及圖片標註的圖像型學童，在英語字彙學習成效顯著高於採用語音標註搭配文字標註的圖像型學童；（5）採用語音標註搭配文字及圖片標註進行英語字彙學習的圖像型學童之英語字彙學習保留成效，顯著優於採用語音標註搭配文字標註的圖像型學童；（6）採用語音標註搭配文字及圖片標註的高分組學童之英語字彙學習成效，顯著高於採用語音標註搭配文字標註以及採用語音標註搭配圖片標註的高分組學童；（7）採用語音標註搭配文字及圖片標註的高分組學童之英語字彙學習保留成效，顯著優於採用語音標註搭配文字標註的高分組學童；（8）學童採用語音標註搭配文字、語音標註搭配圖片標註、語音標註搭配文字及圖片標註三種模式進行合作式閱讀標註，其英語字彙學習成效與英語字彙記憶保留程度，以及英語學習動機與學習滿意度之間具顯著相關性。最後，本研究亦提出應用多媒體標註於英語教學以及未來研究方向上的建議。 / English vocabulary knowledge and capability are the key factors in well learning English as well as the major factor in reading fluency. Nonetheless, English vocabulary teaching is often restricted to time and language learning environments to further restrict the learning achievement. With the advance of information technology, a lot of research focuses on developing technology-assisted English vocabulary learning systems to help learners’ English vocabulary learning, expecting to expand English vocabulary learning to the time beyond lessons. Past reading annotation systems were successfully applied to enhance students’ reading and vocabulary capabilities. However, most relevant research stressed on single annotation function of text, picture, and video, but seldom on multimedia reading annotation towards vocabulary learning. Moreover, reading annotation also concentrated on teacher annotation-assist learning, but not on student cooperative annotation assisted English vocabulary learning. In consideration of the assistance of multimedia annotation in teaching, a digital reading annotation system with multimedia annotation functions is utilized for English vocabulary learning to discuss the effects of different multimedia annotation combination on English vocabulary learning. Accordingly, English vocabulary learning assisted with different multimedia reading annotation of text & voice (student recording), picture & voice (student recording), and text, picture & voice (student recording) in English reading texts for elementary pupils is investigated the effects on children’s English vocabulary learning achievement, English learning motivation, learning satisfaction, and memory retention. Furthermore, the above multimedia reading annotation assisted English vocabulary learning are also discussed the effects on English vocabulary learning achievement, English learning motivation, learning satisfaction, and memory retention of pupils with different cognitive styles, English capabilities, and genders, expecting to develop the most effective reading annotation assisted learning model based on multimedia reading annotation assisted English vocabulary learning. The research findings are summarized as below. (1) Pupils applying voice annotation with text & picture annotation significantly outperform the ones with voice annotation and picture annotation on English vocabulary learning achievement, while the ones with voice annotation with picture annotation remarkably outperform those voice annotation and text annotation. (2) Pupils applying voice annotation with text & picture annotation and the ones applying voice annotation with picture annotation notably outperform the others with voice annotation and text annotation on the achievement of English vocabulary memory retention. (3) Among pupils applying different multimedia with cooperative reading annotation, the ones with voice annotation and text annotation present significantly better performance than those with voice annotation and picture annotation on Motive Strength in English learning motivation, but not on the rest dimensions. Besides, the learning satisfaction does not appear remarkable difference. (4) Pictorial pupils applying voice annotation with text & picture annotation notably present higher English vocabulary learning achievement than those pictorial pupils with text annotation. (5) Pictorial pupils applying voice annotation with text & picture annotation significantly outperform the other pictorial pupils with voice annotation and text annotation on the achievement of English vocabulary learning retention. (6) High-score pupils applying voice annotation with text & picture annotation remarkably outperform the other high-score pupils applying voice annotation with text annotation and voice annotation with picture annotation on English vocabulary learning achievement. (7) High-score pupils applying voice annotation with text & picture annotation notably outperform the other high-score pupils with voice annotation and text annotation on the achievement of English vocabulary learning retention. (8) Pupils applying different cooperative reading annotation of voice annotation with text annotation, voice annotation with picture annotation, and voice annotation with text & picture annotation show significant correlations between English vocabulary learning achievement and English vocabulary memory retention as well as English learning motivation and learning satisfaction. Finally, suggestions for applying multimedia annotation to English teaching and for future research directions are also proposed in this study.

多媒體影音教材已被廣泛使用於網路學習之教學場域，並且已發展出許多不同的多媒體影音教材型態，這些不同的多媒體影音教材型態，以及教材的組成元素如何影響數位學習成效值得深入探究。本研究使用可攜式眼動追蹤儀及穩定控制之實驗環境，隨機選取某公立高中高三普通班十六位學生進行眼動實驗，由受測學生依序觀看目前被廣泛採用的圖像影音動態教學式（picture in picture, PIP）及平面講述靜態教學式（voice over presentation, VOP）二種不同型態多媒體影音教材之興趣區（頭像、投影片、字幕、標題），探討包括三個眼動指標（凝視時間、凝視次數、回視次數）與學習成效之間的相關聯性。 結果顯示受測學習者利用圖像影音動態教學式（PIP）多媒體影音教材學習具有顯著的學習成效，而平面講述靜態教學式（VOP）則無顯著的學習成效。此外；圖像影音動態教學式（PIP）多媒體影音教材學習成效顯著優於平面講述靜態教學式（VOP）多媒體影音教材。再則經由皮爾森（Pearson）積差相關係數分析結果顯示，文字型認知風格學習者，使用平面講述靜態教學式（VOP）多媒體影音教材之投影片及標題興趣區之凝視時間、凝視次數及回視次數與學習者後測成績，呈顯著高度正相關，圖像型認知風格學習者則無。圖像影音動態教學式（PIP）多媒體影音教材之投影片、頭像、字幕及標題興趣區之凝視時間、凝視次數及回視次數與學習者後測成績，皆未呈現任何顯著的相關性。最後，圖像影音動態教學式（PIP）與平面講述靜態教學式（VOP）二種不同型態多媒體影音教材，皆以投影片區獲得最多的凝視時間、凝視次數與回視次數，其次為教師的頭像，最少的為標題區。 / Multimedia audio and video materials have been broadly applied to the teaching field of web-based learning, and various multimedia audio and video materials have been developed. It is worth studying the effect of such multimedia audio and video materials and the component elements on digital learning outcome. With a portable eye tracker and stably controlled experiment environment, 16 G12 students, in a public senior high school, are randomly selected for the eye movement experiment. The participants would sequentially view the area of interests (profiles, slides, subtitles, titles) of two types of multimedia audio and video materials, picture in picture (PIP) and voice over presentation (VOP), to discuss the correlations between three eye movement indices (fixation time, number of fixation times, number of regression times) and learning outcome. The result reveals that learners using picture in picture (PIP) present significant learning outcome, while ones with voice over presentation (VOP) do not appear remarkable learning outcome. Moreover, picture in picture (PIP) notably outperforms voice over presentation (VOP). The Pearson product-moment correlation coefficient analysis result shows that the learners with verbal-oriented cognitive style present notably positive correlations between the fixation time, number of fixation times, and number of regression times for slides and titles in voice over presentation (VOP) and the posttest results, while those with image-oriented cognitive style do not. The fixation time, number of fixation times, and number of regression times for slides, profile, subtitles, and titles in picture in picture (PIP) do not show significant correlations with learners’ posttest results. Finally, both picture in picture (PIP) and voice over presentation (VOP) appear the most fixation time, number of fixation times, and number of regression times for slides, followed by teachers’ profiles, and titles the least.

以往的音樂資訊檢索方法多使用歌詞、曲風、演奏的樂器或一段音頻訊號來當作查詢的媒介，然而，在某些情況下，使用者沒有辦法清楚描述他們想要尋找的歌曲，如：情境式的音樂檢索。本論文提出了一種基於圖像的情境式音樂資訊檢索方法，可以透過輸入圖片來找尋相應的音樂。此方法中我們使用了卷積神經網絡（Convolutional Neural Network）技術來處理圖片，將其轉為低維度的表示法。為了將異質性的多媒體訊息映射到同一個向量空間，資訊網路表示法學習（Network Embedding）技術也被使用，如此一來，可以使用距離計算找回和輸入圖片有關的多媒體訊息。我們相信這樣的方法可以改善異質性資訊間的隔閡（Heterogeneous Gap），也就是指不同種類的多媒體檔案之間無法互相轉換或詮釋。在實驗與評估方面，首先利用從歌詞與歌名得到的關鍵字來搜尋大量圖片當作訓練資料集，接著實作提出的檢索方法，並針對實驗結果做評估。除了對此方法的有效性做測試外，使用者的回饋也顯示此檢索方法和其他方法相比是有效的。同時我們也實作了一個網路原型，使用者可以上傳圖片並得到檢索後的歌曲，實際的使用案例也將在本論文中被展示與介紹。 / Listening to music is indispensable to everyone. Music information retrieval systems help users find their favorite music. A common scenario of music information retrieval systems is to search songs based on user's query. Most existing methods use descriptions (e.g., genre, instrument and lyric) or audio signal of music as the query; then the songs related to the query will be retrieved. The limitation of this scenario is that users might be difficult to describe what they really want to search for. In this paper, we propose a novel method, called ``image2song,'' which allows users to input an image to retrieve the related songs. The proposed method consists of three modules: convolutional neural network (CNN) module, network embedding module, and similarity calculation module. For the processing of the images, in our work the CNN is adopted to learn the representations for images. To map each entity (e.g., image, song, and keyword) into a same embedding space, the heterogeneous representation is learned by network embedding algorithm from the information graph. This method is flexible because it is easy to join other types of multimedia data into the information graph. In similarity calculation module, the Euclidean distance and cosine distance is used as our criterion to compare the similarity. Then we can retrieve the most relevant songs according to the similarity calculation. The experimental results show that the proposed method has a good performance. Furthermore, we also build an online image-based music information retrieval prototype system, which can showcase some examples of our experiments.

論文提要內容：由於電腦科技的進步，多媒體光碟課體已應用在外語的學習；這種全新的媒體會不會直接或間接影響閱讀策略之使用，及閱讀理解之能力？是本研究所關心並感興趣的主題。 這個研究主要是觀察六十位以英語為外語學習目標的學生，他們在閱讀電腦光碟及閱讀書面時，如何使用閱讀策略？研究的方法包括出聲思考及實驗比較兩種方式。結果發現，使用資源型(resourcing)閱讀策略比傳統閱讀策略(local-meaning-getting, global-meaning-getting, monitoring/ regulatory reading strategies)來得多；但書面閱讀的閱讀理解能力表現卻比多媒體閱讀來得好。 所以，語言學習者不應該太仰賴那些不須自己花太多功夫，隨手可得的線上資源型閱讀策略；然而多媒體閱讀課體仍可以提供讀者很多相關背景知識來輔助他們閱讀。因此閱讀策略的教學應該列入語言教師的教學計畫裏，傳統閱讀策略的使用應該在閱讀中被視為一個很重要而且根本的元素，資源型閱讀策略是無法取代它們的。 透過這個研究，我們希望提供一些實證資料及有用的建議給語言學習者，讓他們利用電腦科技來成功地學習語言。 / Due to the advancement of computer technology, multimedia CD-ROM courseware is applied to foreign language learning. Will this brand-new media directly bring about an impact on reading strategies and indirectly make a contribution to reading comprehension? This is the main concern and interest of the study. The study investigated how 60 EFL students used reading strategies in CD-ROM and in printed context. The research methods included think-aloud protocol and experimental comparison approach. It was found that resourcing strategies are used more than the traditional reading strategies, that is, local-meaning-getting, gtobal-meaning-getting, and monitoring/ regulatory strategies. However, the reading comprehension performance in paper reading is better than that in multimedia reading. Thus, language. learners should be taught not to refer to on-line resourcing strategies too much without exercising efforts themselves, but they can take much background knowledge from reading courseware to facilitate their reading. The instruction of reading strategies should be included in the teaching programs of language teachers. Traditional reading strategies ought to be regarded as a fundamental element in reading and can not be replaced by resourcing strategies. Through this study, it is hoped that learning language with computer technology can bring a success to language learners by providing empirical data and useful suggestions.