Instant messaging dialogue is used for communication by hundreds of millions of people worldwide, but has received relatively little attention in computational linguistics. We describe methods aimed at providing a shallow interpretation of messages sent via instant messaging. This is done by assigning labels known as dialogue acts to utterances within messages. Since messages may contain more than one utterance, we explore automatic message segmentation using combinations of parse trees and various statistical models to achieve high accuracy for both classification and segmentation tasks. Finally, we gauge the immediate usefulness of dialogue acts in conversation management by presenting a dialogue simulation program that uses dialogue acts to predict utterances during a conversation. The predictions are evaluated via qualitative means where we obtain very encouraging results.

Dialogue act (DA) classification is an important step in the process of developing dialog systems. DA classification is a problem usually solved by supervised machine learning (ML) approaches that all require hand labeled data. Since hand labeling data is a resource-intensive task, many have proposed to focus on unsupervised or semi-supervised ML approaches to solve the problem of DA classification. This master’s thesis explores a novel method for semi-supervised approach to DA classification: K-Means+HMM. The method combines K- Means and Hidden Markov Model (HMM) modeling in addition to abstracting away the words in the utterances to their part-of-speech (POS) tags and the utterances to their cluster labels produced by K-Means prior to HMM training. The focus are the following hypotheses: H1) incorporating context of the utterances leads to better results (HMM is a method specifically used for sequential data and thus incorporates context, while K-Means does not); H2) increasing the number of clusters in K-Means+HMM leads to better results; H3) increasing the number of examples of cluster labels and hand labeled DAs pairs in K-Means+HMM leads to better results (the examples of pairs are used to create the emission probabilities used to define the HMM). One of the conclusions is that K-Means performs better than K-Means+HMM (the result for K-Means measured with one-to-one accuracy is 35.0%, while the result for K-Means+HMM is 31.6%) given 14 clusters and one example pair. However, when the number of examples is increased to 15 the result is 40.5% for K-Means+HMM; the biggest improvement is when the number of examples is increased to 20 resulting in 44% one-to-one accuracy. That is, K-Means+HMM outperforms K-Means provided that a certain number of examples is given. Another conclusion is that the number of examples has a much larger impact on the results - compared to the number of clusters - thus perhaps concluding that the statement “there is no data like labeled data” holds. / Klassificering av dialoghandlingar är ett viktigt steg i processen för utveckling av dialogsystem. Klassificering av dialoghandlingar är ett problem som vanligtvis löses med hjälp av övervakade maskininlärningsmetoder som alla behöver uppmärkt data. Eftersom uppmärkning av data är en resurskrävande uppgift har många föreslagit att fokusera på oövervakade eller delvis övervakade maskininlärningsmetoder för att lösa problemet av klassificering av dialoghandlingar. Denna masteruppsats utforskar en ny delvis övervakad maskininläningsmetod för klassificering av dialoghandlingar: K-Means+HMM. Föru- tom att metoden kombinerar K-Means och Hidden Markiv Model (HMM) modellering, abstraheras orden i yttranden till deras ordklasstaggar och yttranden till deras klusteretiketter som produceras av K-Means före HMM träningen. Projektets fokus är följande tre hypoteser: H1) en intergration av yttrandenas kontext leder till ett bättre resultat (HMM är en metod som används specifikt för sekventiell data och den integrerar således kontexten, medan K-Means gör inte det); H2) ökning av antalet kluster i K- Means+HMM leder till bättre resultat; H3) ökning av antalet exempel av par av klusteretiketter och dialoghandligar uppmärkta för hand i K- Means+HMM leder till bättre resultat (parexemplen används för att skapa emissionssannolikheter som definierar HMM). En av slutsatserna är att K-Means presterar bättre än K-Means+HMM (resultatet för K-means mätt med en-till-en noggrannhet är 35,0%, medan resultatet för K-Means+HMM är 31,6%) givet 14 kluster och ett exempelpar. Däremot, när antalet av exempelpar ökar till 15 ökar resultatet för K-Means+HMM till 40,5%. Den största ökningen är när antalet exempelpar är 20, vilket ger ett resulat på 44% en-till-en noggrannhet. Med andra ord, presterar K-Means+HMM bätre än K-Means då att ett visst antal exempelpar är tillgängligt. En annan slutsats är att antalet av exempelpar har en mycket större effekt på resultaten jämfört med antalet kluster, vilket då möjligtvis leder till slutsatsen att “det finns ingen bättre data än uppmärkt data”.

Relating statistical machine learning approaches to the automatic analysis of multiparty communicative events, such as meetings, is an ambitious research area. We have investigated automatic meeting segmentation both in terms of “Meeting Actions” and “Dialogue Acts”. Dialogue acts model the discourse structure at a fine grained level highlighting individual speaker intentions. Group meeting actions describe the same process at a coarse level, highlighting interactions between different meeting participants and showing overall group intentions. A framework based on probabilistic graphical models such as dynamic Bayesian networks (DBNs) has been investigated for both tasks. Our first set of experiments is concerned with the segmentation and structuring of meetings (recorded using multiple cameras and microphones) into sequences of group meeting actions such as monologue, discussion and presentation. We outline four families of multimodal features based on speaker turns, lexical transcription, prosody, and visual motion that are extracted from the raw audio and video recordings. We relate these lowlevel multimodal features to complex group behaviours proposing a multistreammodelling framework based on dynamic Bayesian networks. Later experiments are concerned with the automatic recognition of Dialogue Acts (DAs) in multiparty conversational speech. We present a joint generative approach based on a switching DBN for DA recognition in which segmentation and classification of DAs are carried out in parallel. This approach models a set of features, related to lexical content and prosody, and incorporates a weighted interpolated factored language model. In conjunction with this joint generative model, we have also investigated the use of a discriminative approach, based on conditional random fields, to perform a reclassification of the segmented DAs. The DBN based approach yielded significant improvements when applied both to the meeting action and the dialogue act recognition task. On both tasks, the DBN framework provided an effective factorisation of the state-space and a flexible infrastructure able to integrate a heterogeneous set of resources such as continuous and discrete multimodal features, and statistical language models. Although our experiments have been principally targeted on multiparty meetings; features, models, and methodologies developed in this thesis can be employed for a wide range of applications. Moreover both group meeting actions and DAs offer valuable insights about the current conversational context providing valuable cues and features for several related research areas such as speaker addressing and focus of attention modelling, automatic speech recognition and understanding, topic and decision detection.