No description available.

Co-training is a semi-supervised learning method that is designed to take advantage of the redundancy that is present when the object to be identified has multiple descriptions. Co-training is known to work well when the multiple descriptions are conditional independent given the class of the object. The presence of multiple descriptions of objects in the form of text, images, audio and video in multimedia applications appears to provide redundancy in the form that may be suitable for co-training. In this paper, we investigate the suitability of utilizing text and image data from the Web for co-training. We perform measurements to find indications of conditional independence in the texts and images obtained from the Web. Our measurements suggest that conditional independence is likely to be present in the data. Our experiments, within a relevance feedback framework to test whether a method that exploits the conditional independence outperforms methods that do not, also indicate that better performance can indeed be obtained by designing algorithms that exploit this form of the redundancy when it is present. / Singapore-MIT Alliance (SMA)

<p dir="ltr">In today's big data environment, development of robust knowledge discovery solutions depends on integration of data from various sources. For example, intelligence agencies fuse data from multiple sources to identify criminal activities; e-commerce platforms consolidate user activities on various platforms and devices to build better user profile; scientists connect data from various modality to develop new drugs, and treatments. In all such activities, entities from different data sources need to be aligned---first, to ensure accurate analysis and more importantly, to discover novel knowledge regarding these entities. If the data sources are networks, aligning entities from different sources leads to the task of network alignment, which is the focus of this thesis. The main objective of this task is to find an optimal one-to-one correspondence among nodes in two or more networks utilizing graph topology and nodes/edges attributes. </p><p dir="ltr">In existing works, diverse computational schemes have been adopted for solving the network alignment task; these schemes include finding eigen-decomposition of similarity matrices, solving quadratic assignment problems via sub-gradient optimization, and designing iterative greedy matching techniques. Contemporary works approach this problem using a deep learning framework by learning node representations to identify matches. Node matching's key challenges include computational complexity and scalability. However, privacy concerns or unavailability often prevent the utilization of node attributes in real-world scenarios. In light of this, we aim to solve this problem by relying solely on the graph structure, without the need for prior knowledge, external attributes, or guidance from landmark nodes. Clearly, topology-based matching emerges as a hard problem when compared to other network matching tasks.</p><p dir="ltr">In this thesis, I propose two original works to solve network topology-based alignment task. The first work, Graphlet-based Alignment (Graphlet-Align), employs a topological approach to network alignment. Graphlet-Align represents each node with a local graphlet count based signature and use that as feature for deriving node to node similarity across a pair of networks. By using these similarity values in a bipartite matching algorithm Graphlet-Align obtains a preliminary alignment. It then uses high-order information extending to k-hop neighborhood of a node to further refine the alignment, achieving better accuracy. We validated Graphlet-Align's efficacy by applying it to various large real-world networks, achieving accuracy improvements ranging from $20\%$ to $72\%$ over state-of-the-art methods on both duplicated and noisy graphs.</p><p dir="ltr">Expanding on this paradigm that focuses solely on topology for solving graph alignment, in my second work, I develop a self-supervised learning framework known as Self-Supervised Topological Alignment (SST-Align). SST-Align uses graphlet-based signature for creating self-supervised node alignment labels, and then use those labels to generate node embedding vectors of both the networks in a joint space from which node alignment task can be effectively and accurately solved. It starts with an optimization process that applies average pooling on top of the extracted graphlet signature to construct an initial node assignment. Next, a self-supervised Siamese network architecture utilizes both the initial node assignment and graph convolutional networks to generate node embeddings through a contrastive loss. By applying kd-tree similarity to the two networks' embeddings, we achieve the final node mapping. Extensive testing on real-world graph alignment datasets shows that our developed methodology has competitive results compared to seven existing competing models in terms of node mapping accuracy. Additionally, we establish the Ablation Study to evaluate the two-stage accuracy, excluding the learning representation part and comparing the mapping accuracy accordingly.</p><p dir="ltr">This thesis enhances the theoretical understanding of topological features in the analysis of graph data for network alignment task, hence facilitating future advancements toward the field.</p>

Multimedia' nature of an application refers to the presence of several media streams in parallel. Whether it is receiving real-time data or retrieving stored data, there exists an end-to-end delay in data transfer from source to destination over the network. This delay experienced can be split into a fixed part and a variable part. Data processing time like coding and decoding at the source and destination are the fixed delays experienced. The variable delay occurs mainly due to queuing at the intermediate nodes during its flow through the network. The variable or unequal delays introduce gaps or discontinuities within a stream. In multi-stream applications where each stream may flow on different routes based on the bandwidth availability experiencing different delays, mismatch between them can also occur. These discontinuities and skews result in poor quality of playout. Clock drift and variations in drift rates between the source/s and destination/s, clock also lead to poor quality of play out. To eliminate these skews and discontinuities, there must be mechanisms, viz., and synchronisation services to convey, reintroduce and maintain the temporal relationship between the media streams for presentation throughout the playout, at the destination. The reintroduction of this lost temporal relationship within a stream and between various media streams for presentation at the destination is the object of multimedia synchronisation and is the subject matter of this thesis. In the presence of synchronised clocks, the main cause of asynchronies between media streams is the difference in delays experienced and the jitter. In this work, to convey the temporal relationship between streams of an application to the playout site, each stream is assigned a priority л, based on its importance to the user. The media streams are then divided into synchronisation units called 'Groups' based on that stream's characteristics which has the highest priority л. A group may therefore consist of one video frame and other data which were generated in that interval. Or may consist of silence and talk-spurt periods of the voice stream with data units of other streams generated in the same interval. Since the quality of playout of temporally related delay-sensitive streams depends upon the delay-experienced, the concept of QoS can be extended to describe the presentation requirements of uch data. Depending on the user perception and the delay experienced, an application can have a range of playout times, giving the best performance. The presentation of many real-time applications can be considered satisfactory even when the delay bound is exceeded by a small amount for a short period of time under varying network conditions. This property can be exploited by defining two sets of QoS parameters, namely QoS optimum and QoSlimit for each real-time application. As the delay and its variations increase, the optimum playout time range decreases. QoS optimum specifies the performance parameters required to perceive 'realtime'. Multimedia data can be played out at its QoSlimit with a deterioration in quality under poor network conditions still maintaining the synchronisation between streams. To control the playout at two levels of QoS, and maintain intra-media and inter-media synchronisation, stream controllers and super stream controllers have been used. The dynamic delay compensation algorithm and synchronisation services were simulated using network delay models and performances studied. It is shown that the proposed algorithm not only synchronised media streams and smoothened jitter but also optimised buffer space and buffer occupancy time while meeting the desired quality of service requirements

Ο ταχύτατα εξελισσόμενος τομέας των δικτύων κινητών επικοινωνιών έχει επιφέρει μία ιδιαίτερα αυξανόμενη απαίτηση για ασύρματη, πολυμεσική επικοινωνία. Στη ραγδαία εξέλιξη του τομέα αυτού συμβάλουν τα μέγιστα και οι απαιτήσεις της σύγχρονης αγοράς για ένα ενοποιημένο και λειτουργικό σύστημα κινητής τηλεφωνίας παρέχοντας παράλληλα πληθώρα ευρυζωνικών υπηρεσιών ψηφιακού περιεχομένου στους πελάτες – χρήστες του. Είναι γεγονός ότι, τα τελευταία χρόνια τα δίκτυα επικοινωνιών τρίτης γενιάς (3G) – UMTS γνωρίζουν μεγάλη άνθηση και η χρήση τους έχει επεκταθεί στις περισσότερες ευρωπαϊκές χώρες, όπως και στην Ελλάδα. Τα νέα αυτά κινητά δίκτυα αντικαθιστούν τα υπάρχοντα κινητά δίκτυα δεύτερης γενιάς και επιπλέον προσφέρουν προηγμένες υπηρεσίες στους κινητούς χρήστες. Ωστόσο, η αδήριτη ανάγκη για μεγαλύτερες (ευρυζωνικές) ταχύτητες πρόσβασης οδήγησε στην περαιτέρω ανάπτυξη των 3G δικτύων και στην υιοθέτηση νέων τεχνολογιών, με κυριότερο εκπρόσωπο τους την τεχνολογία HSPA. Η τεχνολογία HSPA αποτελεί τη φυσιολογική μετεξέλιξη του UMTS, η οποία πολλές φορές συναντάται και ως 3.5G ή 3G+, προκειμένου να δηλώσει την αναβάθμιση του 3G (UMTS) προτύπου. Ωστόσο, παρά το γεγονός ότι η τεχνολογία HSPA αναμένεται να προσφέρει τη δυνατότητα παροχής πληθώρας ευρυζωνικών υπηρεσιών, το 3GPP ήδη μελετά και επεξεργάζεται νέες τεχνολογίες που θα επικρατήσουν την αμέσως επόμενη δεκαετία στην αγορά των κινητών επικοινωνιών. Το νέο αυτό project αποκαλείται Long Term Evolution (LTE) και στοχεύει στην επίτευξη ακόμη υψηλότερων ρυθμών μετάδοσης σε συνδυασμό με την αξιοποίηση μεγαλύτερου εύρος ζώνης. Κύρια προοπτική του LTE αποτελεί η διασφάλιση της ανταγωνιστικότητας και η επικράτηση του προτύπου στο χρονικό ορίζοντα της επόμενης δεκαετίας. Κατά συνέπεια, η αγορά κινητών επικοινωνιών σταδιακά μεταλλάσσεται προς τη δημιουργία δικτύων κινητών επικοινωνιών επόμενης γενιάς, με απώτερο σκοπό την επίτευξη του αποκαλούμενου «Mobile Broadband». Ταυτόχρονα με την εκτεταμένη εξάπλωση των δικτύων κινητών επικοινωνιών επόμενης γενιάς καθώς και τις αυξημένες δυνατότητες των κινητών συσκευών, οι πάροχοι πολυμεσικού περιεχομένου και υπηρεσιών ενδιαφέρονται όλο και περισσότερο για την υποστήριξη της πολυεκπομπής δεδομένων (multicasting) στα δίκτυα αυτά με σκοπό την αποτελεσματική διαχείριση και επαναχρησιμοποίηση των διαθέσιμων πόρων του δικτύου. Επιπρόσθετα, οι χρήστες των κινητών δικτύων έχουν πλέον την απαίτηση να προσπελαύνουν εφαρμογές και υπηρεσίες οι οποίες μέχρι σήμερα μπορούσαν να διατεθούν αποκλειστικά από τα συμβατικά ενσύρματα δίκτυα. Έτσι λοιπόν στις μέρες μας γίνεται λόγος για υπηρεσίες πραγματικού χρόνου όπως mobile TV, mobile gaming, mobile streaming κ.α. Ένα από τα σημαντικότερα βήματα των δικτύων κινητών επικοινωνιών προς την κατεύθυνση της παροχής νέων, προηγμένων πολυμεσικών υπηρεσιών είναι η εισαγωγή της υπηρεσίας Multimedia Broadcast / Multicast Service (MBMS). Η MBMS υπηρεσία έχει σαν κύριο σκοπό την υποστήριξη IP εφαρμογών πανεκπομπής (broadcact) και πολυεκπομπής (multicast) επιτρέποντας με αυτό τον τρόπο την παροχή υπηρεσιών υψηλού ρυθμού μετάδοσης σε πολλαπλούς χρήστες με οικονομικό τρόπο. Η multicast μετάδοση δεδομένων σε κινητά δίκτυα επικοινωνιών είναι μια νέα λειτουργικότητα η οποία βρίσκεται ακόμη στο στάδιο των δοκιμών και της προτυποποίησης της. Ένας multicast μηχανισμός μεταδίδει τα δεδομένα μόνο μία φορά πάνω από κάθε ασύρματο σύνδεσμο που αποτελεί τμήμα των μονοπατιών προς τους προορισμούς-κινητούς χρήστες. Το κρισιμότερο σημείο που εντοπίζεται κατά τη multicast μετάδοση δεδομένων στα κινητά δίκτυα επικοινωνιών είναι ο αποτελεσματικός έλεγχος ισχύος. Οι σταθμοί βάσης των κυψελωτών αυτών δικτύων διαθέτουν περιορισμένους πόρους ισχύος (άρα και περιορισμένη χωρητικότητα κυψέλης), γεγονός που επιβάλλει τη χρήση μίας βέλτιστης στρατηγικής για την όσο το δυνατόν καλύτερη αξιοποίηση των διαθέσιμων πόρων ισχύος. Ο έλεγχος ισχύος στοχεύει στη μείωση της εκπεμπόμενης ισχύος, στην ελαχιστοποίηση του θορύβου στο κυψελωτό δίκτυο και κατά συνέπεια στη διασφάλιση μεγαλύτερης χωρητικότητας επιπλέον χρηστών. Ένα από τα βασικότερα στοιχεία του ελέγχου ισχύος στα δίκτυα κινητών επικοινωνιών επόμενης γενιάς κατά τη multicast μετάδοση πολυμεσικών δεδομένων αποτελεί η επιλογή του κατάλληλου καναλιού μεταφοράς για τη μετάδοση των δεδομένων από τον κόμβο RNC στον κινητό χρήστη. Συγκεκριμένα, πρόκειται για ένα κρίσιμο ζήτημα το οποίο είναι ακόμα υπό εξέταση στο 3GPP. Προς την κατεύθυνση αυτή, στο MBMS στάνταρ έχει αναπτυχθεί ένας μηχανισμός που αποκαλείται Counting Mechanism. Ο στόχος του μηχανισμού αυτού είναι η βελτιστοποίηση της ροής δεδομένων για την υπηρεσία MBMS, όταν αυτά διέρχονται από τις διεπαφές του UTRAN (διεπαφές Iub και Uu). Ωστόσο, η υπάρχουσα μορφή του μηχανισμού αυτού διακρίνεται από πολλές αδυναμίες που δεν επιτρέπουν την αποτελεσματική και μαζική μετάδοση πολυμεσικών δεδομένων. Τα σημαντικότερα προβλήματα του υπάρχοντος Counting Mechanism είναι η απουσία ευρυζωνικών χαρακτηριστικών καθώς και η σπατάλη σημαντικού τμήματος των (ούτως ή άλλως περιορισμένων) πόρων ισχύος. Εν γένει, η επιλογή του κατάλληλου καναλιού μεταφοράς των πολυμεσικών δεδομένων στο ασύρματο μέσο είναι μια δύσκολη διαδικασία καθώς μια λανθασμένη επιλογή καναλιού μπορεί να οδηγήσει στην αστοχία ενός ολόκληρου κελιού. Γίνεται σαφές, λοιπόν, ότι απαιτείται μία βελτιωμένη έκδοση του υπάρχοντος Counting Mechanism για την αποτελεσματικότερη και οικονομικότερη μετάδοση πολυμεσικού περιεχομένου σε μεγάλο πλήθος χρηστών. Στόχος της παρούσας μεταπτυχιακής εργασίας είναι η μελέτη του ελέγχου ισχύος στα δίκτυα κινητών επικοινωνιών επόμενης γενιάς καθώς και η ανάπτυξη νέων μεθόδων για τη βελτιστοποίηση του Counting Mechanism. Ιδιαίτερο χαρακτηριστικό της μεταπτυχιακής αυτής εργασίας είναι η ενσωμάτωση και η «εκμετάλλευση» όλων των ιδιαίτερων χαρακτηριστικών της HSDPA τεχνολογίας στην MBMS υπηρεσία. Η MBMS υπηρεσία μέχρι τώρα βασίζεται στη λειτουργικότητα των υπαρχόντων UMTS δικτύων. Ο συνδυασμός των δύο αυτών προτύπων, δηλαδή του MBMS και του HSDPA, υπόσχεται τόσο την παροχή ευρυζωνικών πολυμεσικών δεδομένων σε μεγάλο πλήθος κινητών χρηστών όσο και τη βέλτιστη επίτευξη ελέγχου ισχύος. Προς αυτή την κατεύθυνση, θα πραγματοποιηθεί ανάλυση όλων των υπαρχόντων καναλιών μεταφοράς του UMTS καθώς και της τεχνολογίας HSDPA και τα οποία μπορούν να χρησιμοποιηθούν για τη multicast μετάδοση πολυμεσικών δεδομένων. Πιο συγκεκριμένα, τα κανάλια τα οποία αξιολογούνται είναι τα: Forward Access Channel, High Speed–Downlink Shared Channel και Dedicated Channel. Τα παραπάνω κανάλια μεταφοράς αξιολογούνται με βάση την απαιτούμενη ισχύ που πρέπει να ανατεθεί από το σταθμό βάσης για καθένα από αυτά, και κατά συνέπεια με βάση το ρυθμό μετάδοσης τους, τον αριθμό των χρηστών που μπορούν να εξυπηρετήσουν, την ποιότητα υπηρεσιών για κάθε χρήστη, τη μέγιστη δυνατή κάλυψη της κυψέλης κ.α. Επίσης, αντικείμενο της παρούσας μεταπτυχιακής εργασίας είναι η εύρεση ενός κατάλληλου σημείου εναλλαγής μεταξύ των διάφορων τύπων καναλιών κατά τη μετάδοση πολυμεσικών δεδομένων. Θα διερευνηθούν τεχνικές μείωσης της εκπεμπόμενης ισχύος με απώτερο σκοπό την αποδοτικότερη χρήση των πόρων του συστήματος και θα προταθούν νέες παραλλαγές του Counting Mechanism με ανώτερα χαρακτηριστικά διαχείρισης και κατανομής πόρων ισχύος. Οι νέοι αυτοί μηχανισμοί υπόσχονται βελτιωμένη απόδοση, μείωση της καταναλισκόμενης ισχύος και κατά συνέπεια αύξηση της χωρητικότητας των κινητών δικτύων επόμενης γενιάς. Το γεγονός αυτό μπορεί να επιτρέψει τη μαζική μετάδοση πολυμεσικών δεδομένων σε πληθώρα κινητών χρηστών. Τέλος, θα διερευνηθούν και νέες, πιο αποδοτικές τεχνικές για τη μετάδοση πολυμεσικών δεδομένων στα μελλοντικά δίκτυα LTE. Στην περίπτωση αυτή λαμβάνονται υπόψιν όλες οι βασικές τεχνικές μετάδοσης δεδομένων όπως τα MIMO κεραιοσυστήματα. / Due to rapid growth of mobile communications technology, the demand for wireless multimedia communications thrives in today’s consumer and corporate market. The need to evolve multimedia applications and services is at a critical point given the proliferation and integration of wireless systems. Consequently, there is a great interest in using the IP-based networks to provide multimedia services. One of the most important areas in which the issues are being debated, is the development of standards for the Universal Mobile Telecommunications System (UMTS). UMTS constitutes the third generation (3G) of cellular wireless networks which aims to provide highspeed data access along with real time voice calls. Wireless data is one of the major boosters of wireless communications and one of the main motivations of the next generation standards. Through the 3G mobile networks, the mobile users have the opportunity to run applications and realize services that offered until today only by wired networks. Such broadband services are mobile Internet, mobile TV, mobile gaming, mobile streaming, video calls etc. High Speed Packet Access (HSPA) constitutes a significant step towards the so-called Mobile Broadband. HSPA supports both downlink and uplink communication through the HSDPA and HSUPA channels, respectively. HSPA promises the provision of enhanced end-users’ experience with a wide range of novel, interactive applications, faster performance and reduced delays. Furthermore, from the operators’ prism, HSPA ensures improved network performance, increased capacity and higher coverage. Multimedia Broadcast Multicast Service (MBMS) is a novel framework, extending the existing UMTS infrastructure that constitutes a significant step towards the so-called Mobile Broadband. MBMS is intended to efficiently use network and radio resources, both in the core network and, most importantly, in the air interface of UMTS Terrestrial Radio Access Network (UTRAN), where the bottleneck is placed to a large group of users. Actually, MBMS is a point-to-multipoint service in which data is transmitted from a single source entity to multiple destinations, allowing the networks resources to be shared. MBMS is an efficient way to support the plethora of the emerging wireless multimedia and application services such as IP Video Conferencing, Streaming Video by supporting both broadcast and multicast transmission modes. Long Term Evolution (LTE) will stretch the performance of 3G systems with improved coverage and system capacity, as well as increased data rates and reduced latency. LTE also provides a tight integration between unicast and multicast/broadcast MBMS transport bearers. Moreover, it also takes 3G-MBMS one step further to provide highly efficient multi-cell broadcast. By transmitting not only identical signals from multiple cell sites (with identical coding and modulation), but also synchronize the transmission timing between cells, the signal at the mobile terminal will appear exactly as a signal transmitted from a single cell site and subject to multi-path propagation. There is a growing demand for wireless data applications, which although face low penetration today, are expected to gain high interest in future mobile networks. These applications actually reflect a modern, future way of communication among mobile users. For instance, mobile TV is expected to be a ‘killer’ application for 3G’s. Such mobile TV services include streaming live TV (news, weather forecasts etc.) and streaming video (such as video clips). All the above constitute a series of some indicative emerging applications that necessitate advanced transmission techniques. However, increased improvements have to be made both in the uplink and downlink transmission and in better radio resource management, in order to meet future demands and provide rich multimedia services to large users’ population. In addition, several obstacles, mainly regarding the interoperability and ubiquitous access between different access technologies and services, have to be overcome (thus leading to 4G). The main target of this dissertation is the study of power control issues, the development and the performance evaluation of an efficient power scheme for the provision of broadband, multicast services and applications to mobile users. This will be effectively implemented through the efficient use of MBMS and HSPA technologies in both 3G and its evolution LTE. An important aspect of this work is the investigation of the selection of the most efficient radio bearer for the transmission of MBMS multicast data. MBMS services can be provided in each cell by either multiple Point to Point (PTP) channels or by a single Point to Multipoint (PTM) channel. PTM transmission uses a single channel reaching down to the cell edge, which conveys identical traffic. On the other hand, PTP transmission uses dedicated channel allocated to each user, which conveys identical content. Obviously, a decision has to be made on the threshold between these two approaches. Therefore, improvements of the currently existing Counting Mechanism in MBMS will be studied. Although relative research work in this field considers the need for a power-based Counting Mechanism and not a UE-based Counting Mechanism, the case of HSDPA usage in such a power mechanism could be further investigated, taking also into account the availability of multi-mode cells. This could lead to an optimal scheme for the MBMS Counting Mechanism. The fundamental selection criterion of channel type is the amount of base station power required to transmit to a group of users. To this direction, the role of power control in the MBMS multicast transmission in UMTS is studied and analysed. A power control scheme for the efficient radio bearer selection in MBMS is then proposed. The choice of the most efficient transport channel in terms of power consumption is a key point for the MBMS since a wrong transport channel selection for the transmission of the MBMS data could result to a significant decrease in the total capacity of the system. Various UMTS transport channels are examined for the transmission of the multicast data and a new algorithm is proposed for the more efficient usage of power resources in the base station.

The rapid advances in technologies make the explosive growth of multimedia data possible and available to the public. Multimedia data can be defined as data collection, which is composed of various data types and different representations. Due to the fact that multimedia data carries knowledgeable information, it has been widely adopted to different genera, like surveillance event detection, medical abnormality detection, and many others. To fulfil various requirements for different applications, it is important to effectively classify multimedia data into semantic concepts across multiple domains. In this dissertation, a correlation-based multimedia semantic concept detection framework is seamlessly integrated with the deep learning technique. The framework aims to explore implicit and explicit correlations among features and concepts while adopting different Convolutional Neural Network (CNN) architectures accordingly. First, the Feature Correlation Maximum Spanning Tree (FC-MST) is proposed to remove the redundant and irrelevant features based on the correlations between the features and positive concepts. FC-MST identifies the effective features and decides the initial layer's dimension in CNNs. Second, the Negative-based Sampling method is proposed to alleviate the data imbalance issue by keeping only the representative negative instances in the training process. To adjust dierent sizes of training data, the number of iterations for the CNN is determined adaptively and automatically. Finally, an Indirect Association Rule Mining (IARM) approach and a correlation-based re-ranking method are proposed to reveal the implicit relationships from the correlations among concepts, which are further utilized together with the classification scores to enhance the re-ranking process. The framework is evaluated using two benchmark multimedia data sets, TRECVID and NUS-WIDE, which contain large amounts of multimedia data and various semantic concepts.

No description available.

In this work, we focus on automatically classifying music by mood. For this purpose, we propose computational models using information extracted from the audio signal. The foundations of such algorithms are based on techniques from signal processing, machine learning and information retrieval. First, by studying the tagging behavior of a music social network, we find a model to represent mood. Then, we propose a method for automatic music mood classification. We analyze the contributions of audio descriptors and how their values are related to the observed mood. We also propose a multimodal version using lyrics, contributing to the field of text retrieval. Moreover, after showing the relation between mood and genre, we present a new approach using automatic music genre classification. We demonstrate that genre-based mood classifiers give higher accuracies than standard audio models. Finally, we propose a rule extraction technique to explicit our models. / En esta tesis, nos centramos en la clasificación automática de música a partir de la detección de la emoción que comunica. Primero, estudiamos cómo los miembros de una red social utilizan etiquetas y palabras clave para describir la música y las emociones que evoca, y encontramos un modelo para representar los estados de ánimo. Luego, proponemos un método de clasificación automática de emociones. Analizamos las contribuciones de descriptores de audio y cómo sus valores están relacionados con los estados de ánimo. Proponemos también una versión multimodal de nuestro algoritmo, usando las letras de canciones. Finalmente, después de estudiar la relación entre el estado de ánimo y el género musical, presentamos un método usando la clasificación automática por género. A modo de recapitulación conceptual y algorítmica, proponemos una técnica de extracción de reglas para entender como los algoritmos de aprendizaje automático predicen la emoción evocada por la música

Thanks to the advanced technologies and social networks that allow the data to be widely shared among the Internet, there is an explosion of pervasive multimedia data, generating high demands of multimedia services and applications in various areas for people to easily access and manage multimedia data. Towards such demands, multimedia big data analysis has become an emerging hot topic in both industry and academia, which ranges from basic infrastructure, management, search, and mining to security, privacy, and applications. Within the scope of this dissertation, a multimedia big data analysis framework is proposed for semantic information management and retrieval with a focus on rare event detection in videos. The proposed framework is able to explore hidden semantic feature groups in multimedia data and incorporate temporal semantics, especially for video event detection. First, a hierarchical semantic data representation is presented to alleviate the semantic gap issue, and the Hidden Coherent Feature Group (HCFG) analysis method is proposed to capture the correlation between features and separate the original feature set into semantic groups, seamlessly integrating multimedia data in multiple modalities. Next, an Importance Factor based Temporal Multiple Correspondence Analysis (i.e., IF-TMCA) approach is presented for effective event detection. Specifically, the HCFG algorithm is integrated with the Hierarchical Information Gain Analysis (HIGA) method to generate the Importance Factor (IF) for producing the initial detection results. Then, the TMCA algorithm is proposed to efficiently incorporate temporal semantics for re-ranking and improving the final performance. At last, a sampling-based ensemble learning mechanism is applied to further accommodate the imbalanced datasets. In addition to the multimedia semantic representation and class imbalance problems, lack of organization is another critical issue for multimedia big data analysis. In this framework, an affinity propagation-based summarization method is also proposed to transform the unorganized data into a better structure with clean and well-organized information. The whole framework has been thoroughly evaluated across multiple domains, such as soccer goal event detection and disaster information management.

<p>Graph diffusion is the process of spreading information from one or few nodes to the rest of the graph through edges. The resulting distribution of the information often implies latent structure of the graph where nodes more densely connected can receive more signal. This makes graph diffusions a powerful tool for local clustering, which is the problem of finding a cluster or community of nodes around a given set of seeds. Most existing literatures on using graph diffusions for local graph clustering are linear diffusions as their dynamics can be fully interpreted through linear systems. They are also referred as eigenvector, spectral, or random walk based methods. While efficient, they often have difficulty capturing the correct boundary of a target label or target cluster. On the contrast, maxflow-mincut based methods that can be thought as 1-norm nonlinear variants of the linear diffusions seek to "improve'' or "refine'' a given cluster and can often capture the boundary correctly. However, there is a lack of literature to adopt them for problems such as community detection, local graph clustering, semi-supervised learning, etc. due to the complexity of their formulation. We addressed these issues by performing extensive numerical experiments to demonstrate the performance of flow-based methods in graphs from various sources. We also developed an efficient LocalGraphClustering Python Package that allows others to easily use these methods in their own problems. While studying these flow-based methods, we find that they cannot grow from small seed set. Although there are hybrid procedures that incorporate ideas from both linear diffusions and flow-based methods, they have many hard to set parameters. To tackle these issues, we propose a simple generalization of the objective function behind linear diffusion and flow-based methods which we call generalized local graph min-cut problem. We further show that by involving p-norm in this cut problem, we can develop a nonlinear diffusion procedure that can find local clusters from small seed set and capture the correct boundary simultaneously. Our method can be thought as a nonlinear generalization of the Anderson-Chung-Lang push procedure to approximate a personalized PageRank vector efficiently and is a strongly local algorithm-one whose runtime depends on the size of the output rather than the size of the graph. We also show that the p-norm cut functions improve on the standard Cheeger inequalities for linear diffusion methods. We further extend our generalized local graph min-cut problem and the corresponding diffusion solver to hypergraph-based machine learning problems. Although many methods for local graph clustering exist, there are relatively few for localized clustering in hypergraphs. Moreover, those that exist often lack flexibility to model a general class of hypergraph cut functions or cannot scale to large problems. Our new hypergraph diffusion method on the other hand enables us to compute with a wide variety of cardinality-based hypergraph cut functions and still maintains the strongly local property. We also show that the clusters found by solving the new objective function satisfy a Cheeger-like quality guarantee.</p> <p>Besides clustering, recent work on graph-based learning often focuses on node embeddings and graph neural networks. Although these GNN based methods can beat traditional ones especially when node attributes data is available, it is challenging to understand them because they are highly over-parameterized. To solve this issue, we propose a novel framework that combines topological data analysis and diffusion to transform the complex prediction space into human understandable pictures. The method can be applied to other datasets not in graph formats and scales up to large datasets across different domains and enable us to find many useful insights about the data and the model.</p>