Målet med projektet är att skapa en fungerande hörseltränings prototyp som kan vara användbar för en vidareutveckling av träningssystem för personer med nedsatt hörsel. Idén bygger på “The Music Puzzle” (Hansen et al. 2012). Prototypen benämndes SoundCubes och består av tre kuber med olika motiv (Fiducial markörer) på fyra av de sex sidorna. Ett ljudspår delades i tre delar och tilldelades till kuberna. Varje del fragmenterades i fyra nivåer där den första spelar upp ett instrument ända fram till att alla instrumenten hörs. Dessa faser tilldelades till Fiducial markörerna på kubens fyra sidor. Kuberna är slumpmässigt placerade horisontellt framför en kamera som är ansluten till en dator. Alla fyra sidor av kuberna som har motivet fastsatt kommer att spela upp reproduktionen av ett fragment från ljudspåret när de ställs inför kamerans synfält. Endast en specifik kombination av sidorna och horisontell ordning av kuberna kommer att leda till de fullständiga ljudspårskomponenterna och till ett applådljud som signalerar den framgångsrika reproduktionen av det ursprungliga ljudspåret. SoundCubes prototypen utvecklades funktionellt genom TUIO, ReacTIVision och Pure Data. För att kunna fastställa ifall prototypens interaktion fungerade så testades SoundCubes på 15 friska personer utan nedsatt hörsel. Testpersonerna introducerades individuellt till prototypen, och de fick sedan lyssna på det kompletta ljudspåret. Därefter fick de uppgiften att reproducera det kompletta ljudspåret genom att hitta kubernas rätta sidor och horisontella ordning. Av totalt 15 individer klarade 12 av testet inom tidsgränsen (tio minuter). Vi kunde därför komma fram till att prototypen och dess interaktion fungerar, vi kunde även observera att användningen av SoundCubes upplevdes som underhållande och engagerande. Detta första test kan därför lägga grund för vidareutveckling av SoundCubes konceptet för hörselträning på ett interaktivt och underhållande sätt. / The aim of the present project was to create a functional sound-training prototype which may be useful for further development of training systems for individuals with impaired hearing. The idea is based on the concept of “The Music Puzzle” (Hansen et al. 2012). The prototype proposed here was termed SoundCubes and it is composed of three cubes with different motives (Fiducial markers) on four of the six sides. A soundtrack was divided into three parts and was assigned to the cubes. Each part was fragmented into four parties where the first playing an instrument until all the instruments could be heard. These parts were assigned to Fiducial markers on the four sides of the cube. The cubes are randomly placed horizontally in front of a camera attached to a computer. All of the four sides with a motive of the cubes will induce the reproduction of fragments of a soundtrack when exposed to the field of the camera thanks to the specific motives attached on it. Only one specific combination of the sides and horizontal order of the cubes will lead to the complete soundtrack instrumental components and to a clapping sound which signals the successful reproduction of the2initial soundtrack. The SoundCubes prototype was developed functionally through Tuio, ReacTIVision and Pure Data. SoundCubes was tested on 15 healthy individuals without hearing impairments to determine if the prototype’s interaction works. The test individuals were individually introduced to SoundCubes, and they were allowed to hear the complete soundtrack. Thereafter, they were given the task of reproducing the complete track by finding the right side and horizontal order of the cubes. Out of a total of 15 individuals, 12 completed the test successfully within the maximum given time frame (ten minutes).We could therefore conclude that the system works properly, and we also observed that the test was experienced as very entertaining and engaging. this first test puts therefore the basis for further development of the SoundCubes concept to train people’s hearing in an interactive and entertaining way.Kjetil

<p>Klassisk mus-interaktion har begränsningen att bara en punkt kan aktiveras på skärmen i taget. Interaktionen med de flesta applikationer blir, på grund av detta, sekventiell vilket kan vara en begränsning eftersom människan är van att utforska problem simultant med båda händerna. Flerpunktspekskärmar är en relativt ny teknik som möjliggör nya interaktionsmöjligheter där flera pekpunkter kan vara aktiva simultant. Det här examensarbetet fokuserar på problemen kring att skapa en applikation med användargränssnitt för flerpunktspekskärmar. Applikationen som utvecklats har en grundstruktur vilken är möjlig att vidareutveckla och på så sätt anpassas för att passa nya behov, till skillnad från de implementationer som granskats vid förstudien vilka fokuserar på att lösa specifika problem. Under examensarbetet har också en flerpunktspekskärm byggts av komponenter som kunnat köpas i detaljhandeln.</p> / <p>Classical mouse interaction is limited in that only a single screen point at a time can be activated. Most application interaction is, due to this, performed in a sequential manner. This may be a limiting factor as humans by nature explore problems with both hands simultaneously. Multi-touch screens are a relatively new type of technology that enables new ways of interaction where multiple touch points can exist simultaneously. This thesis focuses on the problem with developing an application with a user interface designed for multi-touch screens. The application that has been developed defines a base structure from which future applications can be developed, unlike implementations examined in the feasibility study which mainly focus on solving specific problems. A multi-touch screen is also built using standard off the shelf components.</p>

Klassisk mus-interaktion har begränsningen att bara en punkt kan aktiveras på skärmen i taget. Interaktionen med de flesta applikationer blir, på grund av detta, sekventiell vilket kan vara en begränsning eftersom människan är van att utforska problem simultant med båda händerna. Flerpunktspekskärmar är en relativt ny teknik som möjliggör nya interaktionsmöjligheter där flera pekpunkter kan vara aktiva simultant. Det här examensarbetet fokuserar på problemen kring att skapa en applikation med användargränssnitt för flerpunktspekskärmar. Applikationen som utvecklats har en grundstruktur vilken är möjlig att vidareutveckla och på så sätt anpassas för att passa nya behov, till skillnad från de implementationer som granskats vid förstudien vilka fokuserar på att lösa specifika problem. Under examensarbetet har också en flerpunktspekskärm byggts av komponenter som kunnat köpas i detaljhandeln. / Classical mouse interaction is limited in that only a single screen point at a time can be activated. Most application interaction is, due to this, performed in a sequential manner. This may be a limiting factor as humans by nature explore problems with both hands simultaneously. Multi-touch screens are a relatively new type of technology that enables new ways of interaction where multiple touch points can exist simultaneously. This thesis focuses on the problem with developing an application with a user interface designed for multi-touch screens. The application that has been developed defines a base structure from which future applications can be developed, unlike implementations examined in the feasibility study which mainly focus on solving specific problems. A multi-touch screen is also built using standard off the shelf components.

Οι τεχνολογίες πολλαπλής αφής παρουσιάζουν μεγάλο ερευνητικό και τεχνολογικό ενδιαφέρον στον τομέα της αλληλεπίδρασης ανθρώπου-υπολογιστή από τις προηγούμενες κιόλας δεκαετίες. Το κόστος όμως τέτοιων συστημάτων καθιστούσε απαγορευτική τη χρήση τους εκτός έξω από το χώρο ερευνητικών εργαστηρίων. Οι εξελίξεις στον τομέα αυτό είναι ραγδαίες με αποτέλεσμα πλέον τα συστήματα αυτά να καθίστανται προσιτά και ιδιαίτερα ελκυστικά, προσφέροντας έτσι τη δυνατότητα ευρύτερης αποδοχής τους. Η διπλωματική αυτή εργασία, που εκπονήθηκε στο Εργαστήριο της Ερευνητικής Ομάδας Αλληλεπίδρασης Ανθρώπου Υπολογιστή του Τμήματος Ηλεκτρολόγων Μηχανικών και Τεχνολογίας Υπολογιστών του Πανεπιστημίου Πατρών, υπό την επίβλεψη του Καθ. Νικόλαου Αβούρη, περιγράφει σε πρώτο στάδιο τη μελέτη και αξιολόγηση των τεχνολογιών αφής και ιδιαίτερα των οπτικών καθώς αυτές επιτρέπουν την κατασκευή συστημάτων μεγάλων διαστάσεων. Έπειτα από επιλογή της καταλληλότερης τεχνολογίας με βάση τις προαπαιτούμενες προδιαγραφές παρουσιάζεται η κατασκευή ενός λειτουργικού συστήματος πολλαπλής αφής με χρήση της τεχνικής ‘Diffused Illumination’. Η αναγνώριση των σημείων αφής επετεύχθη με χρήση του προγράμματος ανοιχτού λογισμικού CommunityCoreVision (CCV). Στόχος της κατασκευής είναι η αξιολόγηση του ίδιου του συστήματος καθώς και η έρευνα και η αξιολόγηση νέων τρόπων αλληλεπίδρασης με το σύστημα αυτό. Η αξιολόγηση πραγματοποιήθηκε μέσω πειραματικής διαδικασίας που κατέδειξε την αποτελεσματικότητα του συστήματος στη διαχείριση αντικειμένων σε σχέση με έναν επιτραπέζιο υπολογιστή. Η εφαρμογή του πειράματος αναπτύχθηκε με τη γλώσσα Python και το Kivy. Η διαδικασία αυτή οδήγησε στην απόκτηση πρακτικών και θεωρητικών γνώσεων σε μεγάλο μέρος του γνωστικού φάσματος του τομέα της κατασκευής ολοκληρωμένων συστημάτων. / When interacting with a regular desktop computer, indirect devices such as a mouse or keyboard are used to control the computer. Results of the interaction are displayed on a monitor. Current operating systems are restricted to one pointing device. With the introduction of multi-touch, a new form of human computer interaction is introduced. Multi-touch combines display technology with sensors which are capable of tracking multiple points of input. The idea is that this would allow users to interact with the computer in a natural way. Furthermore, unlike interaction on a desktop computer, multi-touch allows multiple users to interact with the same devices at the same time. Due to recent innovations multi-touch technology has become affordable. For this project an optical based multi-touch device has been designed and constructed at the Human Computer Interaction group of the Electrical and Computer Engineering department of the University of Patras. To perform multi-touch point tracking we used CommunityCoreVision (CCV), a free open source multi-touch framework. To demonstrate the possibilities of multi-touch input technology the system was tested with existing applications, which are controlled by gestures. Multi-touch systems are often stand-alone systems that do not have external input devices attached, however, such devices can be used to support difficult task such as writing. In order to simplify common tasks, a gesture recognition engine has been evaluated along with a multi-touch mouse driver compatible with Windows 7 that supports TUIO messages. Through an experiment designed with the Python based Kivy framework, we evaluate how multi-touch input performs on a specific object manipulation task compared to conventional mouse input.