While shopping in a retail store today the customers are often interacting with multiple sale channels when making a purchase. These channels can include in‑store terminals, the customers’ smartphone etc. This thesis looks at how physical mobile interaction can be implemented to enhance the experience when using different channels for a purchase. Physical mobile interaction is a way for a user to interact with real world objects using a mobile device. This thesis examines how some of these techniques can be used in a retail store using the customers’ own smartphone device. With todays smartphone market with several different operating systems available it is important to find find an implementation which suits the majority of the users. This paper details which technologies that are available to implement the physical mobile interaction on the major smartphone platforms. It also discusses important design decisions for it to work on these platforms. The paper concludes that it is possible to implement these techniques on todays major smartphone platforms. It also concludes that the best technology can not always be used because of limitations on one or more platforms.

The prevalent visions of ambient intelligence leverage natural interactions between users and available services in a smart space. In recent years, we have seen a huge interest from industry and academia in using handheld devices to interact with things, places and people in the real world. To facilitate such interactions, things are usually annotated with RFID tags or visual markers. These tags or markers are read by a handheld device equipped with an integrated RFID reader or a camera, in order to fetch related information and initiate further actions. Interacting with the Internet of Things (IoT) in a real environment has become increasingly desirable and feasible. This thesis contributes to the domain of physical interactions with IoT; however, we use a spatial-geometric approach instead of RFID or marker based solutions. Using this approach, for example, a user can point his/her handheld device to an annotated thing, from a distance, for the purpose of interaction. The pointing direction and location is determined based on the fusion of the mobile position and of the accelerometer data of the handheld device. To annotate things, their geometric coordinates are specified and related information or services are associated to them. In this thesis, we present a comprehensive and extensible framework to integrate various physical interactions with IoT into multimedia applications. The framework supports the implementations of pointMe, touchMe, and context-aware based interactions with geometrically annotated IoT. We define specific methods and practices that can be incorporated in order to build the interactions. We realize smart home, atlas learning, presentation interaction, smart haptic interaction, and learning based video interaction game prototypes in order to perform experiments and demonstrate the applicability and potential of the proposed geometric based annotation approach. In the analysis of the interaction techniques of the prototypes, we present the advantages and disadvantages of the geometric based annotation of IoT as seen by potential users, in comparison to RFID tags or visual markers based approaches.