Back to search

Georgia Institute of Technology

Contrastive Multiview Coding with Hard Negatives for Multimodal Human Activity Recognition

Abstract

dc:description.abstract

Human activity recognition (HAR) systems have been extensively studied by the vision and ubiquitous computing communities due to their practical applications in daily life, such as smart homes, surveillance, and health monitoring. However, supervised HAR training requires a large amount of annotated data, making it costly and time-consuming. To address this challenge, contrastive self-supervised learning (SSL) based approaches have gained a lot of attention in the HAR community, as they learn from unannotated data. Nevertheless, a remaining challenge in SSL for HAR is the reliance on positive and negative pairs for feature representation learning. While obtaining true positive pairs is relatively easy and can be achieved by augmenting the anchor sample, obtaining true negative pairs is challenging, as negative samples are uniformly selected from a batch, which may correspond to false negative samples with the same label as the anchor class. In this paper, we propose a hard negative sampling method for multimodal HAR by implementing contrastive multiview coding (CMC) with a hard negative sampling loss namely, CMC-HNL. We exploit hard negatives that have different labels from the anchor and are projected nearby in the latent space with an adjustable concentration parameter to balance improved learning from hard negatives and the potential negative impact of approximately correcting false negatives. Through extensive experiments on two multimodal HAR datasets, UTD-MHAD and MMAct, we demonstrate the robustness of our approach for feature representation learning and semi-supervised learning. Our model outperforms the supervised model and state-of-the-art methods.

Degree

thesis:*
Level thesis:degree_level
Masters
Department dc:contributor.department
Electrical and Computer Engineering
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2023

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Choi, Hyeongju
Advisor dc:contributor.advisor
  • Essa, Irfan
Committee members dc:contributor.committeemember
  • Yeo, Woon-hong
  • Heck, Larry

Subjects

dc:subject × 5

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/75124

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Choi, Hyeongju. Contrastive Multiview Coding with Hard Negatives for Multimodal Human Activity Recognition. Masters thesis, Georgia Institute of Technology, 2023. https://hdl.handle.net/1853/75124