Back to search

Democritus University of Thrace (DUTH)

Spiking neural networks for perception and control in human assistive robotic systems

Abstract

dc:description

The growing demand for intelligent and energy-efficient systems in real-world applications has emerged the exploration of neuromorphic computing approaches, such as spiking neural networks (SNNs). The motivation behind this dissertation stems from the need to design computationally efficient, bioinspired learning models capable of handling complex perception and control tasks, particularly under constrained hardware and energy budgets. Spiking neural networks offer a promising path forward due to their temporal processing capabilities and compatibility with neuromorphic hardware. Thus, this thesis tackles a cascade problem in the context of multimodal perception and control by leveraging the unique properties of SNNs. Specifically, it focuses on the development and evaluation of spike-based learning architectures across three interrelated domains: discriminative multimodal feature discrimination, object manipulation and robotic arm control via reinforcement learning (RL). An important application of this work is Ambient Assisted Living (AAL), where systems must operate reliably and efficiently in non-intrusive ways to support the well-being and autonomy of elderly individuals. The contribution of this dissertation to the state-of-the-art is substantiated through a series of peer-reviewed publications. Additionally the main contributions can be summarized as follows: Introduces the first spiking feature discrimination framework, opening a novel research direction in multimodal spiking perception. Presents the first successful application of gradient-based RL with spiking neural networks for robotic control. Demonstrates the application of this approach to complex tasks such as object manipulation, showcasing its practical utility in robotic systems. Advances biologically inspired machine intelligence for human-assistive robotic systems, particularly within the scope of AAL. The dissertation at hand is articulated into six chapters. Beginning with Chapter 1, it laid the theoretical foundation of this dissertation. It introduces the principles of spiking neural networks, neuron models (e.g., LIF, Izhikevich), encoding strategies (i.e., rate, temporal), and plasticity mechanisms. Additionally, this chapter presents an extended literature review, which presents spiking RL approaches in robotics systems, focused on learning mechanisms, neuronal models, and evaluation metrics. The spiking RL models fall into two primary categories, the bioinspired (e.g. STDP-based) and the gradient-based adaptations of traditional RL methods. Thus, we further categorized the surveyed works based in robotic application, extracting valuable information about inadequacies in the field along with future directions. This chapter also provides an overview of feature discrimination methods and multimodal fusion approaches. Lastly, it clarifies the baseline of SNN research and contextualizes the challenges related to convergence, scalability, and real-world deployment. Chapter 2 presents the databases exploited to conduct our empirical studies. The above includes the utilized datasets from the field of audio-visual classification, along with the custom one, specifically collected for the task of elderly human action recognition in the field of AAL. The chapter also introduces the metrics adopted to quantify classification performance, feature separability, and robotic control accuracy. Proceeding with the main analysis of our study, Chapter 3 introduces a novel spiking multimodal fusion architecture for perception tasks. It paves the path in spiking perception with a spiking multimodal feature discrimination pipeline, where both visual and auditory modalities are processed using SNNs. Unlike existing multimodal approaches using traditional deep layers, the proposed pipeline emphasizes early-stage feature discrimination using spiking layers tailored for each modality. A spiking residual network processes the visual input, while an efficient auditory SNN complements the temporal dynamics of the auditory inputs. Subsequently, these networks are fused through a novel spiking MLP (SMLP) incorporating an L2-normalization layer to maximize angular separation in the latent feature space. The results reveal that spiking feature discrimination is a prominent research field, not only matching but in some approaches exceeding the performance of classical networks under low-power constraints. Following this, an action recognition application for AAL is examined. Here, the challenge is to identify and classifyelderly individuals’ actions using non-intrusive, low-power sensor setups. This chapter introduces a complete pipeline for real-time action recognition based on RGB video inputs. The proposed approach utilizes a biologically-inspired Hierarchical Temporal Memory (HTM), and SNNs implemented in Nengo and evaluates their suitability in constrained data availability approaches. The results reveal that spiking models can not only match but also outperform conventional models. This application directly supports the integration of bioinspired intelligence in AAL systems that must function continuously and discreetly to support human living. The fourth and fifth chapters focus on the integration of spiking neural networks into reinforcement learning frameworks for robotic manipulation tasks. Recognizing the limitations of conventional deep reinforcement learning (DRL) approaches, such as their high energy consumption and dependence on resource-intensive training, these chapters proposes a hybrid framework that combines the temporal efficiency of SNNs with the optimization capacity of deep networks. Specifically, a hybrid Deep Deterministic Policy Gradient (DDPG) algorithm is introduced, where the actor network is realized as a spiking neural network while the critic is retained as a conventional deep network, used solely during the training phase. This architectural decoupling facilitates several methodological advantages. First, the spiking actor network is trained to generate motor commands directly from sensory states, enabling real-time inference compatible with deployment. Once training is complete, the energy-intensive critic is discarded, and only the spiking actor remains active during inference, significantly reducing computational and power demands. This design was specifically designed and adapted for two distinct robotic control scenarios. The first involves a 2-degree-of-freedom robotic arm equipped with a dual-finger gripper, tasked with performing object manipulation through reach, grasp, and transfer operations. The second scenario addresses a more complex 6-degree-of-freedom robotic arm required to perform 3D target-reaching, incorporating both RGB imagery and laser-based range sensing for planning and obstacle avoidance. Experimental results across both scenarios demonstrated that the hybrid approach not only matches but in some cases surpasses the performance of classical DDPG baselines in terms of task success rate, trajectory smoothness, and convergence time. More importantly, it offers a substantial improvement in energy efficiency, as the spiking actor alone is responsible for action generation during deployment. This validates the practical viability of the proposed hybrid architecture in scenarios where energy constraints are present, such as in AAL systems. The dissertation concludes with Chapter 6 summarizing our findings and their contribution to the field. Finally, we discuss the open challenges and future directions that aim to extend the research described here.

Degree

thesis:*
Grantor dc:publisher
Democritus University of Thrace (DUTH)
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Oikonomou, Aikaterini-Maria
  • Οικονόμου, Αικατερίνη-Μαρία

Subjects

dc:subject × 21

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/59503

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Oikonomou, Aikaterini-Maria; Οικονόμου, Αικατερίνη-Μαρία. Spiking neural networks for perception and control in human assistive robotic systems. Democritus University of Thrace (DUTH), 2025. http://hdl.handle.net/10442/hedi/59503