University of York
Combining Event Cameras with Spiking Neural Networks for Bio-Inspired Event-Based Visual Inertial Odometry
Abstract
dc:description.abstractThe domain of Neuromorphic Computing applies models and concepts from neuroscience, emulating the computational behaviours observed in biological neurons through the medium of spiking neural networks, and applying them to engineering tasks. In recent years, Neuromorphic Computing has seen rapid growth, with applications that have transitioned from lab-based testing and simulation, to real-world scenarios. Navigation tasks are the subject of extensive and varied research due to the variety of implementations afforded by sensor fusion and the range of possible solutions, e.g. Simultaneous Localisation and Mapping (SLAM), Visual Inertial Odometry (VIO), visual place recognition. An entirely event-based system capable of performing VIO on real-world data has yet to be proposed. Here, we combine event cameras with spiking neural networks to perform localisation tasks on complex real-world driving data, creating a spike-based representation of position and orientation. We show how navigation-oriented neural networks that have been observed in the mammalian brain, such as head direction, grid, and place cells, can be adapted to track both orientation and position in real-world driving scenarios through inertial signals. It is also shown that stereo event-based visual place recognition can be combined with inertial integration for loop closure, providing a robust solution that corrects accumulated errors. These results show the viability of a system that can take advantage of the asynchronous, low-power benefits of neuromorphic processing in an end-to-end fashion by maintaining sparse event encoding from the event camera input to the spiking neural network outputs. It is anticipated that neuromorphic navigation systems can provide an optimal solution for applications where low latency and low power are constraints, such as autonomous robotics. This work should be expanded upon with the development of more complex visual place recognition spiking neural networks, refinement of the spatial network behaviours, and with increased dimensionality to allow flying agents to also be tracked accurately through 3D space.
Degree
thesis:*- Name dc:type.qualificationname
- Ph.D
- Level dc:type.qualificationlevel
- doctoral
- Grantor dc:publisher.institution
- University of York
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Mackenzie, Gregor Sean
- Advisor dc:contributor.advisor
-
- Halliday, David
- ORCID person:identifier.orcid
Rights
- Language dc:language.dcterms_iso639_2
- eng