Back to search

Cal Poly

Design and Parametric Study of a MEMS-based Reservoir Computer for Reinforcement Learning

Abstract

dc:description.abstract

Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks with both algorithms. Most notably, the system learned a successful gait for the 4-degree-of-freedom bipedal locomotion task, BipedalWalker-v3. In the parametric studies, increasing reservoir size led to increased robustness to random initialization while increasing typical performance. A strong sensitivity to reservoir initialization was observed, motivating reservoir initialization searches during tuning. Optimal neuron separation was shown to be task dependent; however, each task was solved with neuron separations near the fundamental period of the MEMS device. In contrast to ultra-fast optical RCs, MEMS-based reservoir computers can operate at frequencies compatible with real-time sensing and control, enabling integration into mechanical systems. Overall, this work demonstrates the successful application of reinforcement learning algorithms to MEMS-based reservoir computing on both discrete and continuous control tasks while highlighting the relationship between reservoir dynamics and reinforcement learning performance.

Degree

thesis:*
Name thesis:degree_name
MS in Mechanical Engineering
Discipline thesis:degree_discipline
Mechanical Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Carr, Andrew P
Contributors dc:contributor
  • Mohammad Hasan
  • Mechanical Engineering
  • College of Engineering

Subjects

dc:subject × 12

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.calpoly.edu/theses/3304

Chain of custody

source
Harvested from
Cal Poly
Base URL
digitalcommons.calpoly.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Carr, Andrew P. Design and Parametric Study of a MEMS-based Reservoir Computer for Reinforcement Learning. https://digitalcommons.calpoly.edu/theses/3304