{"id":{"repo_id":"calpoly","oai_identifier":"oai:digitalcommons.calpoly.edu:theses-4993"},"canonical_url":"https://search.ndltd.org/etd/calpoly/oai:digitalcommons.calpoly.edu:theses-4993","repository":{"repo_id":"calpoly","name":"Cal Poly","base_url":"https://digitalcommons.calpoly.edu/do/oai/"},"display":{"title":"Design and Parametric Study of a MEMS-based Reservoir Computer for Reinforcement Learning","abstract":"Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks with both algorithms. Most notably, the system learned a successful gait for the 4-degree-of-freedom bipedal locomotion task, BipedalWalker-v3. In the parametric studies, increasing reservoir size led to increased robustness to random initialization while increasing typical performance. A strong sensitivity to reservoir initialization was observed, motivating reservoir initialization searches during tuning. Optimal neuron separation was shown to be task dependent; however, each task was solved with neuron separations near the fundamental period of the MEMS device. In contrast to ultra-fast optical RCs, MEMS-based reservoir computers can operate at frequencies compatible with real-time sensing and control, enabling integration into mechanical systems. Overall, this work demonstrates the successful application of reinforcement learning algorithms to MEMS-based reservoir computing on both discrete and continuous control tasks while highlighting the relationship between reservoir dynamics and reinforcement learning performance.","abstract_html":"Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks with both algorithms. Most notably, the system learned a successful gait for the 4-degree-of-freedom bipedal locomotion task, BipedalWalker-v3. In the parametric studies, increasing reservoir size led to increased robustness to random initialization while increasing typical performance. A strong sensitivity to reservoir initialization was observed, motivating reservoir initialization searches during tuning. Optimal neuron separation was shown to be task dependent; however, each task was solved with neuron separations near the fundamental period of the MEMS device. In contrast to ultra-fast optical RCs, MEMS-based reservoir computers can operate at frequencies compatible with real-time sensing and control, enabling integration into mechanical systems. Overall, this work demonstrates the successful application of reinforcement learning algorithms to MEMS-based reservoir computing on both discrete and continuous control tasks while highlighting the relationship between reservoir dynamics and reinforcement learning performance.","abstract_has_math":false,"creators":["Carr, Andrew P"],"institution":null,"degree_name":"MS in Mechanical Engineering","degree_level":null,"degree_discipline":"Mechanical Engineering","degree_department":null,"school":null,"contributors":["Mohammad Hasan","Mechanical Engineering","College of Engineering"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T07:12:51Z","subjects":["Reservoir computing","single-node","reinforcement learning","RC","RL","MEMS","Acoustics, Dynamics, and Controls","Electro-Mechanical Systems","Hardware Systems","Other Computer Engineering","Other Electrical and Computer Engineering","Robotics"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.calpoly.edu/theses/3304","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.calpoly.edu/theses/3304","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.calpoly.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.calpoly.edu%3Atheses-4993","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Mohammad Hasan","Mechanical Engineering","College of Engineering"]},{"key":"dc:creator","label":"Author","values":["Carr, Andrew P"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2026-06-11T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2026-06-01T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Mechanical Engineering"]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Mechanical Engineering"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Reservoir computing","single-node","reinforcement learning","RC","RL","MEMS","Acoustics, Dynamics, and Controls","Electro-Mechanical Systems","Hardware Systems","Other Computer Engineering","Other Electrical and Computer Engineering","Robotics"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.calpoly.edu/theses/3304"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks with both algorithms. Most notably, the system learned a successful gait for the 4-degree-of-freedom bipedal locomotion task, BipedalWalker-v3. In the parametric studies, increasing reservoir size led to increased robustness to random initialization while increasing typical performance. A strong sensitivity to reservoir initialization was observed, motivating reservoir initialization searches during tuning. Optimal neuron separation was shown to be task dependent; however, each task was solved with neuron separations near the fundamental period of the MEMS device. In contrast to ultra-fast optical RCs, MEMS-based reservoir computers can operate at frequencies compatible with real-time sensing and control, enabling integration into mechanical systems. Overall, this work demonstrates the successful application of reinforcement learning algorithms to MEMS-based reservoir computing on both discrete and continuous control tasks while highlighting the relationship between reservoir dynamics and reinforcement learning performance."]},{"key":"dc:title","label":"Title","values":["Design and Parametric Study of a MEMS-based Reservoir Computer for Reinforcement Learning"]}]}],"canonical_facts":{"dc:contributor":["Mohammad Hasan","Mechanical Engineering","College of Engineering"],"dc:creator":["Carr, Andrew P"],"dc:date.available":["2026-06-11T07:00:00Z"],"dc:date.created":["2026-06-01T07:00:00Z"],"dc:description.abstract":["Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks with both algorithms. Most notably, the system learned a successful gait for the 4-degree-of-freedom bipedal locomotion task, BipedalWalker-v3. In the parametric studies, increasing reservoir size led to increased robustness to random initialization while increasing typical performance. A strong sensitivity to reservoir initialization was observed, motivating reservoir initialization searches during tuning. Optimal neuron separation was shown to be task dependent; however, each task was solved with neuron separations near the fundamental period of the MEMS device. In contrast to ultra-fast optical RCs, MEMS-based reservoir computers can operate at frequencies compatible with real-time sensing and control, enabling integration into mechanical systems. Overall, this work demonstrates the successful application of reinforcement learning algorithms to MEMS-based reservoir computing on both discrete and continuous control tasks while highlighting the relationship between reservoir dynamics and reinforcement learning performance."],"dc:identifier":["https://digitalcommons.calpoly.edu/theses/3304"],"dc:subject":["Reservoir computing","single-node","reinforcement learning","RC","RL","MEMS","Acoustics, Dynamics, and Controls","Electro-Mechanical Systems","Hardware Systems","Other Computer Engineering","Other Electrical and Computer Engineering","Robotics"],"dc:title":["Design and Parametric Study of a MEMS-based Reservoir Computer for Reinforcement Learning"],"thesis:degree_discipline":["Mechanical Engineering"],"thesis:degree_name":["MS in Mechanical Engineering"]},"updated_at":"2026-11-19T07:12:51Z"}