Work

I'm interested in the intersection of machine learning and control, with applications to experimental robotics. With Kris, I am working on direct synthesis of robot controllers with model-based reinforcement learning where we do not need any past system knowledge. For an overview of my recent work, you can find a shortened version of my qualifying exam slides here, or a private recording here.

My high level interests.
  1. Novel Robotics: I want to be able to build useful robots from whatever pieces an engineer has.

  2. Model-based Reinforcement Learning: I am optimistic about interpretable learning for robots.

  3. Robot Learning in Weak-sensor Environments: As a practical roboticist (or a data-scientist), I want to make systems that work in all parts of the world.

Research

Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning
Nathan Lambert, Albert Wilcox, Howard Zhang, Kristofer S.J. Pister, Roberto Calandra
Arxiv Pre-print, 2020.
PDF

In this paper, we propose a new parametrization to supervised learning on state-action data to stably predict at longer horizons – that we call a trajectory-based model.

Nonholonomic Yaw Control of an Underactuated Flying Robot with Model-based Reinforcement Learning
Nathan Lambert, Daniel Drew, Craig Schindler, Kristofer S.J. Pister
Arxiv Pre-print, 2020.
Arxiv   /  PDF

In the context of novel robotics, which are often hand assembled, a perfect understanding of the dynamics is improbable. We investigate if the performance of a novel nonholonomic control law can be matched with data-driven, reinforcement learning techniques.

Learning for Microrobot Exploration: Model-based Locomotion, Robust Navigation, and Low-Power Deep Classification
Nathan Lambert, Fahran Toddywala, Brian Liao, Eric Zhu, Lydia Lee, Kristofer S.J. Pister
International Conference on Manipulation, Automation and Robotics at Small Scales, 2020.
Arxiv   /  PDF   /  More

A collections of steps towards an autonomous microrobot. Recent work has pushed capabilities of the device forward, but little progress has been made in creating an autonomous platform.

Objective Mismatch in Model-based Reinforcement Learning
Nathan Lambert, Brandon Amos, Omry Yadan, Roberto Calandra
Learning for Decision and Control, 2020.
Arxiv   /  PDF   /  Workshop Presentation   /  More

Studying the numerical effects of a dual-optimization problem in model-based reinforcement learning. When optimizing model accuracy, there is no guarantee on improving task performance!

Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning
Tianyu Li, Nathan Lambert, Roberto Calandra, Franziska Meier, Akshara Rai
International Conference on Robotics and Automation, 2020.
Paper   /  Related Press

Learning how to walk with a real-world hexapod using a hierarchy of model-free RL for basic motion primitives with model-based RL for higher level planning.

Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning
Nathan Lambert, Daniel Drew, Joseph Yaconelli, Roberto Calandra, Sergey Levine, Kristofer Pister
IEEE Robotics and Automation Letters (RA-L), 2019.
Paper  /  website

We used deep model-based reinforcement learning to have a quadrotor learn to hover from less than 7 minutes of all experimental training data. No system knowledge was needed for these experiment, reading raw sensor values and commanding motor PWMs.

Toward Controlled Flight of the Ionocraft: A Flying Microrobot Using Electrohydrodynamic Thrust With Onboard Sensing and No Moving Parts
Daniel Drew, Nathan Lambert, Craig Schindler, Kris Pister
IEEE Robotics and Automation Letters (RA-L), 2018.
Paper  /  website

A collection of steps towards controlled flight of The Ionocraft, a completely silent microrobot with ion thrust!

Enhanced Lithium Niobate Pyroelectric Ionizer for Chip-Scale Ion Mobility-Based Gas Sensing
K.B. Vinayakumar, Ved Gund, Nathan Lambert, S Lodha, Amit Lal
IEEE Sensors, 2016.
Paper  

We used a pyroelectric crystal to cause dielectric breakdown events in the air, which can be used for chip scale ion based gas sensing.

Teaching

row

Lecturer - CS188 Introduction to Artificial Intelligence, Spring 2020.
Graduate Student Instructor - Summer, Fall 2020.
Website / Co-Instructor / Lecture 0 Cast / Other Lectures

Graduate Student Instructor - EECS16B Designing Information Devices and Systems II, Fall 2019.
Website / Instructor

row

Teaching Assistant - ECE 3250: Mathematics of Signal and System Analysis, Fall 2016.
Lectures / Instructor

Grader - ECE 4320: Integrated Micro-Sensors and Actuators, Spring 2017.
Instructor