Reinforcement Learning for Robotics
Biography
I am currently a doctoral fellow advised by Prof. Marco Hutter and Prof. Andreas Krause at ETH AI Center, Switzerland. Before that, I received my master’s degree in Robotics, Systems and Control from ETH Zurich and my bachelor’s degree from Tongji University.
During my master studies, I worked on structured motion representation and active learning methods at the Biomimetic Robotics Lab at Massachusetts Institute of Technology (MIT) with Prof. Sangbae Kim. Prior to that, I was engaged in agile motion imitation from limited demonstrations and intrinsic skill exploration with Prof. Georg Martius at the Autonomous Learning Group at Max Planck Institute for Intelligent Systems (MPI-IS), Tübingen, Germany.
My research interests focus on algorithms that can enable autonomous agents to acquire complex behaviors through learning, especially general-purpose methods that could enable any autonomous system to learn to solve any task. On legged robots particularly, I am thrilled to apply these techniques to achieve animal-level agility and naturalness.
For more information, please refer to my latest Resumé .
Interests
- Reinforcement Learning
- Developmental Robotics
- Legged Intelligence
Education
PhD in Robot Learning
ETH AI Center
MSc in Robotics, Systems and Control
ETH Zurich
BSc in Mechanical Engineering
Tongji University
News
2024
Apr 05: I received the ETH Medal at my master's graduation ceremony.
I feel extremely honored to receive the ETH Medal at my master's graduation ceremony at ETH Zurich. The ETH Medal is awarded to students with outstanding master's and doctoral theses. I am deeply grateful to view this as a recognition and acknowledgement of my work during my master's studies. I would like to express my sincere gratitude to my advisors Prof. Georg Martius, Prof. Sangbae Kim, and especially, Prof. Marco Hutter for their guidance and support.Apr 05: An unforgettable reunion of friends at my master's graduation ceremony.
The graduation ceremony officially concludes my master's studies at ETH Zurich. It was an unforgettable reunion of friends who shared this great journey with me. I am grateful for the opportunity to have met so many wonderful people during my studies. I am excited to see where our paths will lead us next.Mar 13: Our work was accepted to ICLR 2024 Generative Models for Decision Making workshop.
Our work FLD: Fourier Latent Dynamics for Structured Motion Representation and Learning was accepted to the Generative Models for Decision Making (GenAI4DM) workshop at The Twelfth International Conference on Learning Representations (ICLR). This workshop aims to bring together researchers and practitioners from the fields of generative AI and decision making to explore the latest advances, methodologies, and applications.Mar 04: Our defense won the second price in the LLM CTF competition at IEEE SaTML 2024.
Our team won the second price with our Llama-2-70b-chat defense in the Large Language Models Capture-the-Flag (LLM CTF) competition at 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). In this competition, participants assume the roles of defenders that craft prompts and filters to instruct an LLM to keep a secret, aiming to prevent its discovery in a conversation.Jan 29: Our work was accepted to ICRA 2024.
Our work Learning Diverse Skills for Local Navigation under Multi-constraint Optimality was accepted by IEEE International Conference on Robotics and Automation (ICRA) 2024. ICRA 2024 will be held in Yokohama, Japan from May 13 to 17, 2024.Jan 16: Our work was accepted to ICLR 2024 for a spotlight presentation.
Our work FLD: Fourier Latent Dynamics for Structured Motion Representation and Learning was accepted by The Twelfth International Conference on Learning Representations (ICLR) for a spotlight presentation. ICLR 2024 will be held in Vienna, Austria from May 7 to 11, 2024.2023
Nov 01: I started my Ph.D. journey as a doctoral fellow at ETH AI Center.
I am excited to share that I started my Ph.D. program as a doctoral fellow at ETH AI Center at ETH Zurich, Switzerland. I am honored to be co-supervised by Prof. Andreas Krause and Prof. Marco Hutter.Oct 03: I finished my visit at the Biomimetic Robotics Lab at MIT.
I am thrilled to wrap up my visit with the talented team at the Biomimetic Robotics Lab at Massachusetts Institute of Technology (MIT), Cambridge, United States. It has been an invaluable experience, collaborating on cutting-edge legged robotics. I am eager to see our collective efforts make a meaningful impact on the robot learning community!Jun 03: Great time at ICRA 2023.
It was wonderful to have met my old and new friends during IEEE International Conference on Robotics and Automation (ICRA) 2023 in London, United Kingdom. I am excited to see that our work Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions has been acknowledged by the greatest roboticists in the field. Thanks to the team Sebastian Blaes, Pavel Kolev, Marin Vlastelica, Jonas Frey and Georg Martius, who made this work possible.Apr 27: Our work was accepted to ICRA 2023 Agile Movements workshop.
Our work Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions was accepted to the Agile Movements: Animal Behavior, Biomechanics, and Robot Devices workshop at IEEE International Conference on Robotics and Automation (ICRA) 2023. This workshop invites researchers to identify the challenges and opportunities of understanding and improving robotic agility from a diverse range of areas, including biology, biomechanics, mechanism design, and control.Mar 02: I received the ETH AI Center Doctoral Fellowship.
I am excited to share that I received the ETH AI Center Doctoral Fellowship at ETH Zurich, Switzerland. ETH AI Center Fellowship is awarded to doctoral students, where young researchers are co-supervised by two faculty members from different departments within ETH Zurich, thus fostering a culture of interdisciplinary exchange and collaboration. I am extremely honored to receive this offer from Prof. Andreas Krause and Prof. Marco Hutter.Feb 16: Our work on Solo robots at MPI-IS was reported by German TV network.
My work on Solo robots during my internship at the Max Planck Institute for Intelligent Systems was covered in NANO of 3sat. In the coverage, Prof. Georg Martius explained how autonomous learning systems can obtain extraordinary intelligence from interactions with the environment.Feb 04: I received admissions to the Ph.D. program at MIT EECS and other prominent institutes.
I am excited to share that I received admission invitation to the Ph.D. program at the department of Electrical Engineering and Computer Science (EECS) and Mechanical Engineering (MechE) at Massachusetts Institute of Technology (MIT), Cambridge, United States. I was also lucky to be admitted to the Robotics Institute at Carnegie Mellon University (CMU), Pittsburgh, United States. I am honored to view these as a recognition and acknowledgement of my work during my master's studies.Jan 16: Our work was accepted to ICRA 2023.
Our work Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions was accepted by IEEE International Conference on Robotics and Automation (ICRA). ICRA 2023 will be held in London, United Kingdom from May 29 to June 2, 2023.Jan 09: I started my visit at Biomimetic Robotics Lab at MIT.
Starting from January, I am honored to work with Prof. Sangbae Kim at Biomimetic Robotics Lab at Massachusetts Institute of Technology (MIT), Cambridge, United States. My main research topic focuses on developing latent skill representation for MIT Humanoid robot. I am happy to collaborate with Steve Heim and Elijah Stanger-Jones on this exciting topic.2022
Dec 19: Great time at CoRL 2022 and satisfying results for my first publication.
It was wonderful to have met all the great names and my future collaborators and advisors during Conference on Robot Learning (CoRL) 2022 in Auckland, New Zealand. I am excited to see that our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations has been acknowledged by the greatest roboticists in the field and was announced into the finalist of the best paper award. Thanks to the team Marin Vlastelica, Sebastian Blaes, Jonas Frey, Felix Grimminger and Georg Martius, who made this achievement possible.Dec 16: Our work made it into the finalist of the best paper award at CoRL 2022, Auckland, New Zealand.
It was announced during Conference on Robot Learning (CoRL) 2022 in Auckland, New Zealand that our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations made it into the finalist of the best paper award. As one of the three nominated papers in this list, our work has received great acknowledgment from the community.Dec 02: Our work was presented to the Machines in Motion Laboratory, New York University, United States.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was presented to the scholars at the Machines in Motion Laboratory led by Prof. Ludovic Righetti at New York University. The Machines in Motion Laboratory focuses on understanding the fundamental principles for robot locomotion and manipulation that will endow robots with the robustness and adaptability necessary to efficiently and autonomously act in an unknown and changing environment.Nov 25: I will join the Biomimetic Robotics Lab at MIT for my master thesis.
I am glad to share that I will join the Biomimetic Robotics Lab led by Prof. Sangbae Kim at Massachusetts Institute of Technology (MIT) to complete my master thesis later this year. The Biomimetic Robotics Lab focuses on designing and controlling robots using insights taken from the natural world. I will be working with the MIT Humanoid on human demonstration imitation.Nov 22: Our work was accepted to CoRL 2022 Learning for Agile Robotics workshop.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was accepted to the Learning for Agile Robotics workshop at Conference on Robot Learning (CoRL) 2022. This workshop invites researchers working on making agile robots. The goals include fostering collaboration, understanding the current limitations and inefficiencies, learning common ideas on modern ML based approaches, etc.Nov 21: Our work made it into the Video Friday collection at IEEE Spectrum again.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was covered by Video Friday collection at IEEE Spectrum. Video Friday is a weekly selection of state-of-the-art robotics videos, collected by scholars at IEEE Spectrum Robotics. This time, our overview video was selected and well acknowledged.Sep 15: Our work was submitted to IEEE International Conference on Robotics and Automation (ICRA) 2023.
During the second three months of my internship, I accomplished the work Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions under the guidance by Sebastian Blaes, Pavel Kolev, Marin Vlastelica, Jonas Frey and Prof. Georg Martius. In this work, Solo obtained diverse skills by imitating unlabeled, mixed reference motions.Sep 10: Our work was accepted to CoRL 2022 for an oral presentation with best paper award nomination.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was accepted by Conference on Robot Learning (CoRL) 2022 for an oral presentation with best paper award nomination. CoRL 2022 will be held in Auckland, New Zealand from December 14 to 18, 2022.Aug 10: Our work was presented to the Robotic Systems Lab, ETH Zurich, Switzerland.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was presented to the scholars at the Robotic Systems Lab (RSL), ETH Zurich during the research presentation. In this weekly event, internal and external researchers are invited to inform their state-of-the-art discoveries in fields including robotic system integration and control methods.Jul 02: Our work was presented to the public at the Science & Innovation Days, Tübingen.
The experiments of our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations were presented to the public at the Tübingen Science & Innovation Days. In this event, research institutes including the Max Planck Institute for Intelligent Systems open their doors to the interested public and present their research.Jul 01: Our work made it into the Video Friday collection at IEEE Spectrum.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was covered by Video Friday collection at IEEE Spectrum. Video Friday is a weekly selection of state-of-the-art robotics videos, collected by scholars at IEEE Spectrum Robotics. Among our selected videos, SOLOBACKFLIP gained particular popularity.Jun 29: Our work was presented to scholars from the International Max Planck Research School for Intelligent Systems.
Our work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations was presented to the evaluation committee and Ph.D. scholars from the International Max Planck Research School for Intelligent Systems (IMPRS) during the on-site evaluation.Jun 15: Our work was submitted to Conference on Robot Learning (CoRL) 2022.
During the first three months of my internship, I accomplished the work Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations under the guidance by Marin Vlastelica, Sebastian Blaes, Jonas Frey, Felix Grimminger and Prof. Georg Martius. In this work, Solo developed highly dynamic skills by imitating rough, partial, human-demonstrated reference motions.Apr 01: I started my internship at Autonomous Learning Group at Max Planck Institute for Intelligent Systems.
Starting from April, I am honored to work with Prof. Georg Martius at Autonomous Learning Group at Max Planck Institute for Intelligent Systems (MPI-IS), Tübingen, Germany. My main research topic focuses on agile skill development and style diversification for Solo robot. This work is closely collaborated with Robotic Systems Lab (RSL), ETH Zurich, Switzerland.Mar 30: My semester project at Robotic Systems Lab (RSL) finished with successful policy deployment on the real system.
By employing a novel compositional control structure, ANYmal learned to achieve full-pose trajectory tracking in a global coordinate system. A hierarchical system pipeline was introduced, where the high-level policy proceeds global target information and outputs low-level composition signals and commands with trivial regularization.2021
Oct 01: I started my semester project at Robotic Systems Lab (RSL), ETH Zurich, Switzerland.
From October, I am honored to work on a semester project on the skill integration for ANYmal with hierarchical reinforcement learning at Robotic Systems Lab (RSL) directed by Prof. Marco Hutter. The main research topic focuses on extending locomotion and manipulation capabilities for legged systems by leveraging appropriate combination of acquired skills. This project is mainly supervised by Yuntao Ma, Takahiro Miki and Mayank Mittal.Sep 01: I started my teaching assistant position for Linear & Combinatorial Optimization at ETH Zurich, Switzerland.
After having achieved top grades at the course Linear & Combinatorial Optimization (L&CO, 11 ECTS) instructed by Prof. Rico Zenklusen at ETH Zurich last semester, I was honored to take a teaching assistant position for the new semester. Among my responsibilities, I enjoyed delivering lectures the most.Jul 9: Wonderful week at ETH Robotics Summer School in Wangen an der Aare, Switzerland.
From July 2 to 9, I was honored to participate in the ETH Robotics Summer School directed by ETH RobotX initiative. During this event, a broad scope of components of autonomous mobile robots including state estimation, trajectory optimization, environment mapping, path planning, and artifact detection were introduced and implemented on a wheeled platform, SuperMegaBot. On the last day, we successfully completed an autonomous rescue challenge at the test site.2020
Sep 01: I started my master's program in Robotics, Systems and Control at ETH Zurich, Switzerland.
After receiving offers from top-class universities including University of California, Los Angeles (UCLA), Johns Hopkins University (JHU), École polytechnique fédérale de Lausanne (EPFL), Imperial College London, etc, I decided to pursue my future endeavors in Robotics, Systems and Control under the guidance by Prof. Marco Hutter at ETH Zurich, Switzerland.Jun 15: I was awarded Excellent Graduate of Shanghai.
Together with 11 other students from the School of Mechanical Engineering, it was my honor to be granted this highest honor for all undergraduates of all majors from more than ten universities in Shanghai this year.Feb 01: My research project at Biorobotics Laboratory (BioRob) finished with successful hardware test.
My work on Analysis, Design and Implementation of a Bio-inspired Passive Tail for Amphibious Robots concluded with a novel development of an appropriate chordwise stiffness distribution along a bio-inspired tail for AgnathaX which maximizes the propulsion generated under a combined movement of both passive heaving and pitching.2019
Sep 01: I started my research project at Biorobotics Laboratory (BioRob) at École polytechnique fédérale de Lausanne (EPFL), Switzerland.
I am honored to work as a research assistant on the design of a bio-inspired passive tail for amphibious robots at Biorobotics Laboratory (BioRob) directed by Prof. Auke Ijspeert during the last year of my bachelor's study. This project is mainly supervised by Laura Paez.2018
Nov 16: I was awarded my second National Scholarship for my study at Tongji University, Shanghai, China.
The Ministry of Education of China funded the China National Scholarship to award outstanding full-time undergraduates for their excellent academic records. Being among top 1% of the students of the same year again, I am honored to receive this award a second time.Oct 01: I finished my research internship at Advanced Multifunctional and Multiphysics Metamaterials Lab (AM3L) of the Department of Bioresource Engineering of McGill University, Montréal, Canada.
From July to October, I was honored to be selected by Mitacs Globalink Research Internship program and work on Design and 3D Printing of Lightweight Advanced Energy Harvesters Made of Smart Materials under the supervision of Prof. Abdolhamid Akbarzadeh. The research was devoted into reinforcing mechanical properties of 3D printing materials by affiliating pulverized wood particles with melted polymer through a novel process of extrusion.Apr 01: I was awarded Meritorious Winner in 2018 Mathematical Contest in Modeling.
I am honored to receive my first international award in Mathematics.2016
Nov 16: I was awarded my first National Scholarship for my study at Tongji University, Shanghai, China.
The Ministry of Education of China funded the China National Scholarship to award outstanding full-time undergraduates for their excellent academic records. Being among top 1% of the students of the same year, I am honored to receive this award.Featured Publications
Projects
.js-id-featured
Proposing a constraint grouping method for diversity optimization maintaining near optimality.
Proposing a self-supervised, structured representation and generation method that extracts spatial-temporal relationships in periodic or quasi-periodic motions.
Proposing structured control policies for locomotion tasks via graph neural networks based on the robot configuration morphology.
Proposing a cooperative generative adversarial method for obtaining controllable skill sets from unlabeled datasets containing diverse state transition patterns.
Proposing a generative adversarial method for inferring reward functions from partial and potentially physically incompatible demonstrations for successful skill acquirement where reference or expert demonstrations are not easily accessible.
Employing a hierarchical control method to achieve complex high-level tasks by integrating locomotion and manipulation skills.
Employing a novel compositional control structure to allow legged systems to achieve full-pose trajectory tracking in a global coordinate system by integrating learned skills.
Learning to utilize learned low-level skills in high-level tasks by considering only high-level objectives.
Extending vanilla policy gradients with advantage estimation.
Predicting model posterior distribution with Gaussian Process regression from scratch, which is then applied to an inference problem based on space data.
Joint training of model and objective with appropriate acquisition function and constraint satisfaction.
Comparing classification uncertainty prediction with DenseNets and Bayesian Neural Networks.
Learning image embeddings and distributions given relative distance labels.
Predicting orders of medical tests, sepsis, and key vital signs based on incomplete, imbalanced data.
Tracking system states with nonlinear continuous-time dynamics with Hybrid Extended Kalman Filter.
Tracking system states with general nonlinear system and general noise distributions with Particle Filter.
Developing algorithmic components for a wheeled autonomous mobile platform that operates autonomously in search and rescue scenarios.
Developing control algorithms that simulate the flocking behavior of Boids.
Simulating rigid body dynamics by solving the equations of motion and impulse-based collisions with restitutional contact.
Finding desired joint angles in trajectory tracking problems with inverse kinematic control.
Developing algorithmic components for autonomous mobile robot integration.
Applying the Finite Element Method to simulate the mechanical properties of a soft bar applied with constraints and external forces.
Analyzing topological structures of a graph network in view of different time spans for different local measures.
Optimizing trajectories of a spaceship hitting a target and orbiting a planet in a space gravity field.
Developing nonlinear control of position and yaw angle for quadrotor systems.
Regulating continuous-time linear systems with a variety of Model Predictive Control (MPC) controllers.
Comparing computational complexity of deriving the optimal cost and control policy using value iteration, policy iteration and linear programming algorithms in a discrete infinite horizon setting.
Developing an appropriate chordwise stiffness distribution along a bio-inspired tail for amphibious robot systems which maximizes the propulsion generated under a combined movement of both passive heaving and pitching.
Reinforcing mechanical properties of 3D printing materials by affiliating pulverized wood particles with melted polymer through a novel process of extrusion.
Experience
Doctoral Fellow
Robot learning under the supervision of Prof. Marco Hutter and Prof. Andreas Krause.
Visiting Researcher
Human motion representation and developmental robotics under the supervision of Prof. Sangbae Kim.
Research Intern
Agile motion imitation and intrinsic skill exploration under the supervision of Prof. Georg Martius.
Teaching Assistant
Tutorial delivery for Linear & Combinatorial Optimization (Mathematical Optimization, 11 ECTS) instructed by Prof. Rico Zenklusen.
Research Assistant
Analysis, design and implementation of a bio-inspired passive tail for amphibious robots under the supervision of Prof. Auke Ijspeert.
Research Assistant
Wood reinforced 3D printing materials under the supervision of Prof. Abdolhamid Akbarzadeh.
Invited Talks
Robotic Platforms
I Learned and Grew up with
AgnathaX
AgnathaX is an undulatory swimming robot equipped with distributed force sensors to study the neural mechanisms controlling locomotion in the spinal cord.
ALMA
ALMA is equipped with a robotic arm designed for articulated locomotion and manipulation.
ANYmal
ANYmal is optimized for automated industrial inspection tasks with a wide range of sensors.
ANYmal on Wheels
ANYmal on Wheels is equipped with non-steerable, torque-controlled wheels to overcome the trade-off between mobility and efficiency.
Mini Cheetah
Mini Cheetah is a lightweight, high-power quadruped robot designed for extraordinarily agile motions.
MIT Humanoid
MIT Humanoid is built for highly dynamic skills over challenging terrains.
Solo
Solo is an open-source light-weight research quadruped robot that allows highly dynamic actuation.
Spot
Spot is an agile mobile robot that navigates complex terrains for automated routine inspection and data capture.
SuperMegaBot
SuperMegaBot is a 4-wheeled mobile robot for autonomous site inspection and artifact detection.
Contact
- chenhli@ethz.ch
- Rämistrasse 101, 8092 Zurich
- Message Me