Bibliographic Details
| Title: |
What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study |
| Authors: |
Chen, Jiayu, Yu, Chao, Xie, Yuqing, Gao, Feng, Chen, Yinuo, Yu, Shu'ang, Tang, Wenhao, Ji, Shilong, Mu, Mo, Wu, Yi, Yang, Huazhong, Wang, Yu |
| Publication Year: |
2024 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Robotics, Computer Science - Machine Learning |
| More Details: |
Executing precise and agile flight maneuvers is critical for quadrotors in various applications. Traditional quadrotor control approaches are limited by their reliance on flat trajectories or time-consuming optimization, which restricts their flexibility. Recently, RL-based policy has emerged as a promising alternative due to its ability to directly map observations to actions, reducing the need for detailed system knowledge and actuation constraints. However, a significant challenge remains in bridging the sim-to-real gap, where RL-based policies often experience instability when deployed in real world. In this paper, we investigate key factors for learning robust RL-based control policies that are capable of zero-shot deployment in real-world quadrotors. We identify five critical factors and we develop a PPO-based training framework named SimpleFlight, which integrates these five techniques. We validate the efficacy of SimpleFlight on Crazyflie quadrotor, demonstrating that it achieves more than a 50% reduction in trajectory tracking error compared to state-of-the-art RL baselines. The policy derived by SimpleFlight consistently excels across both smooth polynominal trajectories and challenging infeasible zigzag trajectories on small thrust-to-weight quadrotors. In contrast, baseline methods struggle with high-speed or infeasible trajectories. To support further research and reproducibility, we integrate SimpleFlight into a GPU-based simulator Omnidrones and provide open-source access to the code and model checkpoints. We hope SimpleFlight will offer valuable insights for advancing RL-based quadrotor control. For more details, visit our project website at https://sites.google.com/view/simpleflight/.
Comment: The first two authors contribute equally |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2412.11764 |
| Accession Number: |
edsarx.2412.11764 |
| Database: |
arXiv |
