A realistic simulation of a UAV model in Unity

for learning control policies in OpenAI Gym

Lev Kisselyov



UAV simulation is a complex field that is rapidly evolving, resulting in numerous available software options. The integration and cross-compatibility of the simulators pose a challenge for researchers in designing new robotic applications. Therefore, there is a constant need to expand the existing toolkit stack and integrate the new backward-compatible simulation environments that bring new perspectives to the research by utilizing their advantages. This work introduces Unity3D as a simulation tool in the UAV field, analyzing it from multiple perspectives. The project introduces a realistic simulation of the drone control system in combination with the Unity physics engine. Moreover, the references to the control system and other simulation data can be interchanged with the existing ROS tools through the ROS-TCP connector. Contributing to the multirotor RL research, we provide an examination of the native to Unity ML-Agents framework in terms of performance, convenience of the setup, and flexibility. The thesis provides a testing showcase of an RL model training aimed to teach the drone position control policy while taking into consideration visual odometry constraints. Consequently, the paper compares two state-of-the-art RL algorithms, PPO and SAC, in multiple categories. The final contribution of the project is a toolkit that can be utilized to provide a quick start for other developers in developing their Unity3D multirotor applications.

Multimedia material