TOMSNAV: Topological Multi-modal and Semantic Navigation for Aerial Vehicles

Abstract:

The fundamental research project aims to explore how future mobile aerial robots could utilize their senses to detect and model their surrounding environment. We hypothesize that a significant methodological leap is necessary beyond the incremental improvements of current SLAM (Simultaneous Localization and Mapping) systems. The accuracy-focused SLAMs currently in use are limited by their real-time performance due to their complexity. We propose a biologically-inspired approach for multi-modal semantic feature extraction, which is gradually becoming feasible with advancements in visual object detection and classification. Our objective is to generalize robot navigation in real-world environments without relying on precise motion odometry. To achieve this, we intend to develop novel methods for learning and extracting multi-modal landmarks from sensor data. Additionally, we seek to address the fundamental challenge of sharing plans and paths between robots and humans in environments where there is no prior knowledge available about the environment and no notion of global coordinates.
 

Mapping and planning pipeline

We have a working prototype of a new mapping and planning system built on top of VDB tree structure with a custom ESDF field propagation. We have implemented these technologies in a way that performs significantly faster than the old Octomap-based mapper. With this new mapper, we can run the system in real time and can map significantly larger distances (kilometers scale). We also plan to test this during our next week's experimental campaign.


 

GNSS-denied active global localization

We have recently completed a working prototype of a large-scale active localization framework. Below you can see a visualization of precomputed “localization directions” at each place in a known map of large-scale geometry (trees, buildings). These directions point towards nearby areas with high information value. To relocalize the UAV (e.g., when it is started on an empty field with zero prior location information), we combine the localization directions with the current position probabilities to compute a single direction towards which to fly. We have not done full evaluations yet, but this has shown to guide the robot to the information-rich areas  (e.g., forest-field boundary) and relocalize quickly.
Localization directions visualizations:

Approximate information content of areas visualization:

Visualization of the active localization (blue=true position) after “waking up” on a field:

In Media: