← Paper Digest

UAV-Flow++: A Unified Suite for Benchmarking and Learning Language-Guided UAV Flight

Donglin Yang, Xiangyu Wang, Yantong Zhong, Wenhao Zheng, Zihan Ding, Yue Liao, Hongsheng Li, Ke Xu, Si Liu

November 2026

Real flight, 2x speed: "Go through the trees and head to the left."

TL;DR A real-world dataset of 30,692 expert-piloted drone flights paired with short commands such as "Pass through the tree from the left side", a simulator that scores closed-loop flights, and FlowVLA, a Qwen3-VL policy trained on both. Imitation alone reaches 75.6% success on 273 simulated test episodes (best baseline 68.9%), and RL post-training with a tail-aware GRPO raises it to 93.2%.

What they do

Language-guided drone research has mostly targeted long-horizon route planning in simulation. UAV-Flow++ targets short maneuvers relative to what the drone sees, such as “Approach the right side of the sign” or “Circle clockwise around the maple tree at a radius of 2 meters”. It extends the authors’ UAVFlow conference paper with a real-world demonstration dataset, a simulation suite with automatic task scoring, and FlowVLA, a policy trained by imitation and then RL.

How it works

  • Real data. Certified pilots, each with over 800 flight hours, flew DJI Mavic 3T RTK drones over three university campuses (5.02 km²), steering only from the onboard view. Video and flight logs are synchronized at 5 Hz into image, 6-DoF state and action, and labeled with short commands: 30,692 trajectories in eight motion categories, most under 20 m.
  • Simulation. UAV-Flow-Sim (UnrealCV, one campus scene) adds 10,109 trajectories and scores each closed-loop episode on both the maneuver and the final state.
  • FlowVLA. Qwen3-VL-2B with LoRA takes the first and current images, the state as text and the instruction, and predicts ten position-yaw steps (2 s) as 256-bin discrete actions.
  • TGRPO. RL in simulation on four tasks, with GRPO advantages reweighted toward rewards that few rollouts in a group reach. On hardware, a ground station runs inference and skips waypoints to make up for latency.

Results

  • On 273 held-out simulated episodes, imitation-only FlowVLA succeeds 75.58% of the time versus 68.86% for π0-UAV, the best baseline. GRPO raises it to 92.06% and TGRPO to 93.16%; TGRPO passes 90% after about 40 updates, GRPO after about 80.
  • Adding the real flights to simulated training data raises simulated success from 73.14% to 75.58%. A FAST action head drops it to 64.10%.
  • Real flights show passing, approaching and orbiting outdoors (π0-UAV) and indoors (FlowVLA).

Limitations

All numbers come from simulation, in the same campus scene as the simulated training data; real flights are shown only as examples. Inference runs on a ground station over a wireless link, which adds latency. TGRPO slightly lowers success on tasks left out of RL.

Videos

Real flight, 2x speed: "Orbit the person at a distance of 3 meters."
Real flight, 2x speed: "Land on the right side of the lamppost."

Figures

From Figure 1: Overview. Real-world flight demonstrations that pair instructions with onboard views, UAV states and e...
From Figure 1: Overview. Real-world flight demonstrations that pair instructions with onboard views, UAV states and expert actions; UAV-Flow-Sim for closed-loop training and evaluation; and FlowVLA, which predicts position-yaw action chunks from the instruction, the initial and current images and the UAV state, trained by imitation on mixed real and simulated demonstrations and then by RL with trajectory-level rewards.
From Figure 2: Example tasks (hovering, approaching, retreating, passing, orbiting and landing), each with the scene,...
From Figure 2: Example tasks (hovering, approaching, retreating, passing, orbiting and landing), each with the scene, the instruction and several expert 3D flight trajectories for that same instruction.
From Figure 4: Two-stage training. Stage one imitates mixed real-world and simulated demonstrations; stage two runs R...
From Figure 4: Two-stage training. Stage one imitates mixed real-world and simulated demonstrations; stage two runs RL with TGRPO on simulated closed-loop rollouts, using tail-aware advantages computed from task completion scores.
From Figure 6: RL post-training algorithms compared on UAV-Flow-Sim. (a) Mean training reward, and test success rate ...
From Figure 6: RL post-training algorithms compared on UAV-Flow-Sim. (a) Mean training reward, and test success rate (solid) and nDTW (dashed), over policy updates. (b) Success-rate gains over the pretrained policy on the four RL training tasks (top) and on the remaining tasks left out of post-training (bottom).
Abstract

Language-guided UAV navigation has largely focused on long-horizon route planning, leaving short-range, object-relative flight maneuvers underexplored. To address this gap, we formulate Flying-on-a-Word (Flow), a task requiring a UAV to convert concise natural-language instruction, egocentric visual observation, and current state dynamics into continuous flight actions. Flow encompasses both primitive motion commands and complex object-interactive behaviors, assessing motion intent comprehension and spatial context grounding. To enable policy learning and closed-loop evaluation, we introduce UAV-Flow, a comprehensive dataset comprising 30,692 expert-piloted real-world trajectories across eight distinct motion categories, alongside 10,109 simulation trajectories in the UAV-Flow-Sim environment equipped with task-specific completion criteria. Furthermore, we develop FlowVLA, which undergoes imitation pretraining on mixed real-sim demonstrations and online post-training via Tail-Aware GRPO (TGRPO). By leveraging group reward distributions, TGRPO effectively reinforces high-reward action sequences. Evaluated on 273 held-out UAV-Flow-Sim episodes, pretrained FlowVLA achieves a mean success rate of 75.58%, outperforming the strongest baseline by 6.72 percentage points, which TGRPO post-training further elevates to 93.16% with superior sample efficiency. Physical UAV experiments demonstrate robust execution of object-relative commands in both indoor and outdoor real-world scenes. Together, Flow establishes a novel benchmark and learning framework for language-conditioned flight maneuvers.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.