Autonomous Particle Image Velocimetry Using Zero-Shot Sim-to-Real Reinforcement Learning

Chang Rajani, Luke Sebastien Marsden, Stephan Kunz, Claudio Mucignat, Ivan Lunati

November 2026

TL;DR A DreamerV3 policy trained only in an Isaac Sim replica of a water tunnel moves a camera on a 6-DoF arm to find, frame and resolve hydrofoil wakes for particle image velocimetry. Deployed zero-shot, it frames the whole wake in 41 of 42 real views and roughly halves the velocity-deficit error of a fixed wide view.

What they do

In particle image velocimetry (PIV), the camera position is usually fixed before the measurement, so a flow feature can fall outside the frame or be poorly resolved. This paper treats camera placement as an RL problem. A camera on a 6-DoF arm must locate a hydrofoil wake in a water tunnel, frame all of it, and get as close as it can. The policy is trained only in simulation and deployed zero-shot on the real tunnel.

How it works

Observation and action. Each step, the camera records four image pairs. LIMA, a CNN-based PIV estimator, turns them into velocity, velocity-deficit and uncertainty fields. The policy outputs a Cartesian camera position 0.25-0.60 m from the light sheet, with the optical axis kept normal to the sheet. Without the uncertainty input, policies often framed the hydrofoil’s shadow or unlit corners.
Reward. Computed from CFD ground truth in simulation only: the product of the fraction of the wake in view and the fraction of the frame it fills, normalized so that the closest pose capturing the whole wake scores one.
Simulation. An Isaac Sim replica of the lab with path-traced lighting and shadows, and tracers advected through 719 lattice-Boltzmann CFD cases. Seeding density, tracer size and sensor noise are randomized. The agent is unmodified DreamerV3.
Deployment. A calibration table built from an ArUco board in the water maps each commanded view to joint angles.

Results

Simulation: the median error between the settled and required view is below 10 mm in x and y and about 20 mm in distance. Per-step reward is more than twice that of the best fixed camera. From the required pose, LIMA recovers 72% of the true wake region, against 54% from 520 mm.
Real tunnel, two hydrofoil profiles unseen in training, six settings each: the policy frames the whole wake in 41 of 42 views (fixed wide view: 7 of 11), at 0.090 mm per pixel versus 0.136 mm.
Compared with a 15-tile scan mosaic, the velocity-deficit error falls from 0.419 to 0.205. An episode takes a median of 85 s and 160 image pairs, against 148 s and 1500 pairs for the scan.

Limitations

The policy does not set exposure, inter-frame time or aperture. Moving to another tunnel needs accurate CAD models, lighting that simulation reproduces well, and extensive camera calibration. Hardware tests used one small tunnel and two hydrofoil profiles.

Videos

Figures

From Figure 1: (a) The water tunnel, seeded with tracers and lit by an LED light sheet, with a hydrofoil shedding a w... — From Figure 1: (a) The water tunnel, seeded with tracers and lit by an LED light sheet, with a hydrofoil shedding a wake and a camera on the wrist of an igus ReBeL 6-DoF arm. (b) Each step, LIMA estimates velocity, velocity deficit and uncertainty from four image pairs; the policy maps these to a new camera position, converted to joint angles.

From Figure 3: Simulated (top) and real (bottom) observations of a similar hydrofoil: a preprocessed camera image, fl... — From Figure 3: Simulated (top) and real (bottom) observations of a similar hydrofoil: a preprocessed camera image, flow speed relative to the free stream, and LIMA uncertainty. The wake is a slow band behind the trailing edge; the shadow below the foil gives spurious flow.

From Figure 5: One simulated episode. The policy finds the wake within three steps; after the hydrofoil is exchanged ... — From Figure 5: One simulated episode. The policy finds the wake within three steps; after the hydrofoil is exchanged at step 16 it finds the new, larger wake within two steps and moves back to fit it. The plot shows the fraction of the wake in view per step.

From Figure 7: Real deployments. (a) Scan-mosaic velocity deficit for each hydrofoil setting, with the fixed wide vie... — From Figure 7: Real deployments. (a) Scan-mosaic velocity deficit for each hydrofoil setting, with the fixed wide view (red) and the settled policy views (blue); the policy frames the whole wake in 41 of 42 views. (b) Mean velocity and fluctuation profiles 10, 25 and 50 mm behind the trailing edge: the policy views follow the scan mosaic more closely than the fixed view.

Abstract

Particle image velocimetry (PIV) reconstructs a flow field from the motion of tracer particles but the reconstruction quality is bounded by how well the flow structure of interest is resolved in the images. The acquisition position is typically fixed by the operator before the measurement. We cast camera placement as a sequential decision-making problem and train a reinforcement learning agent to control a 6-DoF manipulator equipped with a camera, enabling it to locate, frame and resolve a hydrofoil wake in a water tunnel. The agent commands the camera pose and observes the scene only through the velocity field and uncertainty estimated from time-separated image pairs. It is trained entirely in a photorealistic simulation of our water tunnel, with wakes generated by Computational Fluid Dynamics (CFD) simulations of hydrofoils with varying profiles, positions and angles of attack. After zero-shot transfer to the real water tunnel, the policy reliably locates and frames the wakes of randomly placed hydrofoils. It resolves the wake at a higher resolution than a fixed camera view and with lower error in the measured flow statistics, in roughly half the time of a full scan and with far fewer image pairs.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong