Improving CMA-ES Convergence Speed, Efficiency, and Reliability in Noisy Robot Optimization Problems

Russell M. Martin, Steven H. Collins

January 2026

TL;DR AS-CMA adds per-candidate sample-time selection to CMA-ES, measuring longer only when a candidate is predicted to be hard to rank against its neighbors. On four simulated noisy landscapes it reached fine convergence in 98% of runs, 24-65% faster than the best fixed sample time.

What they do

Human-in-the-loop exoskeleton tuning measures each candidate controller for minutes, and the measurement time trades speed against noise. Usually every condition gets the same fixed time. AS-CMA leaves CMA-ES unchanged but picks a separate sample time for each candidate before it is evaluated, aiming for consistent ranking precision without ever re-measuring a candidate.

How it works

Rank-based precision. CMA-ES only uses the order of candidates, so each one needs just enough precision to be ranked against its nearest-cost neighbor. AS-CMA estimates that cost gap from the parameter-space distance to the nearest candidate, scaled by a local slope fit to the previous generation’s pairwise cost differences.
Noise target to time. It picks the noise level that gives a signal-to-noise ratio of 1.3, then inverts a measured noise-vs-time model to get the sample time (0.5-5.5 min, i.e. 34.2% to 0.4% metabolic error). Beyond that model it needs only rough cost bounds for the first generation.
Evaluation. 100 runs per strategy on four simulated landscapes with multiplicative noise: a 4D ankle-exoskeleton cost model, 4D Rosenbrock, 4D Levy and 20D Sphere. Fine convergence means staying within 5% of the true minimum.

Results

Coarse convergence in 400/400 runs, fine in 393/400. Against each landscape’s best fixed time (2-4 min), fine convergence came 24-65% sooner (not significant on Levy) at 29-76% lower cumulative cost.
Bayesian optimization was faster on the simple Ankle and Sphere landscapes (71% and 50% less time) but reached fine convergence in only 90% and 42% of Rosenbrock and Levy runs. KL-KG resampling CMA-ES was 13% faster on 20D Sphere but took 294-564% more time elsewhere.
One-participant ankle exoskeleton pilot: 42% lower metabolic cost than zero-torque walking (39% expected), with sample times spanning nearly the full range.

Limitations

It needs a known link between measurement time and noise; if noise depends on other factors it does not help. Comparisons are simulated: the human pilot had one participant and no fixed-time control, and a power analysis puts a proper comparison at 71 participants.

Figures

From Figure 1: Measuring every candidate for the same medium time (green) gives too much precision for some pairs and... — From Figure 1: Measuring every candidate for the same medium time (green) gives too much precision for some pairs and too little for others. Adaptive sampling (blue) measures briefly when the cost gap to the nearest-costing neighbor is large and longer when it is small.

From Figure 2: How AS-CMA picks a sample time. The distance to the nearest candidate in parameter space stands in for... — From Figure 2: How AS-CMA picks a sample time. The distance to the nearest candidate in parameter space stands in for the unknown cost difference, a local landscape model turns it into the target noise level that gives the desired sorting precision, and inverting the noise model gives the sample time.

From Figure 7: Fixed-time CMA-ES (colors, 0.5-5 min) versus AS-CMA (black) on four landscapes. Top: share of runs rea... — From Figure 7: Fixed-time CMA-ES (colors, 0.5-5 min) versus AS-CMA (black) on four landscapes. Top: share of runs reaching fine convergence. Middle and bottom: time and cumulative cost to convergence, normalized to AS-CMA (lower is better).

From Figure 10: Pilot ankle exoskeleton optimization with AS-CMA: normalized cost of each condition, the sample time ... — From Figure 10: Pilot ankle exoskeleton optimization with AS-CMA: normalized cost of each condition, the sample time chosen per condition, and the four assistance parameters over time. Open circles mark the CMA-ES mean of each generation.

Abstract

Experimental robot optimization often requires evaluating each candidate policy for seconds to minutes. The chosen evaluation time influences optimization because of a speed-accuracy tradeoff: shorter evaluations enable faster iteration, but are also more subject to noise. Here, we introduce a supplement to the CMA-ES optimization algorithm, named Adaptive Sampling CMA-ES (AS-CMA), which assigns sampling time to candidates based on predicted sorting difficulty, aiming to achieve consistent precision. We compared AS-CMA to CMA-ES and Bayesian optimization using a range of static sampling times in four simulated cost landscapes. AS-CMA converged on 98% of all runs without adjustment to its tunable parameter, and converged 24-65% faster and with 29-76% lower total cost than each landscape's best CMA-ES static sampling time. As compared to Bayesian optimization, AS-CMA converged more efficiently and reliably in complex landscapes, while in simpler landscapes, AS-CMA was less efficient but equally reliable. We deployed AS-CMA in an exoskeleton optimization experiment and found the optimizer's behavior was consistent with expectations. These results indicate that AS-CMA can improve optimization efficiency in the presence of noise while minimally affecting optimization setup complexity and tuning requirements.

Summaries are written automatically from each paper's text. Figures and videos come from the paper and its project page; all credit goes to the authors.

Matt Strong