SDN: Selected Diffusion Noise

Presentation video (3 min).

SDN overview: clean and perturbed inputs pass through the same VLA; the diffusion policy produces a positive set G and a negative set B of action chunks; candidates are pulled toward G and pushed away from B, then the smoothest chunk is selected. — **Overview of SDN.** The frozen VLA samples N chunks from the clean observation (𝒢) and N with the task objects masked (𝓑). *Stage 1* keeps the top-M by R = d_B − d_G; *Stage 2* executes the lowest-jerk one.

01 · Abstract

The initial noise is a free choice at test time. SDN chooses it.

Diffusion-based VLA policies generalize well, but spurious visual reliance and stochastic sampling still cause grounding failures and jerky motion at deployment. Fine-tuning a multi-billion-parameter policy for each variation is costly and risks forgetting.

Selected Diffusion Noise (SDN) is training-free: it samples several action chunks, keeps those grounded in task-relevant objects (via a counterfactual negative set), and executes the one with the lowest JerkRMS. On π₀, GR00T N1.5 and N1.6, SDN adds 5 to 9 points on SIMPLER and 18.33 points on real ALOHA tasks, with no parameter updates.

02 · Motivation

Two ways a frozen VLA fails at deployment

Spurious grounding

Ghost grasps

The policy follows irrelevant cues, e.g. “grasps” the carrot and carries nothing to the plate.

Kinematic instability

High jerk

A correct chunk can still oscillate, destabilizing execution and wearing the hardware.

Same conditioning

Different noise

Both failures have real probability under the same input; the initial noise decides.

π_θ(A|o,ℓ) = (1−ε) p*(A|o,ℓ) + ε p_fail(A|o,ℓ)

p_fail holds the wrong or unstable chunks, with context-dependent weight ε. They are not random outliers, so the question is which sample to execute.

WidowX carrot-on-plate episode. Base policy: the gripper closes next to the carrot and the arm moves to the plate holding nothing. Base with SDN: the carrot is grasped and placed on the plate. — **A ghost grasp on WidowX.** Base π₀ misses the carrot but carries on; with SDN it places it.

Attention heatmaps for 'pick up the banana and place it into the pot'. Base: peaks on irrelevant table regions. SDN: peak on the banana. Below, the attention matrix with one dominant token. — **Where the policy looks.** Base attends to the table (red); SDN's chunk attends to the banana (green), with a sparser attention matrix.

03 · Method

Sample N chunks, keep the grounded ones, execute the smoothest

Guidance methods like CFG steer every denoising step and need a tuned scale. SDN leaves denoising untouched: it samples chunks from different noises and picks one. Try it on a toy action space.

Toy action spaceN = 12 candidates per set, k-NN grounding, JerkRMS

Sample

Positive set G (clean observation) ×Negative set B (target masked) Top-M after Stage 1 Executed chunk A* Executed horizon Click a blue point to see its neighbours

k6 M3

Synthetic points stand in for action chunks; the scores are the paper's equations. The paper uses Grounding-DINO + SAM2 masks, N = 12, k ∈ {6, 10}, M ∈ {3, 5}.

Stage 1 · Grounding filter

Close to the clean samples, far from the counterfactual ones

The negative set comes from the same policy with the task objects masked or inpainted away, so it can only follow spurious cues. SDN ranks candidates by how much likelier they are under p_G than p_B, using k-NN distances:

d_S⁽ⁱ⁾ = (1/k) Σ ||A_i − A_j||, R_ground⁽ⁱ⁾ = d_B⁽ⁱ⁾ − d_G⁽ⁱ⁾

If no target is detected, the score falls back to −d_G.

Stage 2 · Kinematic refinement

Among grounded chunks, the one with the least jerk

The top-M are scored by RMS jerk over the full predicted chunk, so late oscillations still count.

S(A_i) = RMS of a_t+3 − 3a_t+2 + 3a_t+1 − a_t; A* = argmin S over the top-M

Left: a real drawer scene perturbed by inpainting (objects removed) and by zero-masking (a box over the drawer). Right: a jerky trajectory that fails and a smooth trajectory that succeeds. — **Negative set and Stage 2.** Left: inpainting (LaMa) and bbox zero-masking. Right: a jittery base trajectory vs the smooth one SDN selects.

Algorithm 1 · SDN inference

Input: policy π_θ, observation o, instruction ℓ, number of samples N, filter size M

for i = 1 … N: A_i⁰ ~ N(0, I), A_i ~ π_θ(·|o, ℓ) into G
if target object is detected then
for i = 1 … N: Ã_i ~ π_θ(·|o_neg, ℓ) into B
R_ground = d_B − d_G for every A_i ∈ G
G_M ← TopM(G) ▷ Stage 1
A* ← argmin S(A_i) ▷ Stage 2
else
A* ← argmin d_G ▷ positive grounding only
return A*

Why k-NN distances

The score ranks like the true density ratio

Since log p_S ≈ −log d_S + C, the distance difference is a monotone surrogate of the log density ratio. For k, N → ∞ with k/N → 0:

R_ground = C_D (k/N)^1/D (p_B^−1/D − p_G^−1/D)(1 + o_P(1))

so the ranking is asymptotically consistent with p_B^−1/D − p_G^−1/D.

04 · Simulation

Nine tasks, two robots, two backbones

Five Google Robot and four WidowX SIMPLER tasks (as in PCD), 50 rollouts each, official checkpoints. Training-free baselines: CFG, WNG, BYOVLA, ACG, PCD. SDN uses bbox zero-masking.

Compare SDN with

VanillaSDNsuccess rate (%), 50 rollouts per task

Full table: all methods (underlined = best in column; subscript = SDN minus vanilla)

SDN has the best average on both backbones (72.6 with π₀, 54.5 with GR00T N1.6) and never drops below vanilla on any of the 18 task–backbone pairs; every baseline does on some task.

Google Robot apple-in-drawer episode. Base: the gripper reaches the apple but the episode fails. Base with SDN: the apple is placed in the drawer. — **Google Robot, Apple Drawer.** Base reaches the apple but fails; with SDN it completes the task.

05 · Real world

Bimanual ALOHA: higher success and smoother motion

Two real ALOHA tasks. Cube2Drawer: open the top drawer, place a red block inside, close the drawer. Banana2Pot: place a banana in a pot, then close the pot with its lid using the right gripper. — **Two long-horizon ALOHA tasks** with a fine-tuned GR00T N1.5. (a) *Cube2Drawer*: open drawer, place block, close. (b) *Banana2Pot*: banana into pot, then close the lid.

Task success

Success rate (%). Each stage helps; both together are best.

Motion quality

JerkRMS and ATV (×10²), lower is better; subscripts vs vanilla.

Only SDN lowers both; every guidance baseline raises them (ACG: +114% jerk on Banana2Pot).

SDN reaches 48.33% vs 30.00% vanilla and 41.67% for the best baseline (Stage 2 alone +16.67, Stage 1 alone +13.33).

06 · Analysis

Why the negative set works, and what the filter selects

Masking the target collapses the policy

Success (%) with the task objects removed vs random regions masked. Without the target, the policy follows background shortcuts, which is what the negative set captures.

Policy	Google Robot	WidowX	Avg.
π₀	56.4	73.4	63.9
+ masked object	6.0	0.0	3.0
+ inpainting	5.0	0.0	2.5
+ random masking	56.0	72.0	64.0
GR00T N1.6	45.6	54.4	49.5
+ masked object	2.8	0.0	1.4
+ inpainting	2.0	0.0	1.0
+ random masking	46.0	54.0	50.0

The filter picks less spurious actions

Nuisance Mass Ratio (VLA-Explain, GR00T N1.6): share of relevant tokens on irrelevant regions, lower is better. Best-grounded picks lower it; larger pools lower it more.

Task	Worst N=12	Base	Best N=12	Best N=24	Best N=36
Pick Coke Can	0.740	0.668	0.663	0.657	0.656
Move Near	0.780	0.705	0.679	0.660	0.667
Carrot	0.770	0.746	0.712	0.705	0.698
Spoon	0.700	0.684	0.666	0.661	0.664
Average	0.750	0.701	0.680	0.671	0.671

VLA-Explain attribution maps on a Pick Coke Can episode for the worst-grounded candidate, the base policy and the best-grounded candidate. The best-grounded maps concentrate on the can. — **Attribution maps, Pick Coke Can.** Worst-grounded: table edge and background. Best-grounded: the can.

Robust to how the negative set is built

SDN vs PCD with both perturbations: SDN wins on average in all four settings, and with cheap bbox masking already beats PCD with inpainting. Underlined: better of the two.

Robust to its own hyperparameters

π₀, SIMPLER. Success saturates at N = 12 (63.9 → 72.6); every k gives 71.3–74.6.

Left: average success and inference latency against the number of candidates N (1, 5, 12, 24, 36). Right: average success for k = 1, 2, 4, 6, 10: 72.0, 74.0, 71.3, 74.6, 72.0.

Across horizons and action spaces

Gains at every horizon (GR00T N1.6) and for 7-DoF and 14-DoF actions.

Left: success of GR00T N1.6 with and without SDN for action horizons 1, 4, 10, 15, 20; SDN is higher at every horizon. Right: base vs SDN for 7-DoF (63.9 vs 72.6) and 14-DoF (30.0 vs 48.33).

Limitations

Object-centric perturbations may miss spurious cues from lighting, texture or other distractors.
N, k and M are fixed, not adapted per step.
Extra inference compute; latency grows with N.

07 · Citation

BibTeX

@article{nguyen2026sdn,
  title  = {Test-Time Improvement of VLA Policies via Selected Diffusion Noise
            for Spurious-Robust Action Smoothing},
  author = {Nguyen, Duc Minh and Dao, Bao-Ngoc and Luu, Tung M. and Nguyen, Binh Gia and
            Tong, Vinh and Liu, Anji and Duong, Vu N. and Le, Dung D. and Sonntag, Daniel and
            Le, Trung and Le, Ngan and Peters, Jan and Le, An Thai and Vu, Minh Nhat and
            Niepert, Mathias and Doan, Khoa D. and Nguyen, Duy M. H. and Ngo, Vien Anh},
  year   = {2026}
}