Action-conditioned short-horizon predictions
Five observation streams, recorded actions. Eight observed frames, then eight predicted frames (0.53 s at 15 FPS). Final epoch-019 checkpoints for Motherboard, Rope and PushT, plus a matched-epoch ablation of the force term in the action encoding.
Main results · epoch 019
How to read these results
Each video shows GT above prediction. Columns are main view, left wrist, right wrist, left tactile, and right tactile. The first eight frames are observed history; the last eight are predictions. GT visuals are decoded cached latents.
Contact means max(left force, right force) > 0.5 N. None: no contact in the 16-frame window. Full: contact in at least 80% of it. Onset/release: a contact transition inside the eight predicted frames.
Scores cover the eight predicted frames only; the table averages the currently displayed cases equally. PSNR and SSIM are higher-is-better; LPIPS is lower-is-better. Cases are selected for contact coverage, not as a representative full-test benchmark, and the three datasets use different checkpoints, so their scores are not a controlled comparison.
Generation uses 35 UniPC steps per causal latent block, seed 0. Motherboard 0909 uses an older force calibration than 0911/0912.
Download selections and metrics (JSON)Ablation · force term in the action encoding
The with-force model conditions on the force-informed 9D sensor action (pose delta plus a normal-force delta term, sensor_actions_with_dforce_fps15.pt). The no-force arm is trained identically but on the pure-pose action (sensor_actions_fps15_noforce.pt): same data, split, recipe, seed, and batch order; only the action file differs. Both arms are compared at the same epoch and step, on identical test windows, seeds, and sampling settings, so every case is paired.
Δ = no force − with force, averaged over the displayed cases. Green means the no-force arm is better on that metric, red means worse. Wins count the cases where no force is better. Bold marks a paired t-test p<0.05 over all cases of this dataset (the p-values are computed on the full case set, not the current filter).