KineState-SWAM
A Surgical World Action Model with Explicit Kinematics and Causal Interaction Dynamics
Abstract
KineState-SWAM links surgical robot action prediction and video generation through a persistent interaction state. Calibrated URDF kinematics convert predicted joint trajectories into tool motion and instrument renderings. The model uses these signals to update its interaction state and condition future video frames. The paper also introduces dVRK-Chole5.2k, a dataset of 5,255 videos spanning 17 cholecystectomy phases.
Method
Video demo
Clip-placement rollout using an edited PSM1 trajectory. The views show generated RGB, URDF geometry with the tool-tip path, and their overlay.