RoboSegNet: Learning Multi-Task Robot Trajectory Segmentation from Visual and Kinematic Streams

Project page for RoboSegNet (CVPR 2026 Findings).

• Project page: https://berkeleyautomation.github.io/RoboSegNet/
• Code: https://github.com/KeplerC/RoboSegNet
• Paper: Learning Multi-Task Robot Trajectory Segmentation from Visual and Kinematic Streams

RoboSegNet is a multi-task framework that jointly learns robot trajectory segmentation from visual and kinematic proprioceptive signals. Kinematic trajectories are encoded with a DCT-based tokenizer, images with a visual transformer; the two modalities are fused with bidirectional cross-modal attention, and transition boundaries are predicted via Hungarian matching. The paper also introduces RoboSegData, a benchmark built from the Agibot dataset with dense frame-level transition annotations.

Citation

@InProceedings{Chen_2026_CVPR,
    author    = {Chen, Kaiyuan and Xie, Shuangyu and Goldberg, Andrew and Goldberg, Ken},
    title     = {Learning Multi-Task Robot Trajectory Segmentation from Visual and Kinematic Streams},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
    month     = {June},
    year      = {2026},
    pages     = {1452-1461}
}

---

This site is built on the Academic Project Page Template.