MIT · embodied AI & robot learning麻省理工学院 · 具身智能与机器人学习
17 curated papers, code, datasets and hardware from MIT (10 with code), with one-line takeaways.17 个来自麻省理工学院的精选论文、代码、数据集与硬件(10 个附代码),均配一句话要点。
Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。
Generating action sequences by denoising handles multimodal demos far better than regression; now a standard IL baseline.以去噪方式生成动作序列,比回归更能处理多模态演示,已成为模仿学习标准基线。
@misc{chi2023diffusion,
title = {{Diffusion Policy: Visuomotor Policy Learning via Action Diffusion}},
author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song},
year = {2023},
eprint = {2303.04137},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2303.04137},
}
@misc{cheng2024open,
title = {{Open-TeleVision: Teleoperation with Immersive Active Visual Feedback}},
author = {Xuxin Cheng and Jialong Li and Shiqi Yang and Ge Yang and Xiaolong Wang},
year = {2024},
eprint = {2407.01512},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2407.01512},
}
Trains one quadruped policy that exposes gait parameters (frequency, foot swing, posture) as commands, so a human can tune behavior at deploy time for new tasks; open sim-to-real stack for the Unitree Go1.训练一个把步态参数(频率、抬脚高度、姿态等)作为指令开放的四足策略,部署时可由人调节行为以适应新任务;附带面向 Unitree Go1 的开源仿真到真实工具链。
@misc{margolis2022walk,
title = {{Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior}},
author = {Gabriel B Margolis and Pulkit Agrawal},
year = {2022},
eprint = {2212.03238},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2212.03238},
}
@misc{shan2020lio,
title = {{LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping}},
author = {Tixiao Shan and Brendan Englot and Drew Meyers and Wei Wang and Carlo Ratti and Daniela Rus},
year = {2020},
eprint = {2007.00258},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2007.00258},
}
Open C++ library for real-time metric-semantic visual-inertial SLAM: VIO, robust pose graph, fast 3D mesher and dense semantic reconstruction as modular parts.开源 C++ 实时度量-语义视觉惯性 SLAM 库,模块化包含 VIO、鲁棒位姿图、快速三维网格与稠密语义重建。
@misc{rosinol2019kimera,
title = {{Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping}},
author = {Antoni Rosinol and Marcus Abate and Yun Chang and Luca Carlone},
year = {2019},
eprint = {1910.02490},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/1910.02490},
}
Convex MPC for quadrupeds — simplified rigid-body dynamics turn ground-reaction-force planning into a small QP solved in real time; the classic model-based baseline behind many open-source quadruped controllers.四足机器人的凸 MPC:用简化刚体动力学把足端地面反作用力规划变成可实时求解的小规模 QP;众多开源四足控制器背后的经典基于模型的基线。
Model-based toolbox for robotics - multibody dynamics with contact simulation, mathematical programming and trajectory optimization; used in the MIT manipulation and underactuated courses.面向机器人的基于模型的工具箱:带接触的多体动力学仿真、数学规划与轨迹优化;MIT 操作与欠驱动课程即使用它。
Asks whether joint readings alone suffice for millimetre-precise insertion — trains sim-to-real policies with no camera, force or tactile input.探究仅靠关节读数能否完成毫米级精密插装:训练不用相机、力或触觉输入的 sim-to-real 策略。
Diffusion policies trained on sliding-window demos often memorise one action per state; querying the demonstrator k times at one state (vine sampling) keeps the multimodality and aids best-of-N and expert iteration.用滑窗示范训练的扩散策略往往每个状态只记住一个动作。在同一状态下向示范者查询 k 次(vine 采样)可保留其多模态分布,并提升 best-of-N 与专家迭代效果。
@misc{chia2026explicitly,
title = {{Explicitly capturing demonstrator multi-modality for distillation into diffusion policies}},
author = {Shao Yuan Chew Chia and Rebecca H. Jiang and Abhinav Agarwal and Andreas Hofmann and Leslie Pack Kaelbling and Russ Tedrake},
year = {2026},
eprint = {2611.10949},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2611.10949},
}
A language-model agent resolves instructions against a posed RGB-D scan at test time instead of a prebuilt representation; 35 of 50 real Spot trials succeed, and most failures come from physical skills.语言模型智能体在测试时对带位姿的 RGB-D 扫描做指令落地,不依赖预建表示;Spot 真机 50 次试验成功 35 次,失败大多来自物理技能而非推理。
@misc{agarwal2026feijoa,
title = {{FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation}},
author = {Aditya Agarwal and William Shen and Sahit Chintalapudi and Lucy Cai and Nishanth Kumar and Aidan Curtis and Travis Manderson and Tomás Lozano-Pérez and Leslie Pack Kaelbling},
year = {2026},
eprint = {2611.04080},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2611.04080},
}
FLEX lets language corrections edit a robot's task features via add, generalize and mask (not add-only). Reports higher normalized reward than other editing methods in 2D nav, Franka tabletop and a real user study.FLEX 用自然语言纠正来迭代修改机器人的任务特征集合,支持添加、泛化、屏蔽三种操作(而非只能添加);在二维导航、Franka 桌面操作和真实用户研究中报告比其他特征编辑方法获得更高的归一化奖励。
@misc{lee2026flexible,
title = {{Flexible Language-Guided Representation Editing for Adaptive Robot Behavior}},
author = {Audrey Lee and Nathaniel Dennler and Ilia Sucholutsky and Andreea Bobu},
year = {2026},
eprint = {2610.28893},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28893},
}
A 3D-aware LLM with scene, object and action tokens, aligned with diffusion models so it can imagine goal images and point clouds before planning actions; trained on ~2M 3D-language-action pairs the authors assembled.具备三维感知的大语言模型,引入场景、物体与动作交互 token,并与扩散模型对齐,可在规划动作前“想象”目标图像和点云;训练数据为作者整理的约 200 万条三维-语言-动作样本。
@misc{zhen2024d,
title = {{3D-VLA: A 3D Vision-Language-Action Generative World Model}},
author = {Haoyu Zhen and Xiaowen Qiu and Peihao Chen and Jincheng Yang and Xin Yan and Yilun Du and Yining Hong and Chuang Gan},
year = {2024},
eprint = {2403.09631},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2403.09631},
}
HPT: per-robot stems turn proprioception and vision into a few tokens for one shared transformer trunk, pretrained on 52 datasets (robot, sim, human video); reports 20%+ gains on unseen tasks.HPT:为每种机器人设计“输入头”,把本体感知与视觉映射成短 token 序列,送入共享的 Transformer 主干;在 52 个数据集(真机、仿真、人类视频)上预训练,报告在未见任务上微调提升 20% 以上。
Builds an object-level 3D scene graph labelled by vision-language models, which an LLM planner can query in natural language.用视觉语言模型构建带语义标注的物体级三维场景图,大模型规划器可以用自然语言查询。
@misc{gu2023conceptgraphs,
title = {{ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning}},
author = {Qiao Gu and Alihusein Kuwajerwala and Sacha Morin and Krishna Murthy Jatavallabhula and Bipasha Sen and Aditya Agarwal and Corban Rivera and William Paul and Kirsty Ellis and Rama Chellappa and Chuang Gan and Celso Miguel de Melo and Joshua B. Tenenbaum and Antonio Torralba and Florian Shkurti and Liam Paull},
year = {2023},
eprint = {2309.16650},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2309.16650},
}
UniPi — casts decision making as text-conditioned video generation, then recovers actions from the generated frames with an inverse dynamics model.UniPi:把决策问题转化为文本条件的视频生成,再用逆动力学模型从生成的画面中恢复动作。
@misc{du2023learning,
title = {{Learning Universal Policies via Text-Guided Video Generation}},
author = {Yilun Du and Mengjiao Yang and Bo Dai and Hanjun Dai and Ofir Nachum and Joshua B. Tenenbaum and Dale Schuurmans and Pieter Abbeel},
year = {2023},
eprint = {2302.00111},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2302.00111},
}
@misc{chen2022visual,
title = {{Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes}},
author = {Tao Chen and Megha Tippur and Siyang Wu and Vikash Kumar and Edward Adelson and Pulkit Agrawal},
year = {2022},
eprint = {2211.11744},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2211.11744},
}
A slim GelSight-style finger that measures contact shape, force and slip at high resolution, designed to fit robot grippers.纤薄的 GelSight 类触觉手指,可高分辨率测量接触形状、力与滑移,便于装在机器人夹爪上。
@misc{taylor2021gelslim,
title = {{GelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger}},
author = {Ian Taylor and Siyuan Dong and Alberto Rodriguez},
year = {2021},
eprint = {2103.12269},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2103.12269},
}