Publications

    For the up-to-date publication list, please visit the Google Scholar page.

    2021

    DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
    Shiyi Lan, Zhiding Yu, Christopher Choy, Subhashree Radhakrishnan, Guilin Liu, Yuke Zhu, Larry S. Davis, Anima Anandkumar
    International Conference on Computer Vision (ICCV), October 2021

    What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
    Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, Roberto Martín-Martín
    Technical report arXiv:2108.03298, August 2021

    Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations
    Zhenyu Jiang, Yifeng Zhu, Maxwell Svetlik, Kuan Fang, Yuke Zhu
    Robotics: Science and Systems (RSS), July 2021

    Learning Generalizable Skills via Automated Generation of Diverse Tasks
    Kuan Fang, Yuke Zhu, Silvio Savarese, Li Fei-Fei
    Robotics: Science and Systems (RSS), July 2021

    Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning
    Anuj Mahajan, Mikayel Samvelyan, Lei Mao, Viktor Makoviychuk, Animesh Garg, Jean Kossaifi, Shimon Whiteson, Yuke Zhu, Animashree Anandkumar
    International Conference on Machine Learning (ICML), July 2021

    Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition
    Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Animashree Anandkumar
    International Conference on Machine Learning (ICML), July 2021
    Long Talk

    SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies
    Linxi Fan, Guanzhi Wang, De-An Huang, Zhiding Yu, Li Fei-Fei, Yuke Zhu, Animashree Anandkumar
    International Conference on Machine Learning (ICML), July 2021

    MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
    Paul Pu Liang, Yiwei Lyu, Xiang Fan, Zetian Wu, Yun Cheng, Jason Wu, Leslie Chen, Peter Wu, Michelle A. Lee, Yuke Zhu, Ruslan Salakhutdinov, Louis-Philippe Morency
    NeurIPS 2021 Datasets and Benchmarks Track, July 2021

    Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs
    Yifeng Zhu, Jonathan Tremblay, Stan Birchfield, Yuke Zhu
    IEEE International Conference on Robotics and Automation (ICRA), May 2021

    Fast Uncertainty Quantification for Deep Object Pose Estimation
    Guanya Shi, Yifeng Zhu, Jonathan Tremblay, Stan Birchfield, Fabio Ramos, Animashree Anandkumar, Yuke Zhu
    IEEE International Conference on Robotics and Automation (ICRA), May 2021

    Deep Affordance Foresight: Planning Through What Can Be Done in the Future
    Danfei Xu, Ajay Mandlekar, Roberto Martín-Martín, Yuke Zhu, Silvio Savarese, Li Fei-Fei
    IEEE International Conference on Robotics and Automation (ICRA), May 2021

    Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors
    Michelle A. Lee, Matthew Tan, Yuke Zhu, Jeannette Bohg
    IEEE International Conference on Robotics and Automation (ICRA), May 2021

    Learning Multi-Arm Manipulation Through Collaborative Teleoperation
    Albert Tung, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, Silvio Savarese
    IEEE International Conference on Robotics and Automation (ICRA), May 2021
    Best Multi-Robotic Systems Paper Finalist

    Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects
    Xinlei Pan, Animesh Garg, Animashree Anandkumar, Yuke Zhu
    IEEE International Conference on Robotics and Automation (ICRA), May 2021

    Adaptive Procedural Task Generation for Hard-Exploration Problems
    Kuan Fang, Yuke Zhu, Silvio Savarese, Li Fei-Fei
    International Conference on Learning Representations (ICLR), May 2021

    2020

    Human-in-the-Loop Imitation Learning using Remote Teleoperation
    Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, Silvio Savarese
    Technical report arXiv:2012.06733, December 2020

    Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning
    Weili Nie, Zhiding Yu, Lei Mao, Ankit B. Patel, Yuke Zhu, Animashree Anandkumar
    Conference on Neural Information Processing Systems (NeurIPS), December 2020
    Spotlight Presentation

    Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion
    Xingye Da, Zhaoming Xie, David Hoeller, Byron Boots, Animashree Anandkumar, Yuke Zhu, Buck Babich, Animesh Garg
    Conference on Robot Learning (CoRL), November 2020

    robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
    Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín
    Technical report arXiv:2009.12293, September 2020

    RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition
    Linxi Fan*, Shyamal Buch*, Guanhzi Wang, Ryan Cao, Yuke Zhu, Juan Carlos Niebles, Li Fei-Fei
    European Conference on Computer Vision (ECCV), August 2020
    * indicates equal contribution

    OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation
    Hongyu Ren, Yuke Zhu, Jure Leskovec, Animashree Anandkumar, Animesh Garg
    Conference on Uncertainty in Artificial Intelligence (UAI), August 2020

    DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs
    Yunbo Wang*, Bo Liu*, Jiajun Wu, Yuke Zhu, Simon S. Du, Li Fei-Fei, Joshua B. Tenenbaum
    International Joint Conference on Artificial Intelligence (IJCAI), July 2020
    * indicates equal contribution

    6-PACK: Category-Level 6D Pose Tracker with Anchor-Based Keypoints
    Chen Wang, Roberto Martín-Martín, Danfei Xu, Jun Lv, Cewu Lu, Li Fei-Fei, Silvio Savarese, Yuke Zhu
    IEEE International Conference on Robotics and Automation (ICRA), May 2020

    KETO: Learning Keypoint Representations for Tool Manipulation
    Zengyi Qin, Kuan Fang, Yuke Zhu, Li Fei-Fei, Silvio Savarese
    IEEE International Conference on Robotics and Automation (ICRA), May 2020

    Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks
    Michelle A. Lee, Yuke Zhu, Peter Zachares, Matthew Tan, Krishnan Srinivasan, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg
    IEEE Transactions on Robotics (T-RO), March 2020

    2019

    Causal Induction from Visual Observations for Goal Directed Tasks
    Suraj Nair, Yuke Zhu, Silvio Savarese, Li Fei-Fei
    NeurIPS 2019 Workshop on Causal Machine Learning, December 2019

    Regression Planning Networks
    Danfei Xu, Roberto Martín-Martín, De-An Huang, Yuke Zhu, Silvio Savarese, Li Fei-Fei
    Conference on Neural Information Processing Systems (NeurIPS), December 2019

    Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity
    Ajay Mandlekar, Jonathan Booher, Max Spero, Albert Tung, Anchit Gupta, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei
    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019
    Best Cognitive Robotics Paper Finalist

    Continuous Relaxation of Symbolic Planner for One-Shot Imitation Learning
    De-An Huang, Danfei Xu, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei, Juan Carlos Niebles
    IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019

    Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation
    Kuan Fang, Yuke Zhu, Animesh Garg, Silvio Savarese, Li Fei-Fei
    Conference on Robot Learning (CoRL), October 2019
    Oral Presentation

    Situational Fusion of Visual Representation for Visual Navigation
    William B. Shen, Danfei Xu, Yuke Zhu, Leo Guibas, Li Fei-Fei, Silvio Savarese
    International Conference on Computer Vision (ICCV), October 2019

    SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning
    Linxi Fan*, Yuke Zhu*, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei
    Technical report arXiv:1909.12989, September 2019
    * indicates equal contribution

    Closing the Perception-Action Loop: Towards Building General-Purpose Robot Autonomy
    Yuke Zhu
    Stanford University Ph.D. Dissertation, August 2019

    Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision
    Kuan Fang, Yuke Zhu, Animesh Garg, Andrey Kurenkov, Viraj Mehta, Li Fei-Fei, Silvio Savarese
    International Journal of Robotics Research (IJRR), August 2019

    DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
    Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, Silvio Savarese
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration
    De-An Huang*, Suraj Nair*, Danfei Xu*, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese, Juan Carlos Niebles
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
    Oral Presentation
    * indicates equal contribution

    Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks
    Michelle A. Lee*, Yuke Zhu*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg
    IEEE International Conference on Robotics and Automation (ICRA), May 2019
    Best Conference Paper Award
    * indicates equal contribution

    2018 and Before

    RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
    Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei
    Conference on Robot Learning (CoRL), October 2018

    SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark
    Linxi Fan*, Yuke Zhu*, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, Li Fei-Fei
    Conference on Robot Learning (CoRL), October 2018
    * indicates equal contribution

    Reinforcement and Imitation Learning for Diverse Visuomotor Skills
    Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool, János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess
    Robotics: Science and Systems (RSS), June 2018

    Learning Task-Oriented Grasping for Tool Manipulation with Simulated Self-Supervision
    Kuan Fang, Yuke Zhu, Animesh Garg, Viraj Mehta, Andrey Kurenkov, Li Fei-Fei, Silvio Savarese
    Robotics: Science and Systems (RSS), June 2018

    Neural Task Programming: Learning to Generalize Across Hierarchical Tasks
    Danfei Xu*, Suraj Nair*, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese
    IEEE International Conference on Robotics and Automation (ICRA), May 2018
    * indicates equal contribution

    AdaPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems
    James Harrison*, Animesh Garg*, Boris Ivanovic, Yuke Zhu, Silvio Savarese, Li Fei-Fei, Marco Pavone
    International Symposium on Robotics Research (ISRR), December 2017
    * indicates equal contribution

    Visual Semantic Planning using Deep Successor Representations
    Yuke Zhu*, Daniel Gordon*, Eric Kolve, Dieter Fox, Li Fei-Fei, Abhinav Gupta, Roozbeh Mottaghi, Ali Farhadi
    International Conference on Computer Vision, October 2017
    * indicates equal contribution

    Vicus: Exploiting Local Structures to Improve Network-based Analysis of Biological Data
    Bo Wang, Lin Huang, Yuke Zhu, Anshul Kundaje, Serafim Batzoglou, Anna Goldenberg
    PLoS Computational Biology, October 2017

    Adversarially Robust Policy Learning through Active Construction of Physically-Plausible Perturbations
    Ajay Mandlekar*, Yuke Zhu*, Animesh Garg*, Li Fei-Fei, Silvio Savarese
    International Conference on Intelligent Robots and Systems (IROS), September 2017
    * indicates equal contribution

    Knowledge Acquisition for Visual Question Answering via Iterative Querying
    Yuke Zhu, Joseph J. Lim, Li Fei-Fei
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    Scene Graph Generation by Iterative Message Passing
    Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
    Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J. Lim, Abhinav Gupta, Li Fei-Fei, Ali Farhadi
    International Conference on Robotics and Automation (ICRA), May 2017

    Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalanditis, Li-Jia Li, David A. Shamma, Michael Bernstein, Li Fei-Fei
    International Journal of Computer Vision (IJCV), February 2017

    Visual7W: Grounded Question Answering in Images
    Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Action Recognition by Hierarchical Mid-level Action Elements
    Tian Lan*, Yuke Zhu*, Amir Roshan Zamir, Silvio Savarese
    International Conference on Computer Vision (ICCV), December 2015
    * indicates equal contribution

    Building a Large-scale Multimodal Knowledge Base System for Answering Visual Queries
    Yuke Zhu, Ce Zhang, Christopher Ré, Li Fei-Fei
    Technical report arXiv:1507.05670, July 2015

    Reasoning About Object Affordances in a Knowledge Base Representation
    Yuke Zhu, Alireza Fathi, Li Fei-Fei
    European Conference on Computer Vision (ECCV), September 2014

    StrokeBank: Automating Personalized Chinese Handwriting Generation
    Alfred Zong, Yuke Zhu
    Conference on Innovative Applications of Artificial Intelligence (IAAI), July 2014

    Graphical Model-based Learning in High Dimensional Feature Spaces
    Zhao Song, Yuke Zhu
    AAAI Conference on Artificial Intelligence (AAAI), July 2013

    Latent Spatio-temporal Models for Action Localization and Recognition in Nursing Home Surveillance Video
    Yuke Zhu, Tian Lan, Yijian Yang, Steven N. Robinovitch, Greg Mori
    IAPR International Conference on Machine Vision Applications (MVA), May 2013

    Modelling Relational Statistics With Bayes Nets
    Oliver Schulte, Hassan Khosravi, Arthur Kirkpatrick, Tianxiang Gao, Yuke Zhu
    Machine Learning Journal, May 2013