Is Data All You Need?: Large Robot Action Models and Good Old Fashioned Engineering

Ken Goldberg, UC Berkeley

Enthusiasm has been skyrocketing for humanoids based on recent advances in "end-to-end" large robot action models. Initial results are promising, and several collaborative efforts are underway to collect the needed demonstration data. But is data really all you need?

Although end-to-end Large Vision, Language, Action (VLA) Models have potential to generalize and reliably solve all problems in robotics, initial results have been mixed. It seems likely that the size of the VLA state space and dearth of available demonstration data, combined with challenges in getting models to generalize beyond the training distribution and the inherent challenges in interpreting and debugging large models, will make it difficult for pure end-to-end systems to provide the kind of robot performance that investors expect in the near future.

In this presentation, I share my concerns about current trends in robotics, including task definition, data collection, and experimental evaluation. I propose that to reach expected performance levels, we will need "Good Old Fashioned Engineering (GOFE)" – modularity, algorithms, and metrics. I'll present MANIP, a modular systems architecture that can integrate learning with well-established procedural algorithmic primitives such as Inverse Kinematics, Kalman Filters, RANSAC outlier rejection, PID modules, etc. I’ll show how we are using MANIP to improve performance on robot manipulation tasks such as grasping, cable untangling, surgical suturing, motion planning, and bagging, and propose open directions for research.

About the Speaker

Ken Goldberg is William S. Floyd Distinguished Chair of Engineering at UC Berkeley and Chief Scientist of Ambi Robotics and Jacobi Robotics. Ken leads research in robotics and automation: grasping, manipulation, and learning for applications in warehouses, industry, homes, agriculture, and robot-assisted surgery. He is Professor of IEOR with appointments in EECS and Art Practice. Ken is Chair of the Berkeley AI Research (BAIR) Steering Committee (60 faculty) and is co-founder and Editor-in-Chief emeritus of the IEEE Transactions on Automation Science and Engineering (T-ASE). He has published ten US patents, over 400 refereed papers, and presented over 600 invited lectures to academic and corporate audiences.

Open Problem Session, Starting with 10 Problems in Geobotics

Dan Halperin, Tel Aviv University

Can we optimize the coordinated motion of a fleet of robots, when even for two robots we know so little? Can we quickly decide if one object could cage another? Is there an effective way to explain why we failed to find an assembly plan for a new product design? We review these and other challenging problems at the intersection of robotics and computational geometry---let's call this intersection Geobotics. What is common to most of these problems is that the prevalent algorithmic techniques used in robotics do not seem suitable for solving them, or at least do not suggest quality guarantees for the solution. Solving some of them, even partially, can shed light on less well-understood aspects of computation in robotics.

Joint work with Mikkel Abrahamsen.

About the Speaker

Dan Halperin received his Ph.D. in Computer Science from Tel Aviv University, after which he spent three years at the Computer Science Robotics Laboratory at Stanford University. He then joined the Department of Computer Science at Tel Aviv University, where he is currently a full professor and for two years was the department chair. Halperin’s main field of research is Computational Geometry and Its Applications. Application areas he is interested in include robotics, automated manufacturing, algorithmic motion planning, and 3D printing. A major focus of Halperin's work has been in research and development of robust geometric software, in collaboration with a group of European universities and research institutes: the CGAL project and library, which recently won the SoCG test of time award. Halperin was the program-committee chair/co-chair of several conferences in computational geometry, algorithms and robotics, including SoCG, WAFR, ESA, and ALENEX. Halperin is an ACM Fellow and an IEEE Fellow.

What Belongs in a Robot's Brain?

Steven M. LaValle, University of Oulu & UIUC

Imagine building a robot to accomplish one or more tasks, such as vacuuming, patrolling, or exploration. This talk considers an egocentric or situated view of theoretical robot development that takes into account its space of possible environments and specific tasks. How much does a robot need to sense and remember to successfully interact with its environment? This question is fundamental to robotics and distinguishes it from other fields such as computer science or control theory. If machine learning is the goal, then the question becomes what are the minimal, ideal structures that could possibly be learned? Thus, emphasis in this talk is placed on determining the minimal amount of information necessary to solve tasks, thereby giving the robot the smallest possible "brain". At one extreme, strong geometric information is sensed and encoded, leading to problems such as classical motion planning. On the path to minimalism, weak geometric information is considered in the form of combinatorial or relational sensing and filtering. Eventually, topological and set-based information is considered at the minimalist extreme.

About the Speaker

Steven M. LaValle is Professor of Computer Science and Engineering, in Particular Robotics and Virtual Reality, at the University of Oulu. Since 2001 he has been a professor in the Department of Computer Science at the University of Illinois. He has also held positions at Stanford University and Iowa State University. His research interests include robotics, virtual and augmented reality, sensing, planning algorithms, computational geometry, and control theory. In research, he is mostly known for his introduction of the Rapidly exploring Random Tree (RRT) algorithm, which is widely used in robotics and other engineering fields. In industry, he was an early founder and chief scientist of Oculus VR, acquired by Facebook in 2014, where he developed patented tracking technology for consumer virtual reality and led a team of perceptual psychologists to provide principled approaches to virtual reality system calibration, health and safety, and the design of comfortable user experiences. From 2016 to 2017 he was Vice President and Chief Scientist of VR/AR/MR at Huawei Technologies, Ltd. He has authored the books Planning Algorithms, Sensing and Filtering, and Virtual Reality.