Software-Hardware Co-Design For Embodied AI Robots

2024-07-05

AI Summary

To address high latency and energy consumption in LLM-driven robotic manipulation—caused by frame-by-frame action generation—this paper proposes Corki, an algorithm-architecture co-design framework. Methodologically, Corki introduces (i) a novel trajectory-level LLM inference paradigm that predicts entire action trajectories instead of per-frame actions, drastically reducing inference frequency; (ii) hardware-level parallel execution of computation, low-level control, and data communication; and (iii) an end-to-end pipelined workflow enabled by software-hardware co-compiled optimizations. Evaluation demonstrates that Corki reduces LLM inference frequency by up to 8.0×, decreases end-to-end latency by 3.6×, and improves task success rate by 17.3%. The framework is fully open-sourced, enabling complete reproducibility.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate. In particular, today's computing systems for embodied AI robots are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame-basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes Corki, an algorithm-architecture co-design framework for real-time embodied AI robot control. Our idea is to decouple LLM inference, robotic control and data communication in the embodied AI robots compute pipeline. Instead of predicting action for one single frame, Corki predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. Corki largely reduces LLM inference frequency by up to 8.0x, resulting in up to 3.6x speed up. The success rate improvement can be up to 17.3%. Code is provided for re-implementation. https://github.com/hyy0613/Corki

Problem

• High latency and energy in AI robotic manipulation systems
• Disjointed execution pipeline in embodied AI robots
• Frequent LLM inference slowing robotic task performance

Innovation

• Algorithm-architecture co-design for robotic manipulation
• Decouples LLM inference, control, and data communication
• Predicts future trajectory to reduce LLM inference frequency