Chat-GPT designs working tomato-picking robot

Ripe tomatoes are very delicate, so they can’t be shaken from the plant to harvest and have always been collected by hand, like apples, nuts and olives.

Can a robot with a tender touch harvest tomatoes? With their ability to process vast amounts of text data and use this information to answer prompts, neural networks known as large language models (LLMs) like Chat-GPT have been making headlines for their potential to change the way we write, learn and even make art. Now, EPFL researchers have applied the technology to a new sphere – robotic design.

Researchers at the Delft University of Technology in the Netherlands used ChatGPT to design a working robotic tomato-harvester. They recently published their study under the title “How can LLMs transform the robotic design process?” in the prestigious Nature Machine Intelligence journal with Josie Hughes, head of the computational robot design and fabrication lab in the Swiss Federal Institute of Technology Lausanne (EPFL).

Delft doctoral student Francesco Stella and colleague Cosimo Della Santina used the artificial intelligence (AI) Chat-GPT chatbot that uses natural language processing to create humanlike conversational dialogue to design a working robotic tomato-harvester. The study provides a framework for humans and LLMs to design such devices collaboratively.

Based on their experience, the researchers describe opportunities and risks of applying AI tools to robotics, which they argue “could change the way we design robots, while enriching and simplifying the process."

“Even though ChatGPT is a language model and its code generation is text-based, it provided significant insights and intuition for physical design, and showed great potential as a sounding board to stimulate human creativity,” said Hughes.

The process of teaching AI to design robots

In the first phase, the researchers and LLM engaged in an ‘ideation’ discussion to define their robot’s purpose, design parameters and specifications. A second phase was devoted to building the robot, which involved refining the LLM-generated code, fabricating the device, and troubleshooting its functioning.

The researchers began at a high conceptual level, conversing with the LLM on future challenges to humanity, and identifying robotic crop harvesting as a solution to the challenge of global food supply. They then drew upon the LLM’s access to global data from academic publications, technical manuals, books and media to provide the “most probable” answer to prompts such as “What features should a robot harvester have?”

Once a basic robotic format as a motor-driven gripper for grasping ripe tomatoes was identified, the researchers could then pose more specific questions, like “What shape should the gripper have?” and ask the LLM to make technical suggestions including materials and computer code for controlling the device.

“While computation has been largely used to assist engineers with technical implementation, for the first time an AI system can ideate new systems, thus automating high-level cognitive tasks. This could involve a shift of human roles to more technical ones,” Stella explained.

“LLMs will fundamentally change the robotics landscape by providing robots with the unprecedented capability to understand and analyze natural language,” the authors wrote. “The key advantage of LLMs is their ability to process and internalize large amounts of text data such as instructions, technical manuals and academic articles, and to factually and coherently respond to questions using this implicit knowledge.”

More possibilities for human-AI cooperation

In addition to assigning ChatGPT the role of inventor, the researchers outlined other possible human-LLM collaboration modes in their paper. For example, “collaborative exploration” uses AI to augment researchers’ expertise by contributing wide-ranging knowledge beyond their own fields. AI can also act as a “funnel” helping to refine the design process and providing technical input, with humans retaining creative control.

As there are logical and ethical risks associated with each collaboration mode, the researchers caution that the role of LLMs must be carefully evaluated going forward. For example, the use of LLMs raises questions of bias, plagiarism and intellectual property, as it’s unclear whether an LLM-generated design can be considered unique.

“In our study, ChatGPT identified tomatoes as the crop ‘most worth’ pursuing for a robotic harvester. However, this may be biased towards crops that are more covered in literature, as opposed to those where there is truly a real need. When decisions are made outside the scope of knowledge of the engineer, this can lead to significant ethical, engineering, or factual errors,” Hughes added.

Despite these cautions, but based on their experience, Hughes and her team concluded that LLMs have great potential to be a force for good if well managed. “The robotics community must therefore identify how to leverage these powerful tools to accelerate the advancement of robots in an ethical, sustainable, and socially empowering way.”